The WeakSet is supposed to store elements by weak reference. That is, if an object is not referenced by anything else, it should be cleaned from the WeakSet.
I have written the following test:
var weakset = new WeakSet(),
numbers = [1, 2, 3];
weakset.add(numbers);
weakset.add({name: "Charlie"});
console.log(weakset);
numbers = undefined;
console.log(weakset);
Even though my [1, 2, 3] array is not referenced by anything, it's not being removed from the WeakSet. The console prints:
WeakSet {[1, 2, 3], Object {name: "Charlie"}}
WeakSet {[1, 2, 3], Object {name: "Charlie"}}
Why is that?
Plus, I have one more question. What is the point of adding objects to WeakSets directly, like this:
weakset.add({name: "Charlie"});
Are those Traceur's glitches or am I missing something?
And finally, what is the practical use of WeakSet if we cannot even iterate through it nor get the current size?
it's not being removed from the WeakSet. Why is that?
Most likely because the garbage collector has not yet run. However, you say you are using Traceur, so it just might be that they're not properly supported. I wonder how the console can show the contents of a WeakSet anyway.
What is the point of adding objects to WeakSets directly?
There is absolutely no point of adding object literals to WeakSets.
What is the practical use of WeakSet if we cannot even iterate through it nor get the current size?
All you can get is one bit of information: Is the object (or generically, value) contained in the set?
This can be useful in situations where you want to "tag" objects without actually mutating them (setting a property on them). Lots of algorithms contain some sort of "if x was already seen" condition (a JSON.stringify cycle detection might be a good example), and when you work with user-provided values the use of a Set/WeakSet would be advisable. The advantage of a WeakSet here is that its contents can be garbage-collected while your algorithm is still running, so it helps to reduce memory consumption (or even prevents leaks) when you are dealing with lots of data that is lazily (possibly even asynchronously) produced.
This is a really hard question. To be completely honest I had no idea in the context of JavaScript so I asked in esdiscuss and got a convincing answer from Domenic.
WeakSets are useful for security and validation reasons. If you want to be able to isolate a piece of JavaScript. They allow you to tag an object to indicate it belongs to a special set of object.
Let's say I have a class ApiRequest:
class ApiRequest {
constructor() {
// bring object to a consistent state, use platform code you have no direct access to
}
makeRequest() {
// do work
}
}
Now, I'm writing a JavaScript platform - my platform allows you to run JavaScript to make calls - to make those calls you need a ApiRequest - I only want you to make ApiRequests with the objects I give you so you can't bypass any constraints I have in place.
However, at the moment nothing is stopping you from doing:
ApiRequest.prototype.makeRequest.call(null, args); // make request as function
Object.create(ApiRequest.prototype).makeRequest(); // no initialization
function Foo(){}; Foo.prototype = ApiRequest.prototype; new Foo().makeRequest(); // no super
And so on, note that you can't keep a normal list or array of ApiRequest objects since that would prevent them from being garbage collected. Other than a closure, anything can be achieved with public methods like Object.getOwnPropertyNames or Object.getOwnSymbols. So you one up me and do:
const requests = new WeakSet();
class ApiRequest {
constructor() {
requests.add(this);
}
makeRequest() {
if(!request.has(this)) throw new Error("Invalid access");
// do work
}
}
Now, no matter what I do - I must hold a valid ApiRequest object to call the makeRequest method on it. This is impossible without a WeakMap/WeakSet.
So in short - WeakMaps are useful for writing platforms in JavaScript. Normally this sort of validation is done on the C++ side but adding these features will enable moving and making things in JavaScript.
(Of course, everything a WeakSet does a WeakMap that maps values to true can also do, but that's true for any map/set construct)
(Like Bergi's answer suggests, there is never a reason to add an object literal directly to a WeakMap or a WeakSet)
By definition, WeakSet has only three key functionalities
Weakly link an object into the set
Remove a link to an object from the set
Check if an object has already been linked to the set
Sounds more pretty familiar?
In some application, developers may need to implement a quick way to iterate through a series of data which is polluted by lots and lots of redundancy but you want to pick only ones which have not been processed before (unique). WeakSet could help you. See an example below:
var processedBag = new WeakSet();
var nextObject = getNext();
while (nextObject !== null){
// Check if already processed this similar object?
if (!processedBag.has(nextObject)){
// If not, process it and memorize
process(nextObject);
processedBag.add(nextObject);
}
nextObject = getNext();
}
One of the best data structure for application above is Bloom filter which is very good for a massive data size. However, you can apply the use of WeakSet for this purpose as well.
A "weak" set or map is useful when you need to keep an arbitrary collection of things but you don't want their presence in the collection from preventing those things from being garbage-collected if memory gets tight. (If garbage collection does occur, the "reaped" objects will silently disappear from the collection, so you can actually tell if they're gone.)
They are excellent, for example, for use as a look-aside cache: "have I already retrieved this record, recently?" Each time you retrieve something, put it into the map, knowing that the JavaScript garbage collector will be the one responsible for "trimming the list" for you, and that it will automatically do so in response to prevailing memory conditions (which you can't reasonably anticipate).
The only drawback is that these types are not "enumerable." You can't iterate over a list of entries – probably because this would likely "touch" those entries and so defeat the purpose. But, that's a small price to pay (and you could, if need be, "code around it").
WeakSet is a simplification of WeakMap for where your value is always going to be boolean true. It allows you to tag JavaScript objects so to only do something with them once or to maintain their state in respect to a certain process. In theory as it doesn't need to hold a value it should use a little less memory and perform slightly faster than WeakMap.
var [touch, untouch] = (() => {
var seen = new WeakSet();
return [
value => seen.has(value)) || (seen.add(value), !1),
value => !seen.has(value) || (seen.delete(value), !1)
];
})();
function convert(object) {
if(touch(object)) return;
extend(object, yunoprototype); // Made up.
};
function unconvert(object) {
if(untouch(object)) return;
del_props(object, Object.keys(yunoprototype)); // Never do this IRL.
};
Your console was probably incorrectly showing the contents due to the fact that the garbage collection did not take place yet. Therefore since the object wasn't garbage collected it would show the object still in weakset.
If you really want to see if a weakset still has a reference to a certain object then use the WeakSet.prototype.has() method. This method, as the name implies returns a boolean indicating wether the object still exists in the weakset.
Example:
var weakset = new WeakSet(),
numbers = [1, 2, 3];
weakset.add(numbers);
weakset.add({name: "Charlie"});
console.log(weakset.has(numbers));
numbers = undefined;
console.log(weakset.has(numbers));
Let me answer the first part, and try to avoid confusing you further.
The garbage collection of dereferenced objects is not observable! It would be a paradox, because you need an object reference to check if it exists in a map. But don't trust me on this, trust Kyle Simpson:
https://github.com/getify/You-Dont-Know-JS/blob/1st-ed/es6%20%26%20beyond/ch5.md#weakmaps
The problem with a lot of explanations I see here, is that they re-reference a variable to another object, or assign it a primitive value, and then check if the WeakMap contains that object or value as a key. Of course it doesn't! It never had that object/value as a key!
So the final piece to this puzzle: why does inspecting the WeakMap in a console still show all those objects there, even after you've removed all of your references to those objects? Because the console itself keeps persistent references to those Objects, for the purpose of being able to list all the keys in the WeakMap, because that is something that the WeakMap itself cannot do.
While I'm searching about use cases of Weakset I found these points:
"The WeakSet is weak, meaning references to objects in a WeakSet are held weakly.
If no other references to an object stored in the WeakSet exist, those objects can be garbage collected."
##################################
They are black boxes: we only get any data out of a WeakSet if we have both the WeakSet and a value.
##################################
Use Cases:
1 - to avoid bugs
2 - it can be very useful in general to avoid any object to be visited/setup twice
Refrence: https://esdiscuss.org/topic/actual-weakset-use-cases
3 - The contents of a WeakSet can be garbage collected.
4 - Possibility of lowering memory utilization.
Refrence: https://www.geeksforgeeks.org/what-is-the-use-of-a-weakset-object-in-javascript/
##################################
Example on Weakset: https://exploringjs.com/impatient-js/ch_weaksets.html
I Advice you to learn more about weak concept in JS: https://blog.logrocket.com/weakmap-weakset-understanding-javascript-weak-references/
Related
I'm currently developing a little game in Javascript and I'm using Codacy to review my code and help me cleaning it.
One of the most seen error is Generic Object Injection Sink (security/detect-object-injection).
It happens when I'm trying to access a value in an array using a variable. Like in this example :
function getValString(value)
{
var values = ["Misérable", "Acceptable", "Excellente", "Divine"];
return values[value];
}
This function is used to display on screen the value's string of an item. It receives a "value" which can be 0, 1, 2 or 3 and returns the string of the value.
Now here's my problem :
Codacy is telling me that use of var[var] should be prohibited because it causes security issues and since I'm rather new to Javascript, I was wondering why and what are the good practices in that kind of situation.
The security issue present here is that the stringified value of value may be accessing a property that is inherited from the Object's __proto__ hierarchical prototype, and not an actual property of the object itself.
For example, consider the scenario when value is a string literal of "constructor".
const property = "constructor";
const object = [];
const value = object[property];
The result of value in this context will resolve to the Array() function - which is inherited as part of the Object's prototype, not an actual property of the object variable. Furthermore, the object being accessed may have overridden any of the default inherited Object.prototype properties, potentially for malicious purposes.
This behavior can be partially prevented by doing a object.hasOwnProperty(property) conditional check to ensure the object actually has this property. For example:
const property = "constructor";
const object = [];
if (object.hasOwnProperty(property)) {
const value = object[property];
}
Note that if we suspect the object being accessed might be malicious or overridden the hasOwnProperty method, it may be necessary to use the Object hasOwnProperty inherited from the prototype directly: Object.prototype.hasOwnProperty.call(object, property)
Of course, this assumes that our Object.prototype has not already been tampered with.
This is not necessarily the full picture, but it does demonstrate a point.
Check out the following resources which elaborates in more detail why this is an issue and some alternative solutions:
https://github.com/nodesecurity/eslint-plugin-security/blob/master/docs/the-dangers-of-square-bracket-notation.md
Securely set unknown property (mitigate square bracket object injection attacks) utility function
By itself this is not a bad practice, because you do want to develop a system and make it secure. It's difficult to imagine a higher security risk to a system than one which causes the nonexistence of that system.
Yet, not being allowed to use a variable to dynamically create/use/update an index practically reduces your options of hard-coding any indexes that you may use to refer items of an array or members of an object.
Not allowing indexes greatly reduces your options, so much that it threatens with the nonexistence of any system that you may want to create in Javascript. Let's see some of the use-cases:
Numbered loops:
for (let index = 0; index < arr.length; index++) {
//do whatever with arr[index]
}
Of course, this is true to while loops as well.
in loops
for (let index in variable) {
//do whatever with arr[index]
}
of loops
for (let item of variable) {
// do whatever with item
}
see
finding dynamically a value
This is virtually used in quasi infinitely many ways, all the examples above are specific cases of this. Example:
function getItem(arr, index) {
return arr[index];
}
summary
The fear of exploits because of dynamic indexing is the equivalent of the fear from a meteor hitting into the exact place and the exact time one is in. Of course, we cannot exclude it, but one can not live in constant fear of low-chanced catastrophes. Similarly, programming is also impossible with unreasonable, paranoid fears. So, instead of rejecting dynamic indexing altogether because of the possibility of the exploits, we must refer to the actual exploits that could be possible. If we are not allowed to use dynamic instances, then whatever system we are to develop, if it's not simple as pie, will not come to existence. So, whatever threats we are afraid of should be protected against otherwise.
Example: You retrieve values from a data-source and have a field for credit card IBAN. Yeah, if that's shown to a user who is not the owner, that's a high risk. But you should protect against this by making IBAN unavailable by the mere of a use of an index by external sources, such as POST requests sent by the browser of a user.
Attending to it's specification, JSON elements (and javascript objects) are unordered so, even in almost all cases, when you iterate over a javascript object, you get elements in the same order they was defined; you definitively cannot trust in that order because engine is allowed to alter it.
This is extremely rare. I have been able to observe it one time, but I don't find that code right now and I don't remember the exact version of JS engine (but it was node). If I manage to find it, I will add it to this post.
That being said, the point is that code relying in this behaviour can (and should) be considered buggy because, even it will work as expected in most engines, it may fail typically because of internal engine optimisations.
For example:
"use strict";
var x = {
b: 23,
a: 7
};
function someSorting(input) {
var output = {};
Object.keys(input).sort().map(
function(k){
output[k] = input[k];
}
);
return output;
};
x = someSorting(x);
// Some smart engine could notice that input and output objects have the
// exact same properties and guess that, attending the unordered nature of
// javascript Object, there is no need to actually execute someSorting();
console.log(x);
// Usually will display: { a: 7, b: 23 }
// But perfectly we could got: { b: 23, a: 7 }
I know there is too many literature (even StackOverflow questions) about this (NON-) issue and "workarrounds" to achieve the expected behaviour by sorting keys in a separate array.
But doing so code goes too messy compared in simply trusting in key order.
I'm pretty sure that this can be achieved in a more elegant fashion by implementing a so-called "sObject" alternative having native Object as its prototype but overloading it's native iterator and setter so that:
When any new property is added, it's key is appended to an Array index mantained under the hood.
When an sObject instance is iterated, our customized iterator uses that index to retrieve elements in the right order.
In summary: Actual Object specification is right because, in most cases, properties order doesn't care. So I think that engine optimisations that could mess it are wonderfull.
But it would be also wonderful to have an alternative sObject with which we could do something like:
var x = new sObject({b: 23, a: 7});
...and trust that we could iterate it in the same exact order or, also / at least, do some sorting task over it and trust that this will not be altered.
Of course!! I'm initalyzing it with a native javascript Object so, in fact, theoretically we can't trust that it will be populated right (even I can't imagine why any engine optimisation should alter it before any operation).
I used that notation for brevity (and, I confess) because I expect that, in that case should work always (even I'm not really sure). However we even could sort it later (which, in most cases we will do that way) or use other kind of initialization like providing a JSON string or an array of objects (or arrays) with single key and value pairs.
My concern is: Such a thing exists yet? I wasn't able to find it. But sure I'm not the first guy thinking in that...
I can try to implement it (I'm thinking about that). I think it's possible and that I could achieve it. But it's not as simple so first I want to be sure that I'm not reinventing the wheel...
So any comments, suggestions, etc... will be welcome.
Sure, you could do all this. You will need some machinery such as Object.observer, which is currently only available in Chrome. We can define this as the following:
function myObject(object) {
// if the object already has keys, bring them in in whatever order.
var keys = Object.keys(object);
// Override Object.keys to return our list of keys.
Object.defineProperty(object, 'keys', { get: function() { return keys; });
// Watch the object for new or deleted properties.
// Add new ones at the end, to preserve order.
Object.observe(object, function(changes) {
changes.forEach(function(change) {
if (change.type === 'add') keys.push(change.name);
if (change.type === 'delete') keys = keys.filter(function(key) {
return key === change.name;
});
});
});
return object;
}
Note that Object.observe is asynchronous, so even after you add a property to the object, it won't be reflected in the custom keys property until after a tick of the clock, although in theory you could use Object.deliverChangedRecords.
The above approach uses a function which adds the new ordered key functionality to an existing object. Of course there are other ways to design this.
This "solution" obviously cannot control the behavior of for...in loops.
I am working through a memory issue with one of our webapps. I am using Chrome's heap profiler. I want to make sure I understand something very explicitely, as I'm making assumptions on this information.
The # symbol in the heap profile screenshot above. I want to make sure I understand crystal clear: equal object ids implies the same object
a.objectId == b.objectId implies a same as b
a.objectId == b.objectId implies NOT a same as b
Therefore if I have two objects that I expected to actually be the same thing, yet their object id differs, this implies an unexpected copy occurred? This implies I can go and figure out in my code where I might be creating unnecessary duplicates?
The documentation appears to say this, but they don't quite say it explicitly, going on to say why they have an object id, not what it represents.
This is an object ID. Displaying an object's address makes no sense, as objects are moved during garbage collections. Those object IDs are real IDs — that means, they persist among multiple snapshots taken and are unique. This allows precise comparison between heap states. Maintaining those IDs adds an overhead to GC cycles, but it is only initiated after the first heap snapshot was taken — no overhead if heap profiles aren't used.
I get that. But I need to fit this back into my C programmer head. I realize, even with native heaps, pointer values can change over time. Can I effectively treat object ids as pointer addresses unique over time?
So I ran some test code in Chrome and the answer appears to be yes, the same object id implies identical objects. If object id differs, this implies a copy or different object altogether.
I profiled the following snippet of code which can be found in this github repo:
(function heapTest(doc) {
'use strict';
function clone(obj) {
return JSON.parse(JSON.stringify(obj));
}
var b = {'grandchild-key-2': 5};
var a = {'child-key-1': b};
doc.child1 = a;
doc.child1_again = a;
doc.child1_copy = clone(a);
})(document);
The heap profiler confirms the two references share object ids, the copy receives a new object id.
In short, this behaves like I expect. Multiple references to the same object receive the same object id. Copies refer to a different object and receive a different object id.
In Kyle Simpson's new title, You don't know JS: ES6 and beyond, I find the following snippet:
WARNING Assigning an object or array as a constant means that value will not be able to be garbage collected until that constant’s lexical scope goes away, as the reference to the value can never be unset. That may be desirable, but be careful if it’s not your intent!
(Excerpt From: Simpson, Kyle. “You Don’t Know JS: ES6 & Beyond.” O'Reilly Media, Inc., 2015-06-02. iBooks.
This material may be protected by copyright.)
As far as I can see, he doesn't expand on this, and 10 minutes on Google turns up nothing. Is this true, and if so, what does "the reference to the value can never be unset" mean exactly? I have got into the habit of declaring variables that won't be changed as const, is this a bad habit in real concrete performance/memory terms?
WARNING Assigning an object or array as a constant means that value
will not be able to be garbage collected until that constant’s lexical
scope goes away, as the reference to the value can never be unset.
That may be desirable, but be careful if it’s not your intent!
That note sounds a bit more of a warning than is necessary (perhaps even a bit silly) and tries to make some sort of special case out of this situation.
With a const variable declaration, you can't assign to the variable something little like "" or null to clear its contents. That's really the only difference in regard to memory management. Automatic garbage collection is not affected at all by whether it is declared const or not.
So, if you would like to be able to change the contents of the variable in the future for any reason (including to manually remove a reference to something to allow something to be garbage collected sooner), then don't use const. This is the same as any other reason for using or not using const. If you want to be able to change what the variable contains at any time in the future (for any reason), then don't use const. This should be completely obvious to anyone who understand what const is for.
Calling out garbage collection as a special case for when not to use const just seems silly to me. If you want to be able to clear the contents of a variable, then that means you want to modify the variable so duh, don't use const. Yes, manually enabling garbage collection on a large data structure that might be caught in a lasting scope/closure is one reason that you might want to change the variable in the future. But, it's just one of millions of reasons. So, I repeat one more time. If you ever want to change the contents of the variable for any reason in the future, then don't declare it as const.
The garbage collector itself doesn't treat a const variable or the contents it points to any different than a var or let variable. When it goes out of scope and is no longer reachable, its contents will be eligible for garbage collection.
const has a number of advantages. It allows the developer to state some intent that the contents this variable points to are not to be changed by code and may allow the runtime to make some optimizations because it knows the contents of the variable cannot be changed. And, it prevents rogue or accidental code from ever changing the contents of that variable. These are all good things when used in an appropriate case. In general, you SHOULD use const as much as practical.
I should add the even some const data can still be reduced in size and make the majority of its contents available for garbage collection. For example, if you had a really large 100,000 element array of objects (that you perhaps received from some external http call) in a const array:
const bigData = [really large number of objects from some API call];
You can still massively reduce the size of that data by simply clearing the array which potentially makes the large number of objects that was in the array eligible for garbage collection if nothing else had a reference to them:
bigData.length = 0;
Remember, that const prevents assignment to that variable name, but does not prevent mutating the contents that the variable points to.
You could do the same thing with other built-in collection types such as map.clear() or set.clear() or even any custom object/class that has methods for reducing its memory usage.
That note in my book was referring to cases like this, where you'd like to be able to manually make a value GC'able earlier than the end of life of its parent scope:
var cool = (function(){
var someCoolNumbers = [2,4,6,8,....1E7]; // a big array
function printCoolNumber(idx) {
console.log( someCoolNumbers[idx] );
}
function allDone() {
someCoolNumbers = null;
}
return {
printCoolNumber: printCoolNumber,
allDone: allDone
};
})();
cool.printCoolNumber( 10 ); // 22
cool.allDone();
The purpose of the allDone() function in this silly example is to point out that there are times when you can decide you are done with a large data structure (array, object), even though the surrounding scope/behavior may live on (via closure) indefinitely in the app. To allow the GC to pick up that array and reclaim its memory, you unset the reference with someCoolNumbers = null.
If you had declared const someCoolNumbers = [...]; then you would be unable to do so, so that memory would remain used until the parent scope (via the closure that the methods on cool have) goes away when cool is unset or itself GCd.
Update
To make absolutely clear, because there's a lot of confusion/argument in some comment threads here, this is my point:
const absolutely, positively, undeniably has an effect on GC -- specifically, the ability of a value to be GCd manually at an earlier time. If the value is referenced via a const declaration, you cannot unset that reference, which means you cannot get the value GCd earlier. The value will only be able to be GCd when the scope is torn down.
If you'd like to be able to manually make a value eligible for GC earlier, while the parent scope is still surviving, you'll have to be able to unset your reference to that value, and you cannot do that if you used a const.
Some seem to have believed that my claim was const prevents any GC ever. That was never my claim. Only that it prevented earlier manual GC.
No, there are no performance implications. This note refers to the practise of helping the garbage collector (which is rarely enough needed) by "unsetting" the variable:
{
let x = makeHeavyObject();
window.onclick = function() {
// this *might* close over `x` even when it doesn't need it
};
x = null; // so we better clear it
}
This is obviously not possibly to do if you had declared x as a const.
The lifetime of the variable (when it goes out of scope) is not affected by this. But if the garbage collector screws up, a constant will always hold the value it was initialised with, and prevent that from being garbage-collected as well, while a normal variable might no more hold it.
The way garbage collectors (GC) work is when something is referenced by nothing ("cannot be reached"), the GC can safely say that something isn't used anymore and reclaim the memory used by that something.
Being able to replace the value of a variable allows one to remove a reference to the value. However, unlike var, const cannot be reassigned a value. Thus, one can't remove that constant from referencing the value.
A constant, like a variable, can be reclaimed when the constant goes "out of scope", like when a function exits, and nothing inside it forms a closure.
What is a memory efficient way to link two objects? If you store objects using two arrays with corresponding index values the object won't be released from memory.
The implementation should look like the following.
var obj ={};
var linkedobj = getLinkedObject(obj);
var obj2 ={}
var linkedobj2 = getLinkedObject(obj2);
Objects are general containers
If all you want to do is associate one object with another one, then why not have them point to one another?
var obj = {};
var obj2 = {};
obj.linkedObject = obj2;
obj2.linkedObject = obj;
This would a normal thing to do, and doesn't have bad implications on memory.
Releasing memory
If you are asking about holding a table full of object references, you are correct that the object will not disappear until all of the live references to it are gone. Javascript has a garbage collector, and as long as your table or array has a good reference, it will be backed by memory for you. If you give an object to a table and then want it to be deleted from memory entirely, you can simply remove it from your table also. You could also just remove the table, assuming it doesn't need to hold anything else.
If you are determined to have a getLinkedObject function return an object for another object, you can still allow the objects to each hold the associate reference.
function getLinkedObject(anObject) {
return anObject.linkedObject;
}
This would probably be accompanied by a counterpart:
function linkObjects(anObject, anotherObject) {
anObject.linkedObject = anotherObject;
anotherObject.linkedObject = anObject;
}
Doing this allows you to worry a little less about memory management, which in Javascript (a high-level, dynamic environment with a garbage collector) is typically appropriate.
Unless you use keparo's answer, which from your follow-up comment sounds like not an option, what you're asking for is basically impossible. Because, since you can't augment the original object, you need some method of storing a reference to it. But (since JavaScript has no concept of weak references) this will always prevent it from being garbage collected, contradicting your requirements statement.