How can I apply timed back pressure in RxJS5? - javascript

Imagine I have the following code:
let a = Rx.Observable.of(1, 2, 3)
let b = Observable.zip(a, a, (a, b) => a + b)
b.forEach(t => console.log(t))
This immediately outputs the results. Now, how do I put a timed delay between each message as a way of back-pressure (note that I don't want a buffer; instead, I want a and b to become Cold Observables), like:
b.takeEvery(1000).forEach(t => console.log(t))
And have the exact same answer:
<wait 1s>
2
<wait 1s>
4
<wait 1s>
6
Alternative: If backpressure (ou pull mechanisms for some observables) is something not supported in RxJS, then how could one create an infinite generator without running out of resources?
Alternative 2: Other JS frameworks that support both pull and push mechanisms?

In case of RxJS 5.x back pressure is not support, but there is for example pausable operator in 4.x version. It works only with hot observables. More info on back pressure in case of 4.x and here (especially take a loot at the bottom and RxJS related description).
This Erik Meijer's tweet may be bit controversial but relevant: https://twitter.com/headinthebox/status/774635475071934464
For your own implementation of back pressure mechanism you need to have 2-way communication channel, which can be fairly easily created with 2 subjects - one for each end. Basically use next for sending messages and .subscribe for listing to the other end.
Creating a generator is doable as well - again using a subject to bridge between push- and pull-based worlds. Below an exemplary implementation for generating Fibonacci numbers.
const fib = () => {
const n = new Rx.Subject()
const f = n
.scan(c => ({ a: c.b, b: c.b + c.a }), { a: 0, b: 1 })
.map(c => c.a)
return {
$: f,
next: () => n.next()
}
}
const f = fib()
f.$.subscribe(n => document.querySelector('#r').innerHTML = n)
Rx.Observable.fromEvent(document.querySelector('#f'), 'click')
.do(f.next)
.subscribe()
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.6/Rx.js"></script>
<button id='f'>NEXT FIBONACCI</button>
<div id='r'>_?_<div>
Another js library which may be of interest for you is https://github.com/ubolonton/js-csp - did not use it, so not sure how it deals with back pressure.

the idea is to queue the time wait one after the other when the previous one finishes execution Fiddle
let a = Rx.Observable.of(1, 2, 3);
let b = Rx.Observable.zip(a, a, (a, b) => a + b);
// getting values into array
var x = [];
b.forEach(t => x.push(t));
var takeEvery = function(msec,items,action,index=0){
if(typeof(action) == "function")
if(index<items.length)
setTimeout(
function(item,ind){
action(item);
takeEvery(msec,items,action,ind);
},msec, items[index],++index);
};
// queueing over time
takeEvery(1000,x, function(item){
console.log(item);
});

Related

crockjs and apply functions need a function as value

I'm actually learning functional programming, and I'm trying to learn & use crockjs
For now, I'm trying to implement the "monadster" program described in https://fsharpforfunandprofit.com/monadster/ .
Here's what I'm having for now (just the beginning...)
const State = require("crocks/State");
const LivingPart = (unitOfForce, deadThing) => ({ unitOfForce, deadThing });
// Creating the potential living thing
const makeLiveThingM = deadThing => {
const becomeAlive = vitalForce => {
const unitOfForce = 1;
const remaining = vitalForce - unitOfForce;
return { part: LivingPart(unitOfForce, deadThing), remaining };
};
return State.get(becomeAlive);
};
// Using containers
const deadLegM = makeLiveThingM("deadLeg");
const deadArmM = makeLiveThingM("deadArm");
const livingThings = deadLegM.ap(deadArmM).evalWith(1);
console.log(livingThings);
My problem is that it throws the following error:
/Users/pc/Soft/experiments/functional/crocks/node_modules/crocks/State/index.js:101
throw new TypeError('State.ap: Source value must be a function')
^
TypeError: State.ap: Source value must be a function
From what I see there, it's probably because I don't understand the apply function, or the way State.get is running. For me it accepts a function as its internal value in my code, but it doesn't seem so.
Can anybody explains and show me what I'm doing wrong here ?
Thanks for your help
Welcome to functional programming in JS and thank you for giving crocks a shot.
In looking at that article, one of the things to note is that the author is presenting how the mechanics work inside of the State ADT, and not really how to use an existing State ADT.
I will provide an explanation on how to handle the State transactions manually, which is close to what you have in your implementation. Then I will give a brief example of how the construction helpers (like get and modify could be used to decrement the VitalForce) are used to handle the and build State transactions.
Also I will give a brief explanation of using Applicatives.
So to start lets bring in a couple ADTs from crocks
const Pair = require('crocks/Pair')
const State = require('crocks/State')
We need the State constructor to take in a function that will return a Pair (the tuple that the author mentions in the post). How the construction function works can be found here.
Before we can discuss the State function, we need that LivingPart function:
// LivingPart :: (Integer, String) -> Object
const LivingPart = (unitOfForce, part) =>
({ [part]: { unitOfForce } })
I have changed the structure from what you originally had so we could merge any given Part together with another.
Now with that bit in the mix we can implement makeLiveThing. You pretty much had it in your implementation. The only real difference here is we need to construct the State ADT with the function and return the Pair. Notice that we still bring in the String, BUT return a State ADT that will be executed when runWith is called. Remember that the current state will be passed into the function that the State instance wraps (in this case, vitalForce is our state):
// makeLiveThing :: String -> State Integer Object
const makeLiveThing = part => State(
vitalForce => {
const unitOfForce = 1
const remaining = vitalForce - unitOfForce
return Pair(
LivingPart(unitOfForce, part),
remaining
)
}
)
Now that we have a means to create a LivingPart and handle our state transaction (decrementing by 1), we can create a couple parts:
// rightLeg :: State Integer Object
const rightLeg =
makeLiveThing('right-leg')
// leftArm :: State Integer Object
const leftArm =
makeLiveThing('left-arm')
Now comes the task of joining these State instances. You were right to think to use apply, as when an ADT has both an ap method and an of method it is called an Applicative. When we have an Applicative we can think of the type as being able to combine (2) independent instances that do not depend on the result of the other one. We just need to provide a way to tell the type how to combine it.
Typically that is done with a function that can act on the types contained in the ADT. In our case it is an Object, so one way to combine (2) objects is with Object.assign. crocks provides a helper called assign that can do just that, so lets bring it in:
const assign = require('crocks/helpers/assign')
Now that we have a way to combine the internal values, we need to "lift" this function into our State type, crocks also has a function that can be used on Applicatives to lift and apply the internal values of ADT (2) instances called liftA2. Which means "lift a function into an applicative with 2 instances"
So lets bring that in as well and then create a function that will be used to join (2) Parts:
const liftA2 = require('crocks/helpers/liftA2')
// joinParts :: Applicative m => m Object -> m Object -> m Object
const joinParts =
liftA2(assign)
Now with this function we can lift and join those parts and run the result with our VitalForce:
joinParts(rightLeg, leftArm)
.runWith(10)
//=> Pair( { left-arm: { unitOfForce: 1 }, right-leg: { unitOfForce: 1 } }, 8 )
Notice the result has the resultant in the left (the combined living parts) and the state in the right (the remaining VitalForce).
Here are some references to that above:
assign function
liftA2 function
State ADT
Pair ADT
egghead course on the State API
Now I am going to show a brief example of how we can set up a single State transaction for taking VitalForce from our pool. I am not going to explain here in detail, but you should be able to glean some information between this example and the State documentation:
const State = require('crocks/State')
const constant = require('crocks/combinators/constant')
const mapProps = require('crocks/helpers/mapProps')
const { modify } = State
// decrementBy :: Integer -> Integer -> Integer
const decrementBy =
x => y => y - x
// VitalForce :: { units: Integer }
// decUnitsBy :: Integer -> VitalForce -> VitalForce
const decUnitsBy = units =>
mapProps({ units: decrementBy(units) })
// getVitalForce :: Integer -> State VitalForce VitalForce
const getVitalForce = units =>
modify(decUnitsBy(units))
.map(constant({ units }))
getVitalForce(3)
.runWith({ units: 10 })
//=> Pair( { units: 3 }, { units: 7 } )
Here are some docs for those included functions:
constant function
mapProps function
So as a side note, I do a LiveCode broadcast on this channel, I am going to go over this blog post and talk about how to implement this in crocks, if that is something you would be interested in.
Hope this helps!!

Creating millions of Objects in Javascript

Let me be the first to say that this isn't something I normally do, but out of curiousity, I'll see if anyone has a good idea on how to handle a problem like this.
The application I am working on is a simulated example of the game Let's make a Deal featuring the Monty Hall problem.
I won't go into details about my implementation, but it more or less allows a user to enter a number of how many games they want to simulate, and then if an option is toggled off, the player of those x games won't switch their choice, while if it is toggled on, they will switch their choice every single instance of the game.
My object generator looks like this:
const game = function(){
this[0] = null;
this[1] = null;
this[2] = null;
this.pick = Math.floor(Math.random() * 3);
this.correctpick = Math.floor(Math.random() * 3);
this[this.correctpick] = 1;
for (let i=0; i<3; i++){
if ((this[i] !== 1) && (i !== this.pick)){
this.eliminated = i;
break;
}
}
}
const games = arg => {
let ret = [];
for(let i=0; i<arg; i++){
ret.push(new game);
}
return ret;
}
This structure generates an array which i stringify later that looks like this:
[
{
"0": 1,
"1": null,
"2": null,
"pick": 2,
"correctpick": 0,
"eliminated": 1
},
{
"0": null,
"1": null,
"2": 1,
"pick": 2,
"correctpick": 2,
"eliminated": 0
}
]
As sloppy as the constructor for game looks, the reason is because I have refactored it into having as few function calls as possible, where now I'm literally only calling Math functions at the current time (I removed any helper functions that made the code easier to read, in opt for performance).
This app can be ran both in the browser and in node (cross platform), but I have clamped the arg a user can pass into the games function to 5 million. Any longer than that and the process (or window) freezes for longer than a few seconds, or even potentially crashes.
Is there anything else I can do to increase performance if a huge number is given by a user? Also, if you need more information, I will be happy to supply it!
Thanks!
The obvious performance optimisation would be not to create and store 5 million objects at all, relieving memory pressure. Instead you'd create the objects on the fly only when you need them and throw them away immediately after. I'm not sure what your app does, but it sounds like you want to re-use the same game instances when evaluating results with the different options. In that case, you need to store them of course - but I'd advise to re-think the design and consider immediately evaluating each game with all possible options, accumulating only the results for each choice of options but not keeping all games in memory.
Apart from that, I'd recommend to simplify a bit:
You can drop that loop completely and use some clever arithmetic to find the eliminated option: this.eliminated = this.pick == this.correctpick ? +!this.pick : 3 - this.pick - this.correctpick;. Or use a simple lookup table this.eliminated = [1, 2, 1, 2, 0, 0, 1, 0, 0][this.pick * 3 + this.correctpick].
I'd avoid changing the type of the array elements from null (reference) to 1 (number). Just keep them as integers and initialise your elements with 0 instead.
Don't store 6 properties in your object that are completely redundant. You only need 2 of them: pick and correctpick - everything else can be computed on the fly from them when you need it. Precomputing and storing it would only be advantageous if the computation was heavy and the result was used often. Neither of this is the case, but keeping a low memory footprint is important (However, don't expect much from this).
Not sure about your implementation, but do you really need an Array?
How about only using results (see snippet)?
If it's blocking the browser that worries you, maybe delegating the work to a web worker is the solution for that: see this jsFiddle for a web worker version of this snippet.
(() => {
document.querySelector("#doit")
.addEventListener("click", playMontyHall().handleRequest);
function playMontyHall() {
const result = document.querySelector("#result");
const timing = document.querySelector("#timing");
const nOfGames = document.querySelector("#nGames");
const switchDoors = document.querySelector("#switchyn");
// Create a game
const game = (doSwitch) => {
const doors = [0, 1, 2];
const pick = Math.floor(Math.random() * 3);
const correctPick = Math.floor(Math.random() * 3);
const eliminated = doors.filter(v => v !== pick && v !== correctPick)[0];
return {
correctpick: correctPick,
pick: doSwitch ? doors.filter(v => v !== pick && v !== eliminated)[0] : pick,
eliminated: eliminated,
};
};
const getWinner = game => ~~(game.correctpick === game.pick);
// Sum wins using a generator function
const winningGenerator = function* (doSwitch, n) {
let wins = 0;
while (n--) {
wins += getWinner(game(doSwitch));
yield wins;
}
};
// calculate the number of succeeded games
const calculateGames = (nGames, switchAlways) => {
const funNGames = winningGenerator(switchAlways, nGames);
let numberOfWins = 0;
while (nGames--) {
numberOfWins = funNGames.next().value;
}
return numberOfWins;
}
const cleanUp = playOut => {
result.textContent =
"Playing ... (it may last a few seconds)";
timing.textContent = "";
setTimeout(playOut, 0);
};
const report = results => {
timing.textContent = `This took ${
(performance.now() - results.startTime).toFixed(3)} milliseconds`;
result.innerHTML =
`<b>${!results.switchAlways ? "Never s" : "Always s"}witching doors</b>:
${results.winners} winners out of ${results.nGames} games
(${((results.winners/+results.nGames)*100).toFixed(2)}%)`;
};
// (public) handle button click
function clickHandle() {
cleanUp(() => {
const nGames = nOfGames.value || 5000000;
const switchAlways = switchDoors.checked;
report({
switchAlways: switchAlways,
startTime: performance.now(),
winners: calculateGames(nGames, switchAlways),
nGames: nGames
});
});
}
return {
handleRequest: clickHandle
};
}
})();
body {
margin: 2em;
font: normal 12px/15px verdana, arial;
}
#timing {
color: red;
}
<p><input type="number" id="nGames" value="5000000"> N of games</p>
<p><input type="checkbox" id="switchyn"> Always switch doors</p>
<p><button id="doit">Play</button>
<p id="result"></p>
<p id="timing"></p>

Observable delay after long processing of each element of large array

Say we have a big array and processing of each element in that array takes a long time (5s). We want to add some delay (2s) before processing of next element.
I've somewhat managed to achieve that behavior like that:
let arr = [1, 2, 3]
let i = 0
Rx.Observable.of(null)
.map(val => arr[i++])
.do(val => {
console.log(val + ' preloop')
let time = +new Date() + 5000; while (+new Date() < time) {}
console.log(val + ' afterloop')
})
.delay(2000)
.flatMap(val => (i < arr.length) ? Rx.Observable.throw(null) : Rx.Observable.empty())
.retry()
.subscribe(console.log, console.log, () => console.log('completed'))
The output is as expected:
1 preloop
delay 5s
1 afterloop
delay 2s
2 preloop
...
completed
But this code is ugly, not reusable and buggy and doesn't comply with the philosophy of rx. What is the better way?
Note that the array (or it might even be not an array at all) is big and https://stackoverflow.com/a/21663671/2277240 won't work here.
The question is hypothetycal though I can think of some use cases.
I'm not sure why the fact that the array is big important here, and you can solve your issue using the same method from the link you provided.
For your long operation a better practice would probably be a promise. I used an async sleep function to simulate a 5 second operation, you can replace it with your promise.
The trick for the extra delay, is to concat a dummy element, delay it and then ignore it.
const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
Rx.Observable.from([1, 2, 3])
.concatMap(item =>
Rx.Observable.defer(() => sleep(5000))
.mapTo(item)
.concat(Rx.Observable.of(null).delay(2000).ignoreElements())
)
.subscribe(console.log, console.log, () => console.log('completed'))
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.2/Rx.js"></script>
An alternative for delay which does not delay the first emit,
source
.first()
.merge(
source.skip(1).delay(2000)
)

Not truly async?

I have a array of about 18 000 elements. I'm creating a map application where I want to add the elements when the user zooms in to a certain level.
So when the user zooms in under 9 I loop tru the array looking for elements that is in the view.
However, it does take some time looping thru the elements, causing the map application lag each time the user zooms out and in of "level 9". Even if there are no elements to add or not, so the bottleneck is the looping I guess.
I've tried to solve it by asyncing it like:
function SearchElements(elementArr) {
var ret = new Promise(resolve => {
var arr = [];
for (var i in elementArr) {
var distanceFromCenter = getDistanceFromLatLonInKm(view.center.latitude, view.center.longitude, dynamicsEntities[i].pss_latitude, dynamicsEntities[i].pss_longitude);
var viewWidthInKm = getSceneWidthInKm(view, true);
if (distanceFromCenter > viewWidthInKm) continue;
arr.push(elementArr[i]);
}
resolve(arr);
});
return ret;
}
SearchElements(myElementsArray).Then(arr => {
// ...
});
But its still not async, this method hangs while the for loop runs.
Because you still have a tight loop that loops through all the elements in one loop, you'll always have the responsiveness issues
One way to tackle the issue is to works on chunks of the data
Note: I'm assuming elementArr is a javascript Array
function SearchElements(elementArr) {
var sliceLength = 100; // how many elements to work on at a time
var totalLength = elementArr.length;
var slices = ((totalLength + sliceLength - 1) / sliceLength) | 0; // integer
return Array.from({length:slices})
.reduce((promise, unused, outerIndex) =>
promise.then(results =>
Promise.resolve(elementArr.slice(outerIndex * sliceLength, sliceLength).map((item, innerIndex) => {
const i = outerIndex * sliceLength + innerIndex;
const distanceFromCenter = getDistanceFromLatLonInKm(view.center.latitude, view.center.longitude, dynamicsEntities[i].pss_latitude, dynamicsEntities[i].pss_longitude);
const viewWidthInKm = getSceneWidthInKm(view, true);
if (distanceFromCenter <= viewWidthInKm) {
return item; // this is like your `push`
}
// if distanceFromCenter > viewWidthInKm, return value will be `undefined`, filtered out later - this is like your `continue`
})).then(batch => results.concat(batch)) // concatenate to results
), Promise.resolve([]))
.then(results => results.filter(v => v !== undefined)); // filter out the "undefined"
}
use:
SearchElements(yourDataArray).then(results => {
// all results available here
});
My other suggestion in the comment was Web Workers (I originally called it worker threads, not sure where I got that term from) - I'm not familiar enough with Web Workers to offer a solution, however https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers and https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API should get you going
To be honest, I think this sort of heavy task would be better suited to Web Workers

How to use pointfree style in JavaScript without losing readability?

When I tried to write JavaScript in pointfree style, I found that if you forced every function in this style, you sometimes lost its readabilty. For example:
import R from 'ramda'
const ceil = Math.ceil
const pagination = {
total: 101,
itemsPerPage: 10,
currentPage: 1
}
// ================= Pointful style ==================
const pageCount = (pagination) => {
const pages = ceil(pagination.total / pagination.itemsPerPage)
const remainPages = pagination.total % pagination.itemsPerPage === 0 ? 0 : 1
return pages + remainPages
}
pageCount(pagination) // => 11
// ================ Pointfree style ==================
const getPages = R.pipe(
R.converge(R.divide, [R.prop('total'), R.prop('itemsPerPage')]),
ceil
)
const getRemainPages = R.ifElse(
R.pipe(
R.converge(R.modulo, [R.prop('total'), R.prop('itemsPerPage')]),
R.equals(0)
),
R.always(0),
R.always(1)
)
const pageCount2 = R.converge(R.add, [
getPages,
getRemainPages
])
pageCount2(pagination) // => 11
I wrote a simple pagination module to calculate the pageCount of giving total items count and items count per page in pointful style and pointfree style. Apparently the pointful style is much more readable than the pointfree style version. The latter is kind of obscure.
Am I doing it right? Is there any way to make the code in pointfree style more readable?
Manual composition
Let's start with manually composing functions:
const calcPages = (totalItems, itemsPerPage) =>
ceil(div(totalItems, itemsPerPage));
const div = (x, y) => x / y;
const ceil = Math.ceil;
const pagination = {
total: 101,
itemsPerPage: 10,
currentPage: 1
}
console.log(
calcPages(pagination.total, pagination.itemsPerPage)
);
Programmatic composition
With the next step we abstract the parameters away:
const comp2 = (f, g) => (x, y) => f(g(x, y));
const div = (x, y) => x / y;
const ceil = Math.ceil;
const calcPages = comp2(ceil, div);
const pagination = {
total: 101,
itemsPerPage: 10,
currentPage: 1
}
console.log(
calcPages(pagination.total, pagination.itemsPerPage)
);
The function definition is now point-free. But the calling code isn't. Provided you know how the higher order function comp2 works, the expression comp2(ceil, div) is pretty declarative for you.
It is now obvious, that calcPages is the wrong name, because the function composition is much more general. Let's call it ... intDiv (well, there is probably a better name, but I suck at math).
Destructuring Modifier
In the next step we modify intDiv so that it can handle objects:
const destruct2 = (x, y) => f => ({[x]:a, [y]:b}) => f(a, b);
const comp2 = (f, g) => (x, y) => f(g(x, y));
const div = (x, y) => x / y;
const ceil = Math.ceil;
const intDiv = comp2(ceil, div);
const calcPages = destruct2("total", "itemsPerPage") (intDiv);
const pagination = {
total: 101,
itemsPerPage: 10,
currentPage: 1
}
console.log(
calcPages(pagination)
);
I called the modified function calcPages again, because it now expects a specific pagination object and thus is less general.
Provided you know how the involved higher order functions work, everything is declarative and well readable, even though it is written in point-free style.
Conclusion
Point-free style is the result of function composition, currying and higher order functions. It is not a thing in itself. If you stop using these tools in order to avoid point-free style, then you lose a lot of expressiveness and elegance that functional programming provides.
Let's start with a simple example:
// inc :: Number -> Number
const inc = R.add(1);
I find the above clearer than its "pointful" equivalent:
// inc :: Number -> Number
const inc = n => R.add(1)(n);
The lambda in the line above is just noise once one is comfortable with partially applying Ramda functions.
Let's go to the other extreme:
// y :: Number
const y = R.ifElse(R.lt(R.__, 0), R.always(0), Math.sqrt)(x);
This would be much more clearly written in "pointful" style:
// y :: Number
const y = x < 0 ? 0 : Math.sqrt(x);
My suggestion is to use point-free expressions in the simple cases, and revert to "pointful" style when an expression becomes convoluted. I quite often go too far, then undo my last several changes to get back to a clearer expression.
In my opinion, the problem here is the "readability", more specifically the fact, that you cannot read the code continuously from left to right as a text from a book.
Some functions in the ramda library can be read from right to left, like compose():
import { compose, add, multiply } from 'ramda'
// the flow of the function is from right to left
const fn = compose(add(10), multiply) // (a, b) => (a * b) + 10
Then you get to a point, where certain functions require you to give their parameters in a certain order (non-commutative functions), often in a left to right fashion, like lt().
import { __, lt } from 'ramda'
// you need to read the parameters from left to right to understand it's meaning
const isNegative = lt(__, 0) // n => n < 0
When these direction changes meet up, then you have a harder time reading the code, because it takes more effort to find the path, where the code flows.
import { compose, length, gte, __ } from 'ramda'
// 1) you start reading the code from left to right
// 2) you see compose: switch modes to right to left, jump to the end of compose
// 3) you get to gte, where you need to switch back to left to right mode
// 4) glue all previous parts together in your mind, hope you'll still remember where you started
const hasAtLeast3Items = compose(gte(__, 3), length) // arr => arr.length >= 3
There are a few functions, which are really useful, when working with ramda, but can ruin the readability instantly. Those require you to switch reading directions too many times, requiring you to keep track of too many substeps from the code. The number one function for me from that list is converge.
Note: it might seem like the above problem only occures with functions with more, than 1 parameters, but add() doesn't have this issue, add(__, 3) and add(3) are the same.

Categories

Resources