Develop Reference

Map numbers to other numbers with interpolation in Javascript

I have a dataset with a volume for a given surface elevation of an irregular basin. For example:
cm kL
11870 : 6043453
11871 : 6053522
11872 : 6063591
11873 : 6073674
11874 : 6083767
(...1550 rows)
cm is a series that always increments by one; The associated kL values are irregular but always increase and are never duplicated. The mapping never changes and it can be loaded/stored in any convenient format.
Does Javascript have a simple way to convert between cm and kL? Ideally with linear interpolation in both directions. Ultimately I am looking for this functionality:
cm_to_kL(11872.2); //Expect 6065607.6
kL_to_cm(6065600); //Expect 11872.199

I wrote an example of how to start solving this problem. Like already mentioned, there are no internal functionality for interpolating or handling such structures, but you need to write your own logic.
I have to admit I'm not an expert what comes to math (+ it's 2am here, but this question got me interested in :D).
I hope this helps you at least to get started:
const data = {
11870 : 6043453,
11871 : 6053522,
11872 : 6063591,
11873 : 6073674,
11874 : 6083767,
};
const cm_to_kL = (val) => {
const cm_ref = Math.floor(val);
const factor = parseFloat((val % cm_ref).toFixed(5));
const lower = data[cm_ref];
const higher = data[cm_ref + 1];
if (isNaN(lower) || isNaN(higher)) {
throw Error('Data entry point not found');
}
const result = lower + ((higher - lower) * factor);
if (isNaN(result)) {
throw Error('Incorrect data provided');
}
return result;
};
const kL_to_cm = (val) => {
const [cm, kL] = Object.entries(data).find(([k, v]) => val < v);
if (isNaN(cm) || isNaN(kL)) {
throw Error('Data entry point not found');
}
const lower_cm = cm - 1;
const lower_kL = data[lower_cm];
const diff = (val - lower_kL) / (kL - lower_kL);
const result = parseFloat((lower_cm + diff).toFixed(5))
if (isNaN(result)) {
throw Error('Incorrect data provided');
}
return result;
};
console.log('11872.2',
cm_to_kL(11872.2),
);
console.log('6065600',
kL_to_cm(6065600),
);

Yes of course JS have to do what you need! You can create 2 Maps from your given array, one for cm to kL and another for kL to cm. Create two functions for them cm_to_kL and kL_to_cm to gat value from Maps after this you can easily get elements with O(1) complexity

Is this a proper async/await implementation when the nested promises are literally thousands?

Regarding: https://github.com/gvasquez95/binance-trader/issues/8
I'm concerned that the actual "inner" await (in every single invocation of getPriceChange) isn't blocking asynchronous invocation of the about 1.5k calls this method has on the outer loop (getPriceChanges). As if it was blocking, then the loop would be serial processing.
From the performance view and timing, I'd say it is working as expected, as it takes just some seconds to complete the whole process but, I'd like a programming ACK of the assumptions, if possible, please.
Block codes for those that don't want to get into GitHub and, prefer the code more visible here:
Outer loop:
async function getPriceChanges (since, breakpoint) {
const changes = []
for (const [i, ticker] of tickers.entries()) {
if (!ticker.includes('BIDR') && !ticker.includes('BIFI')) {
changes.push(getPriceChange(ticker, since, breakpoint))
}
}
const res = await Promise.all(changes)
let symbol
let maxChange = 0
let slope = 0
for (const ticker of res) {
if (ticker.percentChange > maxChange && ticker.slope > Math.pow(10, -1 * NUM_DECIMALS)) {
maxChange = ticker.percentChange
symbol = ticker.symbol
slope = ticker.slope
}
}
return { symbol, maxChange: Math.round(100 * maxChange) / 100, slope: round(slope, NUM_DECIMALS) }
}
Inner getPriceChange function:
async function getPriceChange (symbol, since, breakpoint) {
const params = {
TableName: DYNAMODB_TABLE,
ProjectionExpression: '#timestamp, price',
KeyConditionExpression: 'symbol = :symbol and #timestamp > :timestamp',
ExpressionAttributeNames: {
'#timestamp': 'timestamp'
},
ExpressionAttributeValues: {
':symbol': symbol,
':timestamp': since
}
}
const res = await documentClient.query(params).promise()
const prev = []
const recent = []
const trendData = []
for (const data of res.Items) {
if (data.timestamp < breakpoint) {
prev.push(data.price)
} else {
trendData.push({ x: trendData.length, y: data.price })
recent.push(data.price)
}
}
let sumPrev = 0
let sumRecent = 0
for (const price of prev) { sumPrev += price }
for (const price of recent) { sumRecent += price }
const avgPrev = sumPrev / prev.length
const avgRecent = sumRecent / recent.length
const trend = createTrend(trendData, 'x', 'y')
return { symbol, percentChange: (100 * (avgRecent - avgPrev)) / avgRecent, slope: trend.slope }
}
Update:
Function execution takes about 30 seconds in a Node.js 14.x AWS Lambda environment, configured with 624 MB RAM and, from a cost perspective the both the DynamoDB query (ReadRequests) costs and Lambda executions are 1/50th of the actual DynamoDB Batch Put operations (WriteRequest), so optimizing the query should be a needed goal.

Your loop is correctly asynchronous. And it seems you're concerned about how many queries will be in flight at the same time. In your case, you intended for many concurrent queries, and you have done as such.
Explanation
In the simple example of ordering 10 pizzas, consider these arrays:
concurrently
This is an array of pizza orders. We can await all 10 orders to get an array of 10 pizzas. Assuming there are at least 10 delivery people working that night, there's a good chance you'll get all your pizzas within 30 minutes.
for (let i = 0; i < 10; ++i) {
orders.push(orderPizza());
}
pizzas = await Promise.all(orders);
sequentially
This is an array of pizzas. Each order is awaited as it is issued. If each pizza is delivered in 30 minutes or it's free this is likely to result in about 5 hours (10 * ~30 minutes) of waiting for that last pizza because the next pizza isn't ordered until the previous one arrives. (No need for a final Promise.all in this case as each promise is already resolved before continuing with the loop.)
for (let i = 0; i < 10; ++i) {
pizzas.push(await orderPizza());
}

Not sure I understand what your question is here exactly. Your assumptions about your code seem to be correct – I (and many others I assume) would be highly surprised if 1500 promises changed anything in their execution model.
As a general note, if you need more control over promise execution, I can recommend checking out Sindre Sorhus' collection of promise helpers. You'll find tools for limiting concurrency, allowing for promise failures, async mapping … and much more.

Javascript circular calculation involving arrays (example: amortization schedule)

I'm fairly early on in building an app that can be used as an alternative to spreadsheets in a number of scenarios. It offers the user a non-tabular approach to creating and organizing the analysis. The user's instructions are transpiled to and executed by Javascript (using pure functions only). (The interface is between something like Google Blockly and straight coding in a text editor.) In place of cell ranges that you would typically use in a spreadsheet, this product uses Javascript arrays. A problem that I'm facing with this approach is (quasi-)circular calculations such as that found in a simple amortization schedule.
My (unsuccessful) attempts to resolve the issue so far involve:
lazy list evaluation (using https://github.com/dankogai/js-list-lazy)
wrapping all transpiled JS in functions to delay eval so that the user's content doesn't need to be topologically sorted (which it currently is).
Below I'll provide hopefully enough context to illustrate the issue, which can be pretty broadly extrapolated. Take the example of an amortization schedule for a mortgage:
Basically, the BOP ("Beginning of Period") Balance depends on the EOP ("End of Period") Balance from the previous period. And the EOP Balance depends on the BOP Balance from the same period. In a spreadsheet that uses ranges of contiguous cells, this isn't circular because every BOP Balance and EOP Balance is a discrete cell. However, if BOP Balance and EOP Balance (and all of the other time-series-based calcs) are arrays then there is a circular reference when trying to retrieve elements. Spreadsheet screenshots of the example are provided at the end of the post as a supplement.
An attempt to build this analysis in my app generates the following JS (which I've edited and reorganized for clarity). This code, if translated over to a spreadsheet, works just fine (see supplemental screenshots at the end of the post):
// Credit: Apache OpenOffice
function payment (rate, periods, present_value, future_value, type) {
var fv = future_value || 0
var t = type || 0
if (rate === 0) {
return -1 * ((present_value + fv) / periods)
} else {
var term = (1 + rate) ** periods // Transpiling with Babel; otherwise, use Math.pow(1 + rate, periods)
if (t === 1) {
return -1 * ((fv * rate / (term - 1) + present_value * rate / (1 - 1 / term)) / (1 + rate))
} else {
return -1 * (fv * rate / (term - 1) + present_value * rate / (1 - 1 / term))
}
}
}
var loan_principal = 1000000
var annual_interest_rate = 0.06
var interest_rate_monthly = annual_interest_rate / 12
var amortization_period_years = 25
var amortization_period_months = amortization_period_years * 12
var months_range = _.range(1, amortization_period_months + 1, 1) // See: http://underscorejs.org/#range [1, 2, 3, ... 298, 299, 300]
var bop_balance = months_range.map(function (current_element, current_list_position) {
if (current_list_position === 0) {
return loan_principal
} else {
// Along with eop_balance, this causes a graph cycle
return eop_balance[current_list_position - 1]
}
})
var monthly_payment = months_range.map(function (current_element, current_list_position) {
return payment(interest_rate_monthly, amortization_period_months, loan_principal)
})
var principal_payment = months_range.map(function (current_element, current_list_position) {
var current_mthly_pmt = monthly_payment[current_list_position]
var current_int_pmt = interest_payment[current_list_position]
return current_mthly_pmt - current_int_pmt
})
var interest_payment = months_range.map(function (current_element, current_list_position) {
if (current_list_position === 0) {
return loan_principal * interest_rate_monthly * -1
} else {
var previous_bal = eop_balance[current_list_position - 1]
return previous_bal * interest_rate_monthly * -1
}
})
var eop_balance = months_range.map(function (current_element, current_list_position) {
// This causes a graph cycle
var cur_bal = bop_balance[current_list_position]
var cur_prin_pmt = principal_payment[current_list_position]
return cur_bal + cur_prin_pmt
})
This code will not topologically sort because of the cycle between bop_balance and eop_balance. And it won't fully evaluate because of the circular reference.
Any suggestions on how to work around this general scenario? Thank you.
Supplemental Info:
Here are two views of the same spreadsheet representing the analysis:
The reliance on pure functions in the app is to try and minimize confusion for users coming from spreadsheets.
If seeing the actual app would help provide context, please feel free to visit https://www.getpinevale.com. I'm placing this at the end so it doesn't distract from the question.

You shouldn't do a double linked structure. I would suggest use a plain for loop and build up both arrays.
for(int i=0; i<range.length;i++){
if(i==0){
bop_balance[i]= loan_principal;
eop_balance[i]= somecalculatedamount(loan_principal);
}else{
bop_balance[i] = eop_balance[i-1];
eop_balance[i] = somecalculatedamount(bop_balance[i])
}
}
I don't know if my function is correct but the essential point I am trying to make:
don't link the data structures
and use a control structure that is outside of both arrays
Based on the comment below I'd make the suggestion of using reduce.
let range = [1,2,3,4];
let output = range.reduce(function(accumulator, currentValue, currentIndex) {
let balance = {};
if (currentIndex == 0) {
balance.bop = 0;
balance.eop = currentValue;
}else{
balance.bop = accumulator[currentIndex-1].eop;
balance.eop = balance.bop+5;
}
accumulator.push(balance);
return accumulator;
}, []);
console.log(output);

Creating millions of Objects in Javascript

Let me be the first to say that this isn't something I normally do, but out of curiousity, I'll see if anyone has a good idea on how to handle a problem like this.
The application I am working on is a simulated example of the game Let's make a Deal featuring the Monty Hall problem.
I won't go into details about my implementation, but it more or less allows a user to enter a number of how many games they want to simulate, and then if an option is toggled off, the player of those x games won't switch their choice, while if it is toggled on, they will switch their choice every single instance of the game.
My object generator looks like this:
const game = function(){
this[0] = null;
this[1] = null;
this[2] = null;
this.pick = Math.floor(Math.random() * 3);
this.correctpick = Math.floor(Math.random() * 3);
this[this.correctpick] = 1;
for (let i=0; i<3; i++){
if ((this[i] !== 1) && (i !== this.pick)){
this.eliminated = i;
break;
}
}
}
const games = arg => {
let ret = [];
for(let i=0; i<arg; i++){
ret.push(new game);
}
return ret;
}
This structure generates an array which i stringify later that looks like this:
[
{
"0": 1,
"1": null,
"2": null,
"pick": 2,
"correctpick": 0,
"eliminated": 1
},
{
"0": null,
"1": null,
"2": 1,
"pick": 2,
"correctpick": 2,
"eliminated": 0
}
]
As sloppy as the constructor for game looks, the reason is because I have refactored it into having as few function calls as possible, where now I'm literally only calling Math functions at the current time (I removed any helper functions that made the code easier to read, in opt for performance).
This app can be ran both in the browser and in node (cross platform), but I have clamped the arg a user can pass into the games function to 5 million. Any longer than that and the process (or window) freezes for longer than a few seconds, or even potentially crashes.
Is there anything else I can do to increase performance if a huge number is given by a user? Also, if you need more information, I will be happy to supply it!
Thanks!

The obvious performance optimisation would be not to create and store 5 million objects at all, relieving memory pressure. Instead you'd create the objects on the fly only when you need them and throw them away immediately after. I'm not sure what your app does, but it sounds like you want to re-use the same game instances when evaluating results with the different options. In that case, you need to store them of course - but I'd advise to re-think the design and consider immediately evaluating each game with all possible options, accumulating only the results for each choice of options but not keeping all games in memory.
Apart from that, I'd recommend to simplify a bit:
You can drop that loop completely and use some clever arithmetic to find the eliminated option: this.eliminated = this.pick == this.correctpick ? +!this.pick : 3 - this.pick - this.correctpick;. Or use a simple lookup table this.eliminated = [1, 2, 1, 2, 0, 0, 1, 0, 0][this.pick * 3 + this.correctpick].
I'd avoid changing the type of the array elements from null (reference) to 1 (number). Just keep them as integers and initialise your elements with 0 instead.
Don't store 6 properties in your object that are completely redundant. You only need 2 of them: pick and correctpick - everything else can be computed on the fly from them when you need it. Precomputing and storing it would only be advantageous if the computation was heavy and the result was used often. Neither of this is the case, but keeping a low memory footprint is important (However, don't expect much from this).

Not sure about your implementation, but do you really need an Array?
How about only using results (see snippet)?
If it's blocking the browser that worries you, maybe delegating the work to a web worker is the solution for that: see this jsFiddle for a web worker version of this snippet.
(() => {
document.querySelector("#doit")
.addEventListener("click", playMontyHall().handleRequest);
function playMontyHall() {
const result = document.querySelector("#result");
const timing = document.querySelector("#timing");
const nOfGames = document.querySelector("#nGames");
const switchDoors = document.querySelector("#switchyn");
// Create a game
const game = (doSwitch) => {
const doors = [0, 1, 2];
const pick = Math.floor(Math.random() * 3);
const correctPick = Math.floor(Math.random() * 3);
const eliminated = doors.filter(v => v !== pick && v !== correctPick)[0];
return {
correctpick: correctPick,
pick: doSwitch ? doors.filter(v => v !== pick && v !== eliminated)[0] : pick,
eliminated: eliminated,
};
};
const getWinner = game => ~~(game.correctpick === game.pick);
// Sum wins using a generator function
const winningGenerator = function* (doSwitch, n) {
let wins = 0;
while (n--) {
wins += getWinner(game(doSwitch));
yield wins;
}
};
// calculate the number of succeeded games
const calculateGames = (nGames, switchAlways) => {
const funNGames = winningGenerator(switchAlways, nGames);
let numberOfWins = 0;
while (nGames--) {
numberOfWins = funNGames.next().value;
}
return numberOfWins;
}
const cleanUp = playOut => {
result.textContent =
"Playing ... (it may last a few seconds)";
timing.textContent = "";
setTimeout(playOut, 0);
};
const report = results => {
timing.textContent = `This took ${
(performance.now() - results.startTime).toFixed(3)} milliseconds`;
result.innerHTML =
`<b>${!results.switchAlways ? "Never s" : "Always s"}witching doors</b>:
${results.winners} winners out of ${results.nGames} games
(${((results.winners/+results.nGames)*100).toFixed(2)}%)`;
};
// (public) handle button click
function clickHandle() {
cleanUp(() => {
const nGames = nOfGames.value || 5000000;
const switchAlways = switchDoors.checked;
report({
switchAlways: switchAlways,
startTime: performance.now(),
winners: calculateGames(nGames, switchAlways),
nGames: nGames
});
});
}
return {
handleRequest: clickHandle
};
}
})();
body {
margin: 2em;
font: normal 12px/15px verdana, arial;
}
#timing {
color: red;
}
<p><input type="number" id="nGames" value="5000000"> N of games</p>
<p><input type="checkbox" id="switchyn"> Always switch doors</p>
<p><button id="doit">Play</button>
<p id="result"></p>
<p id="timing"></p>

Not truly async?

I have a array of about 18 000 elements. I'm creating a map application where I want to add the elements when the user zooms in to a certain level.
So when the user zooms in under 9 I loop tru the array looking for elements that is in the view.
However, it does take some time looping thru the elements, causing the map application lag each time the user zooms out and in of "level 9". Even if there are no elements to add or not, so the bottleneck is the looping I guess.
I've tried to solve it by asyncing it like:
function SearchElements(elementArr) {
var ret = new Promise(resolve => {
var arr = [];
for (var i in elementArr) {
var distanceFromCenter = getDistanceFromLatLonInKm(view.center.latitude, view.center.longitude, dynamicsEntities[i].pss_latitude, dynamicsEntities[i].pss_longitude);
var viewWidthInKm = getSceneWidthInKm(view, true);
if (distanceFromCenter > viewWidthInKm) continue;
arr.push(elementArr[i]);
}
resolve(arr);
});
return ret;
}
SearchElements(myElementsArray).Then(arr => {
// ...
});
But its still not async, this method hangs while the for loop runs.

Because you still have a tight loop that loops through all the elements in one loop, you'll always have the responsiveness issues
One way to tackle the issue is to works on chunks of the data
Note: I'm assuming elementArr is a javascript Array
function SearchElements(elementArr) {
var sliceLength = 100; // how many elements to work on at a time
var totalLength = elementArr.length;
var slices = ((totalLength + sliceLength - 1) / sliceLength) | 0; // integer
return Array.from({length:slices})
.reduce((promise, unused, outerIndex) =>
promise.then(results =>
Promise.resolve(elementArr.slice(outerIndex * sliceLength, sliceLength).map((item, innerIndex) => {
const i = outerIndex * sliceLength + innerIndex;
const distanceFromCenter = getDistanceFromLatLonInKm(view.center.latitude, view.center.longitude, dynamicsEntities[i].pss_latitude, dynamicsEntities[i].pss_longitude);
const viewWidthInKm = getSceneWidthInKm(view, true);
if (distanceFromCenter <= viewWidthInKm) {
return item; // this is like your `push`
}
// if distanceFromCenter > viewWidthInKm, return value will be `undefined`, filtered out later - this is like your `continue`
})).then(batch => results.concat(batch)) // concatenate to results
), Promise.resolve([]))
.then(results => results.filter(v => v !== undefined)); // filter out the "undefined"
}
use:
SearchElements(yourDataArray).then(results => {
// all results available here
});
My other suggestion in the comment was Web Workers (I originally called it worker threads, not sure where I got that term from) - I'm not familiar enough with Web Workers to offer a solution, however https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers and https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API should get you going
To be honest, I think this sort of heavy task would be better suited to Web Workers