Develop Reference

How to make recursive setTimeout with variable intervals

Given I have the following configuration:
let config = {
"requests": [
{
"resource": "foo",
"interval": 1000
},
{
"resource": "bar",
"interval": 500
},
{
"resource": "baz",
"interval": 3000
},
{
"resource": "qux",
"interval": 500
},
{
"resource": "zot",
"interval": 500
},
],
// other configs...
}
I need to make a recursive setTimeout calls where I check what resources should be requested from a server at a given call and make the request to the server.
For example, considering the array above:
After 500ms, since it's the smallest interval, I have to make a request and pass array of ['bar', 'qux', 'zot']. After another 500ms, and since it's already 1000ms, the array should be ['bar', 'qux', 'zot', 'foo']. After another 500 it should be again ['bar', 'qux', 'zot']. When reaches 3000 - ['bar', 'qux', 'zot', 'foo', 'baz'], and so on...
The configuration itself is coming from a server when the app starts and it's a black box to me. Meaning I can't know exactly what intervals may be configured and how many of them are there. Here I made them increase by 500ms for convenience only (though I think I might make it a technical requirement to the back-end guys, lets assume I can't).
I'm not sure how to tackle this problem. I thought maybe I should store an array of required intervals and do something with that array. Perhaps store a current request as an object with timestamp and interval. Something like that:
const { requests } = config
let intervals = new Set()
for (let obj of requests) {
intervals.add(obj.interval)
}
intervals = Array.from(intervals).sort((a, b) => a - b) //[500, 1000, 3000]
let currentRequest
const makeRequest = resources => {
// send request to server with given resources array
// init the next request by calling initRequest again
}
const initRequest = () => {
const interval = 500 // just for example, the actual interval should be determind here
const resources = [] // Here I should determine what should be the requested resources
setTimeout(() => {
makeRequest(resources)
}, interval)
}
initRequest()
But I'm not sure about the logic here. How can this be done? Any ideas, please?

We create an upcoming array, which stores the next timestamp at which each resource should be retrieved. Then, we know that we need to set a timeout that triggers at the time of the next upcoming timestamp.
When the timeout triggers, we execute upcoming items that are scheduled to run right now. We then calculate the next time it should be run.
Finally, we set a new timeout to coincide with the next time a task is scheduled to run.
const config = {
requests: [
{ resource: 'foo', interval: 1000 },
{ resource: 'bar', interval: 500 },
{ resource: 'baz', interval: 3000 },
{ resource: 'qux', interval: 500 },
{ resource: 'zot', interval: 500 }
]
};
let fireAllAtTimeZero = false;
let t = 0;
let upcoming = config.requests.map(
i=>({...i, next: fireAllAtTimeZero ? 0 : i.interval}));
f = () => {
let now = upcoming.filter(i=>i.next===t);
console.log(`time = ${t}, doing: ${now.map(i=>i.resource).join()}`);
// initiate your async request here for every item in 'now' array
now.forEach(i=>i.next = t + i.interval);
let next = upcoming.map(i=>i.next).sort((a,b)=>a-b)[0];
let delay = next-t;
t = next;
setTimeout(f, delay);
};
f();

You could use setTimeouts:
initRequests = () => {
let i = -1;
const resources = [];
const timeoutHandler = () => {
if(i < intervals.length){
setTimeout(timeoutHandler, intervals[++i]);
}
}
timeoutHandler();
}

How do I increment automatically a variable when calling a Cypress method?

createUserPatch is an API custom command to create a new User.
You can see that I have created a variable "A" inside it.
The variable is used in the body emails part [a]+'freddie.doe#example.com','type': 'work','primary': true}]
I want to find a way to automatically increase the variable "A" whenever I call the command createUserPatch.
Cypress.Commands.add('createUserPatch', (user) => {
var A = 1;
cy.request({
method: 'POST',
url: '/scim/v2/users',
qs: {
key : Cypress.env('apiKey')
},
body :{
schemas: user.schemas,
userName: user.userName,
emails: [{ 'value': [A]+'freddie.doe#example.com','type': 'work','primary': true}],
name : 'Maurice'
}
}).then(res => {
return res.body;
});
});
I use this command in the test below in a before each.
let user = {
schemas:'["urn:ietf:params:scim:schemas:core:2.0:User"]',
userName: 'John',
userId: null,
groupID: null
};
describe('Deactivating a user ', () => {
beforeEach(() => {
cy.createUserPatch(user).then((newUser) => {
return user.userId = newUser.id;
});
});
....
Each time I run this test.
I want the value of the email to be increased.
First test -> 0freddie.doe#example.com
Second test -> 1freddie.doe#example.com
Third test -> 2freddie.doe#example.com
etc...

Cypress clears variables between tests, but a few ways to preserve data have been suggested in other questions (e.g write data to file).
Since you already use Cypress.env(name) to access environment variables you could use Cypress.env(name, value) to track the prefix.
Cypress.Commands.add('createUserPatch', (user) => {
let prefix = Cypress.env('emailprefix') || 0; // first time it may be undefined
// so "|| 0" initializes it
Cypress.env('emailprefix', ++prefix); // increment and save
cy.request({
...
emails: [{ 'value': prefix+'freddie.doe#example.com','type': 'work','primary': true}],
Note, the prefix value will be preserved between runs, which may or may not be what you want.
To clear it between runs, add a before() which resets the value
before(() => Cypress.env('emailprefix', 0) );
beforeEach(() => {
cy.createUserPatch().then(console.log)
})

K6 Request an API sequentially based on VU Id

I am making a load testing program using K6, JavaScript in which I want to request an API sequentially based on VU id.
For example, let suppose I have 5 different payloads and 5 VU one for each payload.
I want my program to run such as, for the 1st min only VU-1 will get executed requesting continuously with my 1st payload, then for next min, VU-2 will execute and so on till 5th and then again 1st VU will get executed.
Based on the VU Id I will be changing the payload.
I have tried setTimeout() method of JavaScript but not working with k6.
Also I have tried the scenario option from k6 documentation but its running parallelly. here is the code snippet I tried.
export let options = {
scenarios: {
execution: {
scenario1: {
type: "per-vu-iterations",
vus: 1,
iterations: 1,
maxDuration: "15s",
exec: "scenario1",
},
scenario2: {
type: "per-vu-iterations",
vus: 1,
iterations: 1,
maxDuration: "15s",
exec: "scenario2",
},
}
}
}
export function scenario1() {
let res = http.get("https://test.loadimpact.com/");
console.log(__VU);
check(res, {
"is status 200": (r) => r.status === 200
});
};
export function scenario2() {
let res = http.get("https://test.loadimpact.com/");
console.log(__VU);
check(res, {
"is status 200": (r) => r.status === 200
});
};
Is their any method in K6 to achieve this?

One solution for this problem I found, is using the Scenerios options with executer and startTime parameters. But the issue in this solution I am facing is that I am not able to view the metrics result for the individual scenario.
If there is any way to display the metrics do share.
This is a sample code for sequential running.
export let options = {
scenarios: {
scenario_1: {
executor: "constant-vus",
vus: 1,
duration: "1m",
exec: "scenario1",
tags: { test_type: 'api-test1' },
},
scenario_2: {
executor: "constant-vus",
vus: 1,
duration: "1m",
exec: "scenario2",
startTime: "1m",
tags: { test_type: 'api-test2' },
},
},
}
export function scenario1() {
let res = http.get("https://example.com/");
console.log(JSON.stringify(res));
};
export function scenario2() {
let res = http.get("https://example.com/");
console.log(JSON.stringify(res));
};

Execute Multiple async operation at gets notified when all of them completes (Typescript)

I have the following data structure in my firebase
- ActionSheet
- PendingApproval
- SomeKey1
- user: 1
- data: 'walk the dog'
- SomeKey2
- user: 2
- data: 'brush the cat'
- Approved
- SomeKey3
- user: 1
- data: 'feed fish'
I want to download all data from user 1 in this structure such that I can display to the user to 'walk the dog' and 'feed fish'
At the moment this is how I do it. I am "chaining" the calls such that the 2nd call does not start until the first one finishes.
ngOnInit() {
this.databaseService.searchCurrentUserPendingApproval()
.first()
.subscribe(results => {
for (let result of results) {
this.mySubmissions.push(result);
}
this.databaseService.searchCurrentUserApprovedAction()
.first()
.subscribe(results => {
for (let result of results) {
this.mySubmissions.push(result);
}
// do some logic to the this.mySubmissio array at this point
doSomething(this.mySubmission);
}, error => {
console.log('Error download Actioned Timesheet', error)
})
}, error => {
console.log('error retreiving pending submission from firebase', error)
})
}
searchCurrentUserPendingTimesheets(key: string) {
let path = '/ActionSheet/PendingApproval' + key;
return this.af.database.list(path, {
query: {
orderByChild: 'user',
equalTo: 1
}
})
}
// search for own timesheet in firebase
searchCurrentUserActionedTimesheets(key: string) {
let path = '/ActionSheet/Approved' + key;
return this.af.database.list(path, {
query: {
orderByChild: 'user',
equalTo: 1
}
})
}
This works. However, the biggest problem is that this is a terrible way of doing it and it would be slow for larger "chain".
As the two query does not depend on each other, a better way to do it would be to to query the two branch at the same time and get notified when both of the operation completes (Doesn't matter which one comes back first as I will sort the array at the end').
In other words, I want to run doSomething(this.mySubmission) once all of the async calls mentioned is completed. Is there a proper typescript way to achieve this?
Note. When I was working with Swift, I was able to achieve this with dispath_group_enter/dispatch_group_leave.

You can try with forkJoin as below:
Observable.forkJoin(
this.databaseService.searchCurrentUserPendingApproval().first(),
this.databaseService.searchCurrentUserApprovedAction().first()
).subscribe([pendingResults, approvedResults] => {
this.mySubmissions = this.mySubmissions.concat(pendingResults, approvedResults);
doSomething(this.mySubmissions);
}, error => console.log('Error download Actioned Timesheet', error))

You can wait for multiple observables using forkJoin.

RxJS Observable fire onCompleted after a number of async actions

I'm trying to create an observable that produces values from a number of asynchronous actions (http requests from a Jenkins server), that will let a subscriber know once all the actions are completed. I feel like I must be misunderstanding something because this fails to do what I expect.
'use strict';
let Rx = require('rx');
let _ = require('lodash');
let values = [
{'id': 1, 'status': true},
{'id': 2, 'status': true},
{'id': 3, 'status': true}
];
function valuesObservable() {
return Rx.Observable.create(function(observer) {
_.map(values, function(value) {
var millisecondsToWait = 1000;
setTimeout(function() { // just using setTimeout here to construct the example
console.log("Sending value: ", value);
observer.onNext(value)
}, millisecondsToWait);
});
console.log("valuesObservable Sending onCompleted");
observer.onCompleted()
});
}
let observer = Rx.Observer.create((data) => {
console.log("Received Data: ", data);
// do something with the info
}, (error) => {
console.log("Error: ", error);
}, () => {
console.log("DONE!");
// do something else once done
});
valuesObservable().subscribe(observer);
Running this, I get output:
valuesObservable Sending onCompleted
DONE!
Sending value: { id: 1, status: true }
Sending value: { id: 2, status: true }
Sending value: { id: 3, status: true }
While what I would like to see is something more like:
Sending value: { id: 1, status: true }
Received Data: { id: 1, status: true }
Sending value: { id: 2, status: true }
Received Data: { id: 2, status: true }
Sending value: { id: 3, status: true }
Received Data: { id: 3, status: true }
valuesObservable Sending onCompleted
DONE!
I don't actually care about the order of the items in the list, I would just like the observer to receive them.
I believe what is happening is that Javascript asynchronously fires the timeout function, and proceeds immediately to the observer.onCompleted() line. Once the subscribing observer receives the onCompleted event (is that the right word?), it decides that it's done and disposes of itself. Then when the async actions complete and the observable fires onNext, the observer no longer exists to take any actions with them.
If I'm right about this, I'm still stumped about how to make it behave in the way I would like. Have I stumbled into an antipattern without realising it? Is there a better way of approaching this whole thing?
Edit:
Since I used setTimeout to construct my example, I realised I can use it to partially solve my problem by giving the observable a timeout.
function valuesObservable() {
return Rx.Observable.create(function(observer) {
let observableTimeout = 10000;
setTimeout(function() {
console.log("valuesObservable Sending onCompleted");
observer.onCompleted();
}, observableTimeout);
_.map(values, function(value) {
let millisecondsToWait = 1000;
setTimeout(function() {
console.log("Sending value: ", value);
observer.onNext(value)
}, millisecondsToWait);
});
});
}
This gets me all of the information from the observable in the order I want (data, then completion) but depending on the choice of timeout, I either may miss some data, or have to wait a long time for the completion event. Is this just a inherent problem of asynchronous programming that I have to live with?

Yes there is a better way. The problem right now is that you are relying on time delays for your synchronization when in fact you can use the Observable operators to do so instead.
The first step is to move away from directly using setTimeout. Instead use timer
Rx.Observable.timer(waitTime);
Next you can lift the values array into an Observable such that each value is emitted as an event by doing:
Rx.Observable.from(values);
And finally you would use flatMap to convert those values into Observables and flatten them into the final sequence. The result being an Observable that emits each time one of the source timers emits, and completes when all the source Observables complete.
Rx.Observable.from(values)
.flatMap(
// Map the value into a stream
value => Rx.Observable.timer(waitTime),
// This function maps the value returned from the timer Observable
// back into the original value you wanted to emit
value => value
)
Thus the complete valuesObservable function would look like:
function valuesObservable(values) {
return Rx.Observable.from(values)
.flatMap(
value => Rx.Observable.timer(waitTime),
value => value
)
.do(
x => console.log(`Sending value: ${value}`),
null,
() => console.log('Sending values completed')
);
}
Note the above would work as well if you weren't using demo stream, i.e. if you had really http streams you could even simplify by using merge (or concat to preserve order)
Rx.Observable.from(streams)
.flatMap(stream => stream);
// OR
Rx.Observable.from(streams).merge();
// Or simply
Rx.Observable.mergeAll(streams);

The best way to construct an observable is to use the existing primitive and then a combination of the existing operators. This avoids a few headaches (unsubscription, error management etc.). Then Rx.Observable.create is certainly useful when nothing else fits your use case. I wonder if generateWithAbsoluteTime would fit.
Anyways, here the issue you run into is that you complete your observer before you send him data. So basically you need to come up with a better completion signal. Maybe :
complete x seconds after last value emitted if no new value is emitted
complete when a value is equal to some 'end' value

With thanks to #paulpdaniels, this is the final code that did what I wanted, including the calls to Jenkins:
'use strict';
let Rx = require('rx');
let jenkinsapi = require('jenkins'); // https://github.com/silas/node-jenkins/issues
let jenkinsOpts = {
"baseUrl": "http://localhost:8080",
"options": {"strictSSL": false},
"job": "my-jenkins-job",
"username": "jenkins",
"apiToken": "f4abcdef012345678917a"
};
let jenkins = jenkinsapi(JSON.parse(JSON.stringify(jenkinsOpts)));
function jobInfoObservable(jenkins, jobName) {
// returns an observable with a containing a single list of builds for a given job
let selector = {tree: 'builds[number,url]'};
return Rx.Observable.fromNodeCallback(function(callback) {
jenkins.job.get(jobName, selector, callback);
})();
}
function buildIDObservable(jenkins, jobName) {
// returns an observable containing a stream of individual build IDs for a given job
return jobInfoObservable(jenkins, jobName).flatMap(function(jobInfo) {
return Rx.Observable.from(jobInfo.builds)
});
}
function buildInfoObservable(jenkins, jobName) {
// returns an observable containing a stream of http response for each build in the history for this job
let buildIDStream = buildIDObservable(jenkins, jobName);
let selector = {'tree': 'actions[parameters[name,value]],building,description,displayName,duration,estimatedDuration,executor,id,number,result,timestamp,url'};
return buildIDStream.flatMap(function(buildID) {
return Rx.Observable.fromNodeCallback(function(callback) {
jenkins.build.get(jobName, buildID.number, selector, callback);
})();
});
}
let observer = Rx.Observer.create((data) => {
console.log("Received Data: ", data);
// do something with the info
}, (error) => {
console.log("Error: ", error);
}, () => {
console.log("DONE!");
// do something else once done
});
buildInfoObservable(jenkins, jenkinsOpts.job).subscribe(observer);
By relying on the Rx built-in operators I managed to avoid messing about with timing logic altogether. This is also much cleaner than nesting multiple Rx.Observable.create statements.