Develop Reference

Recursive Observable calls returning no data

I need some help from RxJS professionals :)
I try to recursively load data from a REST API via http request.
Recursive calls are working fine, however when I susbscribe to the final Observable (returned by GetTemperatures), no data is returned within subscribe.
Seems like no data is passed back in the call chain.
Whats going wrong here?
GetTemperatures().subscribe((data: MeasureData) => {
// add data to a chart, etc...
})
GetTemperatures(): Observable<MeasureData> {
const l_startDate = new Date(2019, 0, 1);
var l_httpParams = new HttpParams()
.set('device_id', this._deviceId)
.set('module_id', this._moduleId)
.set('scale', '1hour')
.set('type', 'Temperature')
.set('date_begin', Math.floor(l_startDate.getTime() / 1000).toString())
.set('real_time', 'true')
.set('optimize', 'true');
return this._http.post<MeasureDataInternal>(this._getMeasureUrl, l_httpParams)
.pipe(
map((data: MeasureDataInternal): MeasureData => this.transformMeasureData(data)),
flatMap((data: MeasureData) => {
return this.recursiveLoadData(data);
})
);
}
recursiveLoadData(data: MeasureData): Observable<MeasureData> {
// search until now minus 1,5 hours
const endDate = new Date(Date.now() - (1.5 * 60 * 60 * 1000));
console.error('RECURSIVE begin: ' + data.value[0].date + ' end: ' + data.value[data.value.length - 1].date);
// check if complete
if (data.value[data.value.length - 1].date.getTime() >= endDate.getTime()) {
console.error('recursive ENDs here');
return EMPTY;
}
var l_httpParams = new HttpParams()
.set('device_id', this._deviceId)
.set('module_id', this._moduleId)
.set('scale', '1hour')
.set('type', 'Temperature')
.set('date_begin', Math.floor(data.value[data.value.length - 1].date.getTime() / 1000).toString())
.set('real_time', 'true')
.set('optimize', 'true');
return this._http.post<MeasureDataInternal>(this._getMeasureUrl, l_httpParams)
.pipe(
map((data2: MeasureDataInternal): MeasureData => this.transformMeasureData(data2)),
flatMap((data2: MeasureData) => {
return this.recursiveLoadData(data2);
})
)
}

I have no idea what you're really trying to accomplish, but each new step in your recursion doesn't do anything other than bringing you to the next step. So you'll want to include what you're hoping each step does.
This isn't specific to streams, this is also true of general recursion.
General Recursion
This really isn't any different from how a regular recursive function works. Say you're recursively adding up the numbers in an array, you need to add the tail of the array to the first value. If you just keep recursing on a smaller array without adding up the numbers you've popped off, you'd get the base-case value back.
This returns the last value of the array (The last value of the array is the base-case):
recursiveAdd(array){
if(array.length === 1) return array[0];
return recursiveAdd(array.shift());
}
This adds the array:
recursiveAdd(array){
if(array.length === 1) return array[0];
return array[0] + recursiveAdd(array.shift());
}
In this simple case, the + operand is doing the work at each step of the recursion. Without it, the array isn't summed up. And, of course, I could do anything. Subtract the array from 1000, average the numbers in the array, build an object from the values. Anything.
Before you make a recursive call, you have to do something. Unless what you're after is the value of the base-case (In your case, an empty stream)
Recursion with Streams
When you mergeMap a value into a stream, you don't also pass forward that value.
from([69,70,71]).pipe(
mergeMap(val => from([
String.fromCharCode(val),
String.fromCharCode(val),
String.fromCharCode(val)
]))
).subscribe(console.log);
output
e e e f f f g g g
Notice how the output doesn't include any numbers? When you mergeMap, you map values into streams. If you want the values you're mapping to be part of the stream, you must include them somehow. This is the same as with general recursion.
So, here are two examples that both include your data in the returned stream. They're very basic, but hopefully, you can take some understanding from them and apply that.
This transforms the returned steam to include your data as its first value (recursively, of course)
return this._http.post<MeasureDataInternal>(this._getMeasureUrl, l_httpParams)
.pipe(
map((data: MeasureDataInternal): MeasureData =>
this.transformMeasureData(data)
),
mergeMap((data: MeasureData) =>
this.recursiveLoadData(data).pipe(
startWith(data)
)
)
);
This creates a stream of your data, a stream of your recursive call, and merges the two streams together.
return this._http.post<MeasureDataInternal>(this._getMeasureUrl, l_httpParams)
.pipe(
map((data: MeasureDataInternal): MeasureData =>
this.transformMeasureData(data)
),
mergeMap((data: MeasureData) =>
merge (
of(data),
this.recursiveLoadData(data)
)
)
);

How can I call an individual Idris function from JavaScript?

Suppose I have a function in Idris that does some computation. For simplicity, let it be stringly typed for now.
f: String -> String
How can I compile this function to JavaScript so that it can then be called from any ordinary JavaScript code?
If that is too easy, suppose f, instead of String, deals with Double or even a custom Idris data type.
I know I can compile a whole module with a Main.main function and a more or less inscrutable blob of JavaScript will be output. Can I maybe extract my function from there by hand? How should I go about it?
P.S. Despite my answering myself, I am still looking for a better solution, so welcome.

Using this example, it seems at least with the Node backend this is doable. I've marked interact as export and added a library descriptor:
module Main
import Data.String
f: Double -> Double
f x = x + 1
export interact: String -> String
interact s = let x = parseDouble s in
case x of
Nothing => "NaN"
Just x => show (f x)
main: IO ()
main = do
s <- getLine
putStrLn (interact s)
lib : FFI_Export FFI_JS "" []
lib = Data String "String" $
Fun interact "interact" $
Fun main "main" $
End
I have then compiled with the --interface flag (this fails with --codegen javascript...):
idris --codegen node --interface --output ExportToJS.js ExportToJS.idr
and the resulting .js file has this at the end:
module.exports = {
interact: Main__interact,
main: Main__interact
};
}.call(this))
This should allow you to do require("./ExportToJavaScript.js").interact("42") from Node, and there is probably an equivalent to use from a browser.

Yes, you can extract any function by hand.
Build a module as follows:
module Main
import Data.String
f: Double -> Double
f x = x + 1
interact: String -> String
interact s = let x = parseDouble s in
case x of
Nothing => "NaN"
Just x => show (f x)
main: IO ()
main = do
s <- getLine
putStrLn (interact s)
Compile it as follows:
% idris --codegen javascript --output Main.js Main.idr
A file called Main.js will be created. There will be several megabytes of more or less inscrutable JavaScript code, just as you say.
Edit this file by hand and edit it similarly to this:
--- Resistors.js
+++ Resistors-default.js
## -1,7 +1,5 ##
"use strict";
-(function(){
-
const $JSRTS = {
throw: function (x) {
throw x;
## -36130,7 +36128,3 ##
}
}
}
-
-
-$_0_runMain();
-}.call(this))
Now notice this JS file has comments in it marking the JS functions with their Idris names. For instance, corresponding to our interact function there will be located this JS function:
// Main.interact
function Main__interact($_0_arg){
const $_1_in = Data__String__parseDouble($_0_arg);
if(($_1_in.type === 1)) {
const $cg$3 = Main__bestMatch_39_($_1_in.$1, Main__manyResistors_39_());
let $cg$2 = null;
$cg$2 = $cg$3.$1;
return Prelude__Show__Main___64_Prelude__Show__Show_36_Schema_58__33_show_58_0($cg$2);
} else {
return "NaN";
}
}
If you attach this JS file to a web page as a script, you may then open JS console in a browser and interact with your Idris functions, like this:
Main__interact("10")
"11"
Hope this helps!

How to get the JavaScript Object from the given string?

How to get the Object from a string?
I written a localStorage util, in it there are get and set methods.
in the set method:
function fnGet(name){
var getVal=storage.getItem(name);
if(getVal==null){
return console.log('the localstorage did\'t have'+name);
}
if((getVal.split(':-:')).lenght>1){
return eval('('+getVal.split(':-:')[0]+')');
}
return getVal.split(':-:')[0];
}
You can ignore the :-:, it is the separator of the saved data and timestamp.
there is a problem, if the data is stored a JavaScript Object, such like this:
'{"pk":1,"username":"test01","email":"","first_name":"","last_name":""}:-:1521381469910'
when I use the get method, it will become like this:
'{"pk":1,"username":"test01","email":"","first_name":"","last_name":""}'
How can I get to the JavaScript Object?
How to optimize my get method?

JSON.parse on your response from the store. localStorage stores everything as strings so you would need to stringify the object at first, as Im supposed you do as otherwise you wouldnt have been able to save it to the store.
Then to retrieve it you would need to parse it to get the javascript object again.

Two things:
Use JSON.parse() instead of eval; it's not only safer, but more descriptive as to what your intent is. Note: this requires using JSON.stringify() on the data being saved in localStorage
Correct your spelling errors; you would never get to the eval/parser block because your length was spelled "lenght"
function fnGet(name) {
let getVal = storage.getItem(name)
if (getVal == null) {
return console.log(`the localstorage did't have: ${name}`);
}
let val = getVal.split(':-:'); // for performance cache the split
if (val.length > 1) { // Spelling error: "lenght" -> length
return JSON.parse(val[0]);
}
return val[0];
}

LocalStorage saves the data stringified. So you should use JSON.parse(yourVariable) to get the data back as JSON

function fnGet(name) {
var getVal = storage.getItem(name);
if (getVal == null) {
return console.log('the localstorage did\'t have' + name);
}
if ((getVal.split(':-:')).lenght > 1) {
return eval('(' + JSON.parse(getVal.split(':-:')[0]) + ')');
}
return getVal.split(':-:')[0];
}
all you needed was JSON.parse which takes a string as an argument and if its a valid object string ,returns an object else throws an error

How to get number of request query parameters in express.js?

At the moment I have to check every potentially existing parameter separately.
if (req.query.param1 != undefined ) {
}
if (req.query.param2 != undefined ) {
}
if (req.query.param3 != undefined ) {
}
...

To get all query parameter:
Object.keys(req.query)
To get number of all params:
Object.keys(req.query).length
Then you can iterate through all parameters:
for(p in req.query) {
//... do something
}
UPD:
surround your request with quotes to make right query
curl -X GET "localhost:9090/mypath?param1=123&param2=321"
without quotes the & in terminal makes the command run in the background.

If you hit /mypath?param1=5&param2=10, then the request.query will yield {param1: 5, param2:10}.
This means that the request.query is a JavaScript object with the key as the name of the param, and value as the value of the param. Now you can do anything with it as you want: Find the length or iterate over it as follows:
for (var key in request.query) {
if (request.query.hasOwnProperty(key)) {
alert(key + " -> " + request.query[key]);
}
}
Finding only the length might not work for you that well because you may have param1 and param3, with param2 missing. Iterating will be better IMO.

You want the number of non-undefined params right?
It is as simple as this;
var no = 0;
for (var key in req.query) {
if(req.query[key]) no++;
}

rx: unfold array to multiple streams

I have a stream holding an array, each element of which has an id. I need to split this into a stream per id, which will complete when the source stream no longer carries the id.
E.g. input stream sequence with these three values
[{a:1}, {b:1}] [{a:2}, {b:2}, {c:1}] [{b:3}, {c:2}]
should return three streams
a -> 1 2 |
b -> 1 2 3
c -> 1 2
Where a has completed on the 3rd value, since its id is gone, and c has been created on the 2nd value, since its id has appeared.
I'm trying groupByUntil, a bit like
var input = foo.share();
var output = input.selectMany(function (s) {
return rx.Observable.fromArray(s);
}).groupByUntil(
function (s) { return s.keys()[0]; },
null,
function (g) { return input.filter(
function (s) { return !findkey(s, g.key); }
); }
)
So, group by the id, and dispose of the group when the input stream no longer has the id. This seems to work, but the two uses of input look odd to me, like there could a weird order dependency when using a single stream to control the input of the groupByUntil, and the disposal of the groups.
Is there a better way?
update
There is, indeed, a weird timing problem here. fromArray by default uses the currentThread scheduler, which will result in events from that array being interleaved with events from input. The dispose conditions on the group are then evaluated at the wrong time (before the groups from the previous input have been processed).
A possible workaround is to do fromArray(.., rx.Scheduler.immediate), which will keep the grouped events in sync with input.

yeah the only alternative I can think of is to manage the state yourself. I don't know that it is better though.
var d = Object.create(null);
var output = input
.flatMap(function (s) {
// end completed groups
Object
.keys(d)
.filter(function (k) { return !findKey(s, k); })
.forEach(function (k) {
d[k].onNext(1);
d[k].onCompleted();
delete d[k];
});
return Rx.Observable.fromArray(s);
})
.groupByUntil(
function (s) { return s.keys()[0]; },
null,
function (g) { return d[g.key] = new Rx.AsyncSubject(); });