My question is how can I make use of WebSocket data in general. im using this package https://www.npmjs.com/package/binance to get binance live tickers. what i want to do is take that live data and compare it to some other exchanges data which im calling using axios. but all i get is continous stream of binance data and donno how to make use of it, i am trying to run function inside the stream but that does not seem to help as the data stream is continuous and axios request takes time to return. i can add more if my question is incomplete.
The binance package only supports websocket streams, but REST API (either directly or using a different package) will help you achieve your goal more easily.
You can have two functions - each for retrieving price form the exchange, and then compare the price from the different sources.
const axios = require('axios');
const getBinanceBtcPrice = async () => {
// binance allows retrieving a single symbol
// if you omit the param, it would return prices of all symbols
// and you'd have to loop through them like in the WazirX example
const response = await axios.get('https://api.binance.com/api/v3/avgPrice?symbol=BTCUSDT');
return response.data.price;
}
const getWazirxBtcPrice = async () => {
// WazirX only returns all symbol data
const response = await axios.get('https://api.wazirx.com/api/v2/market-status');
// you need to loop through all of them, until you find the desired pair
for (let market of response.data.markets) {
// in your case `btc` and `usdt` (mind the lowecase letters)
if (market.baseMarket === 'btc' && market.quoteMarket === 'usdt') {
// mind that WazirX only returns price in full integer,
// it does not return floating numbers
return market.last;
}
}
// did not find the combination
return null;
}
const run = async () => {
const binancePrice = await getBinanceBtcPrice();
const wazirxPrice = await getWazirxBtcPrice();
console.log(binancePrice);
console.log(wazirxPrice);
}
run();
Related
How to filter GET request by type https:// url / data?sort_by=type
using Next API?
I tried to make
export const getServerSideProps = async () => {
const result = await fetch(http://localhost:3000/api/data?type=hobbies); a selection but nothing, it returns a regular array with data
I know there are similar questions to this on stack overflow but thus far none have been able to help me get my code working.
I have a function that takes an id, and makes a call to firebase firestore to get all the documents in a "feedItems" collection. Each document contains two fields, a timestamp and a post ID. The function returns an array with each post object. This part of the code (getFeedItems below) works as expected.
The problem occurs in the next step. Once I have the array of post ID's, I then loop over the array and make a firestore query for each one, to get the actual post information. I know these queries are asynchronous, so I use Promise.all to wait for each promise to resolve before using the final array of post information.
However, I continue to receive "undefined" as a result of these looped queries. Why?
const useUpdateFeed = (uid) => {
const [feed, setFeed] = useState([]);
useEffect(() => {
// getFeedItems returns an array of postIDs, and works as expected
async function getFeedItems(uid) {
const docRef = firestore
.collection("feeds")
.doc(uid)
.collection("feedItems");
const doc = await docRef.get();
const feedItems = [];
doc.forEach((item) => {
feedItems.push({
...item.data(),
id: item.id,
});
});
return feedItems;
}
// getPosts is meant to take the array of post IDs, and return an array of the post objects
async function getPosts(items) {
console.log(items)
const promises = [];
items.forEach((item) => {
const promise = firestore.collection("posts").doc(item.id).get();
promises.push(promise);
});
const posts = [];
await Promise.all(promises).then((results) => {
results.forEach((result) => {
const post = result.data();
console.log(post); // this continues to log as "undefined". Why?
posts.push(post);
});
});
return posts;
}
(async () => {
if (uid) {
const feedItems = await getFeedItems(uid);
const posts = await getPosts(feedItems);
setFeed(posts);
}
})();
}, []);
return feed; // The final result is an array with a single "undefined" element
};
There are few things I have already verified on my own:
My firestore queries work as expected when done one at a time (so there are not any bugs with the query structures themselves).
This is a custom hook for React. I don't think my use of useState/useEffect is having any issue here, and I have tested the implementation of this hook with mock data.
EDIT: A console.log() of items was requested and has been added to the code snippet. I can confirm that the firestore documents that I am trying to access do exist, and have been successfully retrieved when called in individual queries (not in a loop).
Also, for simplicity the collection on Firestore currently only includes one post (with an ID of "ANkRFz2L7WQzA3ehcpDz", which can be seen in the console log output below.
EDIT TWO: To make the output clearer I have pasted it as an image below.
Turns out, this was human error. Looking at the console log output I realised there is a space in front of the document ID. Removing that on the backend made my code work.
I have an array of objects which some of the items within it has a "logo" object containing a key of an image stored on a S3 Bucket.
This is an optional field, so sometimes it doesn't exist in the object. To retrieve the image, I am using the Storage api from AWS-Amplify. This Storage function returns me a promise and here is my first problem.
const fetchUsers = async () => {
setLoading(true);
const formatedData = [];
const { data } = await API.graphql(graphqlOperation(listUsers));
console.log({ data });
data.listUsers.items.map(async user => {
if (user.logo) {
console.log(Storage.get());
const image = await Storage.get(user.logo.key);
user.image = image;
formatedData.push(user);
} else {
formatedData.push(user);
}
})
}
While I am waiting the response for the response from the Storage.get function, the loop keep running, which is modifying the original sorting that came from the data array.
This is causing all the entries that has images are appearing at last on the array.
So given this, I assume that
There's a way to prevent the loop to run while I am still waiting for the response
There's a better/more efficient way to get these images from S3.
Any thoughts how to make it run more smoothly ?
I'm using the popular npm package cheerio with request to retrieve some table data.
Whilst I can retrieve and parse the table from a single page easily, I'd like to loop over / process multiple pages.
I have tried wrapping inside loops / various utilities offers by the async package but can't figure this one out. In most cases, node runs out of memory.
current code:
const cheerio = require('cheerio');
const axios = require("axios");
var url = someUrl;
const getData = async url => {
try {
const response = await axios.get(url);
const data = response.data;
const $ = cheerio.load(data);
const announcement = $(`#someId`).each(function(i, elm) {
console.log($(this).text())
})
} catch (error) {
console.log(error);
}
};
getData(url); //<--- Would like to give an array here to fetch from multiple urls / pages
My current approach, after trying loops, is to wrap this inside another function with a callback param. However no success yet and is getting quite messy.
What is the best way to feed an array to this function?
Assuming you want to do them one at a time:
; (async() => {
for(let url of urls){
await getData(url)
}
})()
Have you tried using Promise.all (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all)?
For loops are usually a bad idea when dealing with asynchronous calls. It depends how many calls you want to make but I believe this could be enough. I would use an array of promises that fetch the data and map over the results to do the parsing.
Goal: Front-end of application allows users to select files from their local machines, and send the file names to a server. The server then matches those file names to files located on the server. The server will then return a list of all matching files.
Issue: This works great if you a user select less than a few hundred files, otherwise it can cause long response times. I do not want to limit the number of files a user can select, and I don't want to have to worry about the http requests timing out on the front-end.
Sample code so far:
//html on front-end to collect file information
<div>
<input (change)="add_files($event)" type="file" multiple>
</div>
//function called from the front-end, which then calls the profile_service add_files function
//it passes along the $event object
add_files($event){
this.profile_service.add_files($event).subscribe(
data => console.log('request returned'),
err => console.error(err),
() => //update view function
);
}
//The following two functions are in my profile_service which is dependency injected into my componenet
//formats the event object for the eventual query
add_files(event_obj){
let file_arr = [];
let file_obj = event_obj.target.files;
for(let key in file_obj){
if (file_obj.hasOwnProperty(key)){
file_arr.push(file_obj[key]['name'])
}
}
let query_obj = {files:title_arr};
return this.save_files(query_obj)
}
//here is where the actual request to the back-end is made
save_files(query_obj){
let payload = JSON.stringify(query_obj);
let headers = new Headers();
headers.append('Content-Type', 'application/json');
return this.http.post('https://some_url/api/1.0/collection',payload,{headers:headers})
.map((res:Response) => res.json())
}
Possible Solutions:
Process requests in batches. Re-write the code so that the profile-service is only called with 25 files at a time, and upon each response call profile-service again with the next 25 files. If this is the best solution, is there an elegant way to do this with observables? If not, I will use recursive callbacks which should work fine.
Have the endpoint return a generic response immediately like "file matches being uploaded and saved to your profile". Since all the matching files are persisted to a db on the backend, this would work and then I could have the front-end query the db every so often to get the current list of matching files. This seem ugly, but figured I'd throw it out there.
Any other solutions are welcome. Would be great to get a best-practice for handling this type of long-lasting query with angular2/observables in an elegant way.
I would recommend that you break up the number of files that you search for into manageable batches and then process more as results are returned, i.e. solution #1. The following is an untested but I think rather elegant way of accomplishing this:
add_files(event_obj){
let file_arr = [];
let file_obj = event_obj.target.files;
for(let key in file_obj){
if (file_obj.hasOwnProperty(key)){
file_arr.push(file_obj[key]['name'])
}
}
let self = this;
let bufferedFiles = Observable.from(file_arr)
.bufferCount(25); //Nice round number that you could play with
return bufferedFiles
//concatMap will make sure that each of your requests are not executed
//until the previous completes. Then all the data is merged into a single output
.concatMap((arr) => {
let payload = JSON.stringify({files: arr});
let headers = new Headers();
hearders.append('Content-Type', 'application/json');
//Use defer to make sure because http.post is eager
//this makes it only execute after subscription
return Observable.defer(() =>
self.post('https://some_url/api/1.0/collection',payload, {headers:headers})
}, resp => resp.json());
}
concatMap will keep your server from executing more than whatever the size of your buffer is, by preventing new requests until the previous one has returned. You could also use mergeMap if you wanted them all to be executed in parallel, but it seems the server is the resource limitation in this case if I am not mistaken.
I'd suggest to use websocket connections instead because they don't time out.
See also
- https://www.npmjs.com/package/angular2-websocket
- http://mmrath.com/post/websockets-with-angular2-and-spring-boot/
- http://www.html5rocks.com/de/tutorials/websockets/basics/
An alternative approach would be polling, where the client makes repeated requests in a defined interval to get the current processing state from the server.
To send multiple requests and waiting for all of them to complete
getAll(urls:any[]):Observable {
let observables = [];
for(var i = 0; i < items.length; i++) {
observables.push(this.http.get(urls[i]));
}
return Observable.forkJoin(observables);
}
someMethod(server:string) {
let urls = [
'${server}/fileService?somedata=a',
'${server}/fileService?somedata=b',
'${server}/fileService?somedata=c'];
this.getAll(urls).subscribe(
(value) => processValue(val),
(err) => processError(err),
() => onDone());
}