I've created a data service that gets data sets from an API, but I'd like to have it first cache it locally and check if the same data is already available (nevermind the stale data factor... I'll deal with that next). Here's my code:
getData(url, use_cache = true) {
// Http Fetch Client to retreive data (GET)
let cache_index = this.cache.findIndex(r => { return r.url === url; });
if ((use_cache) && (cache_index > -1) && (this.cache[cache_index].data.length)) {
// Use cached data (available)
console.log("Found cached data!", this.cache[cache_index].data);
//
// I think this next line is the problem... need to return a promise???
//
return this.cache[cache_index].data;
} else {
console.log("Retrieving records from " + url);
return this.httpClient.fetch(url, {
credentials: 'include'
}).then(response => {
// Old statement was simple...
// return response.json();
// New method seems to be working because it's saving the data into the cache
return response.json().then(result => {
this.cache.push({'url': url, 'data': result});
// Not sure why I need this next line, but I do.
return result;
});
});
}
}
It works fine to retrieve the data the first time, and even on the second call I can see (from the console log) that it finds the correct cached data, but I'm getting an error that I believe is related to promises, which is not in my area of expertise yet.
Error message:
ERROR [app-router] TypeError: this.core.getData(...).then is not a function
This error is actually in my viewmodel's caller, which looks like this:
getAccounts() {
this.core.getData('/accounting/account/all').then(response => {
this.accounts = response;
});
}
I guess since when the data is cached, instead of returning a promise it's actually returning the data, and there's no .then method on the raw data.
I suspect I need to either create a fake promise (even though it's not an async transaction) to return when the data is cached or improve the way I'm calling this method from my data service (or returning the data).
Any ideas on how to fix this current problem? Any free advice on this whole topic as it relates to Aurelia?
I guess since when the data is cached, instead of returning a promise it's actually returning the data, and there's no .then method on the raw data.
Yes.
I suspect I need to either create a fake promise (even though it's not an async transaction) to return when the data is cached
Possible (using Promise.resolve), but no.
…or improve the way I'm calling this method from my data service (or returning the data).
No, for sure you shouldn't need that.
Instead, there's a much simpler solution: cache the promise object itself, and return the same promise from every call for that url!
getData(url, use_cache = true) {
// Http Fetch Client to retreive data (GET)
if (use_cache && url in this.cache)
return this.cache[url];
else
return this.cache[url] = this.httpClient.fetch(url, {
credentials: 'include'
}).then(response => response.json());
}
This has the additional benefit that you'll never have two parallel requests for the same resource - the request itself is cached, not only the arrived result. The only drawback is that you also cache errors, if you want to avoid that and retry on subsequent calls then you have to drop the cache on rejections.
Related
In my sveltekit app I make AJAX calls to my api endpoints. For example:
+page.svelte
<script>
async function get_card() {
const url = '/api/card/?category=' + $page.params.slug;
const response = await fetch(url, {
method: 'GET',
})
const card = await response.json();
return card;
}
</script>
In the browser javascript console I get this warning:
Loading /api/card/?category=Neurology using `window.fetch`.
For best results, use the `fetch` that is passed to your `load`
function: https://kit.svelte.dev/docs/load#making-fetch-requests
But as far as I can tell, that fetch function is only accessible to me on the server, and I do not see a way to use it in a script that may run on the client (such as +page.svelte). I tried passing the function as part of the data object from load:
+layout.server.js
export const load = async ({ fetch, locals }) => {
return {
email: locals.user.email,
group: locals.user.group,
fetch: fetch
}
}
But, not surprisingly, that does not work since the function is not serializable.
Am I Doing It Wrong™, or should I just ignore the warning?
fetch is originally a browser API and SvelteKit defines it on the server as well, if it does not exist. The warning is there to tell you that you are creating another round trip to the server (one for the page and one for the data) when you possibly could have loaded the data on the server so it could be transmitted as part of the page (during server-side rendering).
If the code of your function is not executed right away, then this is a false positive (recent issue on this). I.e. if the data should be requested at a significantly later point, there is no way to bundle the request with the page.
(You are definitely not meant to pass on the fetch of load, you are supposed to use it to get the data.)
I am trying to speed up the network critical path on a website, and find out about the great <link rel=preload. So I try to anticipate the call that my single page application do as soon as the JS kicks in, I have put in my index.html
<link rel="preload" href="/api/searchItems" as="fetch" />
Then as the JS starts I make the same call with the help of the axios library:
await axios.get(`/api/searchItems`, { params: queryParams });
I would expect to see the call of Axios returning instantly the preloaded JSON file but instead, I see this:
As you can see the same call is loaded twice.
What I am doing wrong?
EDIT: I have added cache-control: public and nothing changes.
EDIT2: I also tried this code instead of axios:
let data = await fetch('/api/searchItems')
.then(response => {
if (response.ok) {
return response.json();
}
throw new Error('HTTP error ' + response.status);
})
.catch(() => {
data = null; // Just clear it and if it errors again when
// you make the call later, handle it then
});
And nothing change
Three options for you:
It looks like your response has headers making it uncacheable for some reason. You may be able to fix it so it's cacheable.
Use a service worker.
Another approach, if this is really critical path, is to have some inline JavaScript that actually does the call and modify the code that will do the call later to look to see if the previous result is available, like this:
let firstLoad = fetch("/api/searchItems")
.then(response => {
if (response.ok) {
return response.json();
}
throw new Error("HTTP error " + response.status);
})
.catch(() => {
firstLoad = null; // Just clear it and if it errors again when
// you make the call later, handle it then
});
(I'm using fetch there because you may want to do it before you've loaded axios.)
Then in the code that wants this data:
(firstLoad || axios.get("/api/searchItems").then(response => response.data))
.then(/*...*/)
.catch(/*...*/);
firstLoad = null;
If the content requires revalidation (and you're using no-cache, so it does¹), #2 and #3 have the advantage of not requiring a second request to the server.
¹ From MDN:
no-cache
The response may be stored by any cache, even if the response is normally non-cacheable. However, the stored response MUST always go through validation with the origin server first before using it...
(my emphasis)
Im' trying to understand using promises with Google Cloud Functions a bit better. I just learned about the 'finally' method on promises, which is called after all promises in the chain are fully resolved or rejected. In a http function is it good practice to put response.send() inside of the finally method?
The below code uses request-promise-native for the http request. In the first .then() I call parseSchedule, which uses the cheerio web scraping api to loop through some data and on a website, and add it to the scheduledGames array (synchronously, I think).
I return from that and the then log that data to the console in writeDB, but one thing I noticed is that I see response.send() log 'execution finished' before I see the data from scheduleGames in the log. Is that correct?
Should I be using the 'finally' block like this?
Thanks,
const options = {
uri: 'https://www.cbssports.com/nba/schedule/' + urlDate,
Connection: 'keep-alive',
transform: function (body) {
return cheerio.load(body);
}
};
return request(options)
.then(parseSchedule)
.then(writeSchedule)
.catch((err) => console.log("there was an error: " + err))
.finally(res.send("execution finished"));
function parseSchedule($){
const scheduledGames = [];
$('tbody').children('tr').each((i, element) => {
const gameTime = $(element).children('td').eq(2).find('a').text()
const scheduledGame = { gameTime: gameTime};
scheduledGames.push(scheduledGame);
});
return scheduledGames;
}
function writeDB(scheduledGames){
console.log(scheduledGames);
}
}
It typically makes more sense to send a success response at the time in the promise chain when everything is successful, or send an error response in a catch handler. If you do these two things, it doesn't make sense to use finally at all, since success and error are the only two cases you really need to handle. Unless you have some special case, stick to just success and error.
I am building a web app using laravel and vuejs. I have made a axios get request to get a list of users .
I am getting a Promise object, and from what i have read. Reason for getting a promise object is because it's an async request.
I have tried .then() to get data part of the response. But i am getting a huge script instead of desired data.
axios......then(function(response){
console.log(response.data);
})
Initially what i did was
var res = axios.get('/allUsers');
console.log(res)
That time i came to know about promise object and read about.
When i checked network in dev tools, status code is 200 and i can see list of users. So i guess my request is successfully completed.
What should be done to get the list of the users. That list i will be using to update my UI.
Depending on what you're getting back for data there are a few ways to handle this. You may need to convert the data after the you get receive the response.
axios.get('some_url')
.then(res => res.json())
.then(data => {
// do something with the data
}).catch(err) {
conosole.error(err);
}
if you're seeing the data come through properly in the response and you're getting what you need without doing that then just do
axios.get('some url').then(res => {
// do something in here with the data here
})
also make sure you're getting back json if that's what you're looking for. check your response to see if its html or json because they can be handled a bit differently
as an "Edit" you could also handle this with async await so you dont end up in callback hell
async function fetchData() {
try {
const res = await axios.get('some url');
// next step might not be necessary
const data = await res.json();
// do something with the data
console.log(data); // if converting it was necessary
console.log(res); // if not converting
} catch (err) {
console.error(err);
}
}
The situation is that I want to store some information in a DynamoDB database, and then send a notification only if I am sure that this information has been stored successfully. Here's what my code looks like:
const putObj = Bluebird.promisify(docClient.put, { context: docClient })
// ... a promise chain collecting information...
const params = {
TableName: 'my_table',
Item: {
name: itemName
}
}
return putObj(params)
.then((data) => {
if (Ramda.equals(data, {})) {
// ... send notification here
.catch((err) => {
throw err
})
I assume that if there is an error storing the data, this will be caught in the catch block and the notification will not be sent. However, I'm seeing in some cases that notifications are being sent despite the fact that the respective information has not been stored in the database. This only happens infrequently - in almost all cases the code functions as desired.
Any ideas about why/how this could be happening, and what I can do to stop it. DynamoDB doesn't return any useful information into the .then - it is only an empty object in case of success, and I'm already checking this before sending the notification.
Are you using 'eventually consistent' or 'strongly consistent' reads?
When you read data from a DynamoDB table, the response might not
reflect the results of a recently completed write operation. The
response might include some stale data. If you repeat your read
request after a short time, the response should return the latest
data.
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadConsistency.html
You may need to retry your read, or else change the read consistency value.