What is best practice to remove duplicate/redundant API requests? - javascript

So I am working in a React platform that has data that updates every second(I would like to move to web-sockets but its currently only supports gets).Currently, each component makes a fetch request for itself to get the data for the widget. Because the fetch requests are built into the widgets there are redundant api requests for the same data. I am looking for a possible better solution to remove these redundant api requests.
The solution I came up with uses what I call a data service that checks for any subscription to data sources then makes those api calls and places the data in a redux state for the components to then be used. I am unsure if this is the best way to go about handling the issue I am trying to avoid. I don't like how I need an interval to be run every second the app is running to check if there are "subscriptions". I am unsure if thats the correct way to go about it. With this solution I don't duplicate any requests and can add or remove a subscription without affecting other components.
One more thing, the id can change and will change what data I recieve
Here is a simplified version of how I am handling the service.
const reduxState = {
id: "specific-id",
subscriptions: {
sourceOne: ["source-1-id-1", "source-1-id-2", "source-1-id-3"],
sourceTwo: ["source-2-id-1", "source-one-id-2"],
},
data: {
sourceOne: { id: "specific-id", time: "milliseconds", data: "apidata" },
sourceTwo: { id: "specific-id", time: "milliseconds", data: "apidata" },
},
};
const getState = () => reduxState; //Would be a dispatch to always get current redux state
const dataService = () => {
const interval = setInterval(() => {
const state = getState();
if (state.subscriptions.sourceOne.length > 0)
fetchSourcOneAndStoreInRedux();
if (state.subscriptions.sourceTwo.length > 0)
fetchSourceTwoAndStoreInRedux();
}, 1000);
};
const fetchSourcOneAndStoreInRedux = (id) =>{
return async dispatch => {
try {
const res = await axios.get(`/data/one/${id}`)
dispatch(setSourceOneDataRedux(res.data))
} catch (err) {
console.error(err)
}
}
}
I am building my components to only show data from the correct id.

Here is a simple working example of a simple "DataManager" that would achieve what you are looking for.
class DataManager {
constructor(config = {}) {
this.config = config;
console.log(`DataManager: Endpoint "${this.config.endpoint}" initialized.`);
if (this.config.autostart) { // Autostart the manager if autostart property is true
this.start();
}
}
config; // The config object passed to the constructor when initialized
fetchInterval; // The reference to the interval function that fetches the data
data; // Make sure you make this state object observable via MOBX, Redux etc so your component will re-render when data changes.
fetching = false; // Boolean indicating if the APIManager is in the process of fetching data (prevent overlapping requests if response is slow from server)
// Can be used to update the frequency the data is being fetched after the class has been instantiated
// If interval already has been started, stop it and update it with the new interval frequency and start the interval again
updateInterval = (ms) => {
if (this.fetchInterval) {
this.stop();
console.log(`DataManager: Updating interval to ${ms} for endpoint ${this.config.endpoint}.`);
this.config.interval = ms;
this.start();
} else {
this.config.interval = ms;
}
return this;
}
// Start the interval function that polls the endpoint
start = () => {
if (this.fetchInterval) {
clearInterval(this.fetchInterval);
console.log(`DataManager: Already running! Clearing interval so it can be restarted.`);
}
this.fetchInterval = setInterval(async () => {
if (!this.fetching) {
console.log(`DataManager: Fetching data for endpoint "${this.config.endpoint}".`);
this.fetching = true;
// const res = await axios.get(this.config.endpoint);
// Commented out for demo purposes but you would uncomment this and clear the anonymous function below
const res = {};
(() => {
res.data = {
dataProp1: 1234,
dataProp2: 4567
}
})();
this.fetching = false;
this.data = res.data;
} else {
console.log(`DataManager: Waiting for pending response for endpoint "${this.config.endpoint}".`);
}
}, this.config.interval);
return this;
}
// Stop the interval function that polls the endpoint
stop = () => {
if (this.fetchInterval) {
clearInterval(this.fetchInterval);
console.log(`DataManager: Endpoint "${this.config.endpoint}" stopped.`);
} else {
console.log(`DataManager: Nothing to stop for endpoint "${this.config.endpoint}".`);
}
return this;
}
}
const SharedComponentState = {
source1: new DataManager({
interval: 1000,
endpoint: `/data/one/someId`,
autostart: true
}),
source2: new DataManager({
interval: 5000,
endpoint: `/data/two/someId`,
autostart: true
}),
source3: new DataManager({
interval: 10000,
endpoint: `/data/three/someId`,
autostart: true
})
};
setTimeout(() => { // For Demo Purposes, Stopping and starting DataManager.
SharedComponentState.source1.stop();
SharedComponentState.source1.updateInterval(2000);
SharedComponentState.source1.start();
}, 10000);
// Heres what it would look like to access the DataManager data (fetched from the api)
// You will need to make sure you pass the SharedComponentState object as a prop to the components or use another React mechanism for making that SharedComponentState accessible to the components in your app
// Accessing state for source 1: SharedComponentState.source1.data
// Accessing state for source 2: SharedComponentState.source2.data
// Accessing state for source 3: SharedComponentState.source3.data
Basically, each instance of the DataManager class is responsible for fetching a different api endpoint. I included a few other class methods that allow you to start, stop and update the polling frequency of the DataManager instance.

Related

cron jobs on firebase cloud functions

I have two cloud functions
One cloud function is for set or updating existing scheduled job
Canceling an existing scheduled job
I am using import * as the schedule from 'node-schedule'; to manage Scheduling a jobs
The problem is cus createJob function is triggered and jobId is returned, but later when I triger cancelJob function all prev scheduled cron jobs do not exist cus node-schedule lives in memory and I can't access the jobs:
this will return empty object: const allJobs = schedule.scheduledJobs;
Does anyone have some solution on this situation?
UTILS this is the main logic that is called when some of my cloud functions are triggered
enter code here
// sendgrid
import * as sgMail from '#sendgrid/mail';
import * as schedule from 'node-schedule';
sgMail.setApiKey(
'apikey',
);
import {
FROM_EMAIL,
EMAIL_TEMPLATE_ID,
MESSAGING_SERVICE_SID,
} from './constants';
export async function updateReminderCronJob(data: any) {
try {
const {
to,
...
} = data;
const message = {
to,
from: FROM_EMAIL,
templateId: EMAIL_TEMPLATE_ID,
};
const jobReferences: any[] = [];
// Stop existing jobs
if (jobIds && jobIds.length > 0) {
jobIds.forEach((j: any) => {
const job = schedule.scheduledJobs[j?.jobId];
if (job) {
job.cancel();
}
});
}
// Create new jobs
timestamps.forEach((date: number) => {
const job = schedule.scheduleJob(date, () => {
if (selectedEmail) {
sgMail.send(message);
}
});
if (job) {
jobReferences.push({
jobId: job.name,
});
}
});
console.warn('jobReferences', jobReferences);
return jobReferences;
} catch (error) {
console.error('Error updateReminderCronJob', error);
return null;
}
}
export async function cancelJobs(jobs: any) {
const allJobs = schedule.scheduledJobs;
jobs.forEach((job: any) => {
if (!allJobs[job?.jobId]) {
return;
}
allJobs[job.jobId].cancel();
});
}
node-schedule will not work effectively in Cloud Functions because it requires that the scheduling and execution all be done on a single machine that stays running without interruption. Cloud Functions does not fully support this behavior, as it will dynamically scale up and down to zero the number of machines servicing requests (even if you set min instances to 1, it may still reduce your active instances to 0 in some cases). You will get unpredictable behavior if you try to schedule this way.
The only way you can get reliable scheduling using Cloud Functions is with pub/sub functions as described in the documentation. Firebase scheduled functions make this a bit easier by managing some of the details. You will not be able to dynamically control repeating jobs, so you will need to build some way to periodically run a job and check to see if it should run at that moment.

Simulate actioncable websocket receive in webdriverIo

Is there a way during webdriverio runtime to simulate an actioncable receive?
I am using a fork of the package action-cable-react called actioncable-js-jwt for Rails actioncable js connections. Both of these packages are no longer maintained, but actioncable-js-jwt was the only actioncable for react package I could find that supported jwt authentication. I am building an app in my company's platform and jwt authentication is required.
The problem I am running into is that I have a react component which dispatches a redux action to call an api. The api returns a 204, and the resulting data is broadcasted out from Rails to be received by the actioncable connection. This triggers a dispatch to update the redux store with new data. The component does actions based on new data compared to the initial value on component load, so I cannot simply just set initial redux state for wdio - I need to mock the actioncable receive happening.
The way the actioncable subscription is created is:
export const createChannelSubscription = (cable, receivedCallback, dispatch, channelName) => {
let subscription;
try {
subscription = cable.subscriptions.create(
{ channel: channelName },
{
connected() {},
disconnected(res) { disconnectedFromWebsocket(res, dispatch); },
received(data) { receivedCallback(data, dispatch); },
},
);
} catch (e) {
throw new Error(e);
}
return subscription;
};
The receivedCallback function is different for each channel, but for example the function might look like:
export const handleUpdateRoundLeaderWebsocket = (data, dispatch) => {
dispatch({ type: UPDATE_ROUNDING_LEADER, round: data });
};
And the redux state is used here (code snippets):
const [currentLeader, setCurrentLeader] = useState(null);
const userId = useSelector((state) => state.userId);
const reduxStateField = useSelector((state) => state.field);
const onChange = useCallback((id) => {
if (id !== currentLeader) {
if (id !== userId && userId === currentLeader) {
setShow(true);
} else {
setCurrentLeader(leaderId);
}
}
}, [currentLeader, userId]);
useEffect(() => {
onChange(id);
}, [reduxStateField.id, onChange]);
Finally, my wdio test currently looks like:
it('has info dialog', () => {
browser.url('<base-url>-rounding-list-view');
$('#my-button').click();
$('div=Continue').click();
// need new state after the continue click
// based on new state, might show an info dialog
});
Alternatively, I could look into manually updating redux state during wdio execution - but I don't know how to do that and can't find anything on google except on how to provide initial redux state.

How to create streaming endpoint without initial data fetch in redux toolkit

I need to use a streaming connection in redux-toolkit.
The example in docs is implemented using an initial fetch to retrieve the previous message history:
export const api = createApi({
baseQuery: fetchBaseQuery({ baseUrl: '/' }),
endpoints: (build) => ({
getMessages: build.query<Message[], Channel>({
// retrieve initial data
query: (channel) => `messages/${channel}`,
async onCacheEntryAdded(
arg,
{ updateCachedData, cacheDataLoaded, cacheEntryRemoved }
) {
// create a websocket connection when the cache subscription starts
const ws = new WebSocket('ws://localhost:8080')
try {
// wait for the initial query to resolve before proceeding
await cacheDataLoaded
// when data is received from the socket connection to the server,
// if it is a message and for the appropriate channel,
// update our query result with the received message
const listener = (event: MessageEvent) => {
const data = JSON.parse(event.data)
if (!isMessage(data) || data.channel !== arg) return
updateCachedData((draft) => {
draft.push(data)
})
}
ws.addEventListener('message', listener)
} catch {
// no-op in case `cacheEntryRemoved` resolves before `cacheDataLoaded`,
// in which case `cacheDataLoaded` will throw
}
// cacheEntryRemoved will resolve when the cache subscription is no longer active
await cacheEntryRemoved
// perform cleanup steps once the `cacheEntryRemoved` promise resolves
ws.close()
},
}),
}),
})
export const { useGetMessagesQuery } = api
My endpoint only admits streaming updates, but doesnt have an initial data response. How can I implement the streaming endpoint without the initial fetch?
What I have tried:
Removing the query property.
Returning undefined in the responseHandler of query.
One option here would be to use a queryFn that returns an initial empty data value (and thus doesn't make an API call to the server), such as:
streamSomeData: build.query({
queryFn: async () => ({data: null}),
onCacheEntryAdded: async (
arg,
{ updateCachedData, cacheDataLoaded, cacheEntryRemoved }
) => {
// streaming logic here
}
})

Fetch data from api every second

I have a react native app where i am fetching orders made by customer and display them in restaurant side in a specific screen, I am using fetch api to get the data,
so THE WORKFLOW is customer first place order and store it in database and in the restaurant side i have this function :
const loadData = async () => {
const response = await fetch(`${API_URL}/getActiveOrders?ID=${id}`);
const result = await response.json();
if (result.auth === true) {
setCurrentOrders(result.data)
} else {
setCurrentOrders([])
}
}
useEffect(() => {
const interval = setInterval(() => {
loadData();
}, 1000)
return () => clearInterval(interval)
}, [id]);
as in this function it runs every second and it makes api call to express server to fetch the data from database so i keep restaurant receiving orders without delay. But i noticed that the app is lagging when the interval set to 1 second and it keep making frequent calls to the express server.
my question: Is this the best approach to perform same as this scenario (fetching the orders the moment they been placed by the customer) or Is there a better way to do it without lagging as well as when fetching large data will the performance remain the same or there will be some issues?
If I had to guess what the cause of the lagging issue was it would be because you have a dependency on the useEffect hook. Each time the state updates it triggers another render and the effect runs again, and sets up another interval, and your app is doing this once every second.
I suggest removing the dependency and run the effect only once when the component mounts, and clearing the interval when unmounting.
useEffect(() => {
const interval = setInterval(() => {
loadData();
}, 1000)
return () => clearInterval(interval)
}, []);
If the id for the GET requests updates and you need the latest value, then use a React ref to store the id value and use this in the request URL.
const idRef = React.useRef(id);
useEffect(() => {
idRef.current = id;
}, [id]);
const loadData = async () => {
const response = await fetch(`${API_URL}/getActiveOrders?ID=${idRef.current}`);
const result = await response.json();
setCurrentOrders(result.auth ? result.data : []);
}
You should use websockets in this case. This is the best scenario to use websockets. Your scenario is just like a trading website.

Nuxt Composition API, updating 'state', not reflected on UI Template

I had a Nuxt.js application working with the options API. And with the new Nuxt3 coming out, I was trying to migrate things over to the supposedly 'better' alternative. So far i've had nothing but challenges, perhaps that's my lack of knowledge.
I'm building a basic E-Commerce platform with a component of
# products/_id.vue
<template>
<div>
{{ product }}
</div>
</template>
<script>
import {
defineComponent,
useFetch,
useStore,
useRoute,
ssrRef, reactive, watch
} from '#nuxtjs/composition-api'
export default defineComponent({
setup () {
const store = useStore()
const route = useRoute()
const loading = ref(false)
// LOAD PRODUCT FROM VUEX STORE IF ALREADY LOADED
const product = reactive(store.getters['products/loaded'](route.value.params.id))
// GET PAGE CONTENT
const { fetch } = useFetch(async () => {
loading.value = true
await store.dispatch('products/getOne', route.value.params.id)
loading.value = false
})
// WATCH, if a use navigates to another product, we need to watch for changes to reload
watch(route, () => {
if (route.value.params.id) {
fetch()
}
})
return {
loading
product
}
}
})
</script>
One thing I need to note, is, if the product gets a comment/rating, I want the UI to update with the products star rating, thus needing more reactivity.
I continue to get an undefined product var
Inside my VueX store I have my getters
loaded: state => (id) => {
try {
if (id) {
return state.loaded[id]
}
return state.loaded
} catch {
return {}
}
}
Looking for directions on how to get this to work, improve any of the code i've currently setup.
If you want to maintain reactive referece to your getter, then you have to create a computed property.
So, what you return from your setup function is
product: computed(() => getters['products/loaded'](route.value.params.id))
This will make sure that whenever the getter updates, your component will receive that update.
Also, if the product already exists, you should bail out of the fetch function. So that you do not make the extra API call.
And, finally, if there is an error, you could redirect to a 404 error page.
All in all, your setup function could look something like this
setup() {
const route = useRoute();
const { error } = useContext();
const { getters, dispatch } = useStore();
const loading = ref(false);
const alreadyExistingProduct = getters['products/loaded'](route.value.params.id);
const { fetch } = useFetch(async () => {
// NEW: bail if we already have the product
if (alreadyExistingProduct) return;
try {
loading.value = true;
await dispatch('products/getOne', route.value.params.id);
} catch {
// NEW: redirect to error page if product could not be loaded
error({ statusCode: 404 });
} finally {
loading.value = false;
}
});
watch(route, () => {
if (route.value.params.id) {
fetch();
}
});
return {
loading,
// NEW: computed property to maintain reactive reference to getter
product: computed(() => getters['products/loaded'](route.value.params.id)),
};
},
You will probably also run into this harmless issue FYI.

Categories

Resources