Develop Reference

Problem with React Query's Infinite Query, using Edamam API

I currently have some issues trying to add the infinite query feature to a recipes app I'm working on using Edamam API.
All the examples I have looked for (even React Query's documentation) implement the infinite scroll using a page/cursor number system... I understand this is the ideal way, but... Edamam API doesn't work this way with paginated queries.
Instead, the API has the following structure for each recipe query we look for (let's assume we are searching for "chicken", this would be the JSON structure):
from: 1,
to: 20,
count: 10000,
_links: {
next: {
href: "https://api.edamam.com/api/recipes/v2?q=chicken&app_key=APIKEYc&_cont=CHcVQBtNNQphDmgVQntAEX4BYldtBAAGRmxGC2ERYVJ2BwoVX3cVBWQSY1EhBQcGEmNHVmMTYFEgDQQCFTNJBGQUMQZxVhFqX3cWQT1OcV9xBB8VADQWVhFCPwoxXVZEITQeVDcBaR4-SQ%3D%3D&type=public&app_id=APPID"
title: "Next Page"
}
},
hits: [{}] ... (This is where the actual recipes are)
As you can see, there is no numbering system for paginated queries, instead, it's a whole URL and it's giving me a hard time since I'm also new to React Query.
I tried the following, but it just fetches the same data over and over again as I reach the bottom of the page:
const getRecipes = async ({ pageParam }) => {
try {
const path = pageParam
? pageParam
: `https://api.edamam.com/api/recipes/v2?q=${query}&app_id=${process.env.REACT_APP_APP_ID}&app_key=${process.env.REACT_APP_API_KEY}&type=public`;
const response = await axios.get(path);
return response.data;
} catch (error) {
console.log(error);
}
const { ref, inView } = useInView();
useEffect(() => {
inView && fetchNextPage();
}, [inView]);
const {
data,
isFetching,
isFetchingNextPage,
error,
status,
hasNextPage,
fetchNextPage,
} = useInfiniteQuery(
["recipes", query],
({ pageParam = "" }) => getRecipes(pageParam),
{
getNextPageParam: (lastPage) => lastPage._links.next.href,
}
);
Since the next page param is a whole URL, I just say that IF there is a pageParam, then use that URL for the request, if not, then do a normal request using the query value the user is searching for.
Please help!

Since the next page param is a whole URL, I just say that IF there is a pageParam, then use that URL for the request, if not, then do a normal request using the query value the user is searching for.
I'd say that this is the correct approach. The only code issue I can see in your example is that you destruct page param, and then pass the page param string to getRecipes:
({ pageParam = "" }) => getRecipes(pageParam),
but in getRecipes, you expect an object to come in (which you again destructure):
const getRecipes = async ({ pageParam }) => {
You can fix that by either changing the call side, or the function syntax, and then it should work.

Can't keep user state for tests using TestAdapter

I have a set of automated tests that is working fine for my "base case". I have some different defaults (e.g. escalation contacts) that are displayed for users accessing the bot via certain URLs. This works by setting values in user and/or conversation state when a user connects to the bot. I can spoof these values in Emulator to test different cases. This all works fine for manual tests.
However, when I try to test via Mocha using TestAdapter, it seems that user state isn't being retained. I can see in my first step that I am getting the spoofed state values based on my value for channelId in the activity. However, when I send a second activity, the state values have reverted to null.
I think this is due to the processActivity function always creating a new TurnContext, though I'm a little unclear why that doesn't retain at least the user state.
So the question is, how can I modify this test, or is there a different test approach I can take, so that I can keep the values I set in user state across all activities? I can provide the escalationActivity file if needed, but it's just an object with the Test Cases and Test Steps so that I don't have to define them in my test.js file (and so that I can potentially reuse the same framework for other tests).
Here is my test.js file in its entirety.
const { TestAdapter, ActivityTypes, TurnContext, ConversationState, MemoryStorage, UserState } = require('botbuilder');
const { DispatchBot } = require('../../bots/dispatchBot');
const assert = require('assert');
const nock = require('nock');
require('dotenv').config({ path: './.env' });
const { escalationActivity } = require('../testData/escalationDialogTestDataPC');
const memoryStorage = new MemoryStorage();
const userState = new UserState(memoryStorage);
var userDialogStateAccessor = userState.createProperty('userDialogStateProperty');
const conversationState = new ConversationState(memoryStorage);
var dialogState = conversationState.createProperty('dialogState');
const appInsights = require('applicationinsights');
appInsights.setup('dummyKey').start();
const appInsightsClient = appInsights.defaultClient;
// Static texts
const myEatonMessage = `For more information on products and orders, please go to [My.Eaton.com](https://my.eaton.com). If you do not have a My.Eaton account, you can click "Request Access" at the top right.`;
describe('Project Center', async () => {
describe('Escalation', async () => {
const testAdapter = new TestAdapter();
async function processActivity(activity, bot) {
const context = new TurnContext(testAdapter, activity);
await bot.run(context);
}
let bot = new DispatchBot(new ConversationState(memoryStorage), new UserState(memoryStorage), appInsightsClient);
it('Welcome message', async () => {
const conversationUpdateActivity = {
type: ActivityTypes.ConversationUpdate,
channelId: 'projectCenter',
conversation: {
id: 'someId'
},
membersAdded: [
{ id: 'theUser' }
],
from: { id: 'theBot' },
recipient: { id: 'theUser' }
};
await processActivity(conversationUpdateActivity, bot);
let reply = testAdapter.activityBuffer.shift();
assert.strictEqual(reply.attachments[0].content.id, 'menuCard', 'Reply does not match.')
});
// BOT IS LOSING USER STATE ON SUBSEQUENT STEPS
Object.keys(escalationActivity).map((key) => {
describe(key, async () => {
let conversationData = escalationActivity[key].conversation;
//let channel = escalationActivity[key].channel;
//let intent = escalationActivity[key].intialOptions.topIntent;
conversationData.map((interaction, idx) => {
it(idx + '. ' + interaction.type, async () => {
// Create message activity
const messageActivity = {
type: ActivityTypes.Message,
channelId: 'projectCenter',
conversation: {
id: 'someId'
},
from: { id: 'theUser' },
recipient: { id: 'theBot' },
text: interaction.input
};
// Send the conversation update activity to the bot.
await processActivity(messageActivity, bot);
let reply = testAdapter.activityBuffer.shift();
if (idx == 0) { // First index has two extra activities that need to be skipped
reply = testAdapter.activityBuffer.shift();
reply = testAdapter.activityBuffer.shift();
}
assert.strictEqual(reply.text, interaction.reply, 'Reply does not match.');
//assertStrictEqual(1,1);
});
});
});
});
});
});

This isn't really an answer per say, but I was able to get the tests working by "re-spoofing" my projectCenter channel on every turn. For the actual bot function, this information is set in either onEvent via webchat/join event in directline or in onMembersAdded for all other channels. As mentioned in my question, user/conversation state is not being retained after my first Mocha test. I have not fixed that, thus I am not considering this issue completely resolved.
However, from the standpoint of at least being able to automate tests for this functionality, I accomplished that via onMessage handler. I simply look for my test channel, projectCenter, on every message, and reset all of the user and conversation state attributes.
I don't like this because it is a piece of code solely for testing and it is overwriting state every turn which is a bad practice, but as of yet I can't find any better way to get the information which I need to have in user state to persist through all of my tests. I am still hoping that someone can come up with a better solution to this issue... That said, this is at least working for my purposes so I wanted to present it as an answer here.

mongoosejs - find() using nested objects

question is possibly a duplicate but I haven't found anything that provides an appropriate answer to my issue.
I have an ExpressJS server which is used to provide API requests to retrieve data from a MongoDB database. I am using mongoosejs for the MongoDB connection to query/save data.
I am building a route that will allow me to find all data that matches some user input but I am having trouble when doing the query. I have spent a long while looking online for someone with a similar issue but coming up blank.
I will leave example of the code I have at the minute below.
code for route
// -- return matched data (GET)
router.get('/match', async (req, res) => {
const style_data = req.query.style; // grab url param for style scores ** this comes in as a string **
const character_data = req.query.character; // grab url param for character scores ** this comes in as a string **
// run matcher systems
const style_matches = style_match(style_data);
res.send({
response: 200,
data: style_matches
}); // return data
});
code for the query
// ---(Build the finder)
const fetch_matches_using = async function(body, richness, smoke, sweetness) {
return await WhiskyModel.find({
'attributes.body': body,
'attributes.richness': richness,
'attributes.smoke': smoke,
'attributes.sweetness': sweetness
});
}
// ---(Start match function)---
const style_match = async function (scores_as_string) {
// ---(extract data)---
const body = scores_as_string[0];
const richness = scores_as_string[1];
const smoke = scores_as_string[2];
const sweetness = scores_as_string[3];
const matched = [];
// ---(initialise variables)---
let match_count = matched.length;
let first_run; // -> exact matches
let second_run; // -> +- 1
let third_run; // -> +- 2
let fourth_run; // -> +- 3
// ---(begin db find loop)---
first_run = fetch_matches_using(body, richness, smoke, sweetness).then((result) => {return result});
matched.push(first_run);
// ---(return final data)---
return matched
}
example of db object
{
_id: mongoid,
meta-data: {
pagemd:{some data},
name: whiskyname
age: whiskyage,
price: price
},
attributes: {
body: "3",
richness: "3",
smoke: "0",
sweetness: "3",
some other data ...
}
}
When I hit the route in postman the JSON data looks like:
{
response: 200,
data: {}
}
and when I console.log() out matched from within the style match function after I have pushed the it prints [ Promise(pending) ] which I don't understand.
if I console.log() the result from within the .then() I get an empty array.
I have tried using the populate() method after running the find which does technically work, but instead of only returning data that matches it returns every entry in the collection so I think I am doing something wrong there, but I also don't see why I would need to use the .populate() function to access the nested object.
Am I doing something totally wrong here?
I should also mention that the route and the matching functions are in different files just to try and keep things simple.
Thanks for any answers.

just posting an answer as I seem to have fixed this.
Issue was with my .find() function, needed to pass in the items to search by and then also a call back within the function to return error/data. I'll leave the changed code below.
new function
const fetch_matches_using = async function(body, richness, smoke, sweetness) {
const data = await WhiskyModel.find({
'attributes.body': body,
'attributes.richness': richness,
'attributes.smoke': smoke,
'attributes.sweetness': sweetness
}, (error, data) => { // new ¬
if (error) {
return error;
}
if (data) {
console.log(data)
return data
}
});
return data; //new
}
There is still an issue with sending the found results back to the route but this is a different issue I believe. If its connected I'll edit this answer with the fix for that.

How to properly use Firestore's serverTimestamp to set the value of document thats being subscribed to?

Overview/Environment:
a react-native project v0.61.5
using react-native-firebase package
using actions to populate redux state, display firestore data through props
Goal:
Listen to collection of documents
use Firestore's FieldValue.serverTimestamp() to set time value of a document in said collection
use serverTimestamp's toMillis() function inside a snapshot listener
Observations/Errors:
when creating a document in said collection, the document gets created fine, and displays fine
while the doc/time value is created, the applications crashes due to the call to doc.get('time').toMillis() which is inside the snapshot listener: TypeError: null is not an object (evaluating 'doc.get('time').toMillis()')
So far I've tried all the suggestions noted here: Why is Firestore's 'doc.get('time').toMillis' producing a null Type Error?
Nothing seems to resolve this crash.
here's the snapshot listener:
.onSnapshot({ includeMetadataChanges: true }, (querySnapshot) => {
if (querySnapshot.metadata.fromCache && querySnapshot.metadata.hasPendingWrites) {
// ignore cache snapshots where new data is being written
return;
}
const messages = [];
querySnapshot.forEach((doc) => {
const estimateTimestamps = { serverTimestamps: 'estimate' }
const msg = doc.data();
msg.docId = doc.id;
msg.time = doc.get('time', estimateTimestamps).toMillis();
const timestamp = doc.get('time', estimateTimestamps);
if (timestamp) {
msg.time = timestamp.toMillis();
} else {
debugger
console.error(doc.id + ' is missing "time" field!');
}
messages.push(msg);
});
dispatch({ type: types.LOAD_MSGS, payload: messages });
resolve();
});
Here's how document is created:
const addMsg = (msg, userConvos) => {
return firebase.firestore().collection('messages').add({
time: firebase.firestore.FieldValue.serverTimestamp(),
sender: msg.sender,
read: false,
userConvos: [userConvos.sender, userConvos.receiver],
content: {
type: 'msg',
data: msg.text
}
});
};
I understand the value may be null fora small amount of time, I need a way to prevent the app from crashing during that period.

The error is pointing you to this code:
doc.get('time').toMillis()
It's saying that doc.get('time') returns null, and therefore, you can't call toMillis() on that.
The answer to the question you linked to explains exactly why that is. If it's still unclear, I suggest reading it again. The timestamp will simply be null if the event that a server timestamp has not reached the server.
Perhaps you meant to check if the timestamp is null like this, without calling toMillis():
msg.isPending = doc.get('time') === null;

After #DougStevenson helped me understand. Somewhat confusing but its important to understand the listener is constantly running, so once the Time value is available it will be set, so no real performance issues. I reformulated my approach to this, its working:
querySnapshot.forEach((doc) => {
const estimateTimestamps = { serverTimestamps: 'estimate' }
const msg = doc.data();
msg.docId = doc.id;
msg.time = doc.get('time', estimateTimestamps).toMillis();
const timestamp = doc.get('time', estimateTimestamps)
if (doc.get('time') !== null) {
msg.time = doc.get('time').toMillis()
}
messages.push(msg);
});

Angular Firestore - search for and update single document

I'm new to web (and everything asynchronous). I'm trying to complete a two step process in an Angular + Firebase app:
Query a Firestore collection to find the ID of a document that matches a filter (name == 'theName').
Use the ID to then update the document.
I'm coming from the embedded world where I can do something like this (context of the app - I'm trying to keep track of results in a combat robotics competition).
// Update the winning robot's doc with results
// Find the ID of the robot we're looking for (ex. winningRobotName = "Some Name")
this.firestore.collection('robots', ref => ref.where('name', '==', winningRobotName)).snapshotChanges()
.subscribe(data => {
this.bots = data.map(e => {
return {
id: e.payload.doc.id,
name: e.payload.doc.data()['name'],
// other fields
} as Robot;
})
});
let robot = this.bots[0]; <--------- This doesn't work because I reach here before the above call returns.
// Update this robot with new fields
this.firestore.doc('robots/' + robot.id)
.update({
fightCount : robot.fightCount + 1,
winCount : robot.winCount + 1,
// other updates
});
How does one wait for one subscription to return before executing another command? Do you nest the subscriptions? Is there something really basic I just don't know about yet?
Thanks.

I don't think AngularFire helps you here, and would instead do this directly on the regular JavaScript SDK:
ref.where('name', '==', winningRobotName))
.get()
.then((snapshots) => {
snapshots.forEach((doc) =>
doc.ref.update({
id: doc.id,
name: doc.data().name
})
})
})
I'm not exactly sure what name: doc.data().name is supposed to do (as it's an identity operation, delivering the same result as its input), but left it in just in case this matters to you.