I've been playing around with puppeteer for some time now and can't seem to work out the best approach.
My goal is to be able to select one of the days which once automatically clicked loads different data which I will then extract.
I'm having a hard time trying to work out the best way to navigate between the days with puppeteer.
I want to have the ability to input the day I want to select and when launching the page it will navigate to that selected day.
I did have more code written in, however stripped it back.
async function scrapeData(page) {
await page.goto(newPageURL);
// Select the day you want to go to.
let spanDay = await page.$(".MuiTab-wrapper");
await page.$eval(".MuiTab-wrapper", (el) =>
el.forEach((element) => {
console.log(element);
})
);
}
Related
Is there a way to have a command with buttons be used by two people at the same time in the same channel?
I made an adventure system which is going well so far, but I found out that only one person can use it in a channel. If another person uses it in the same channel, the "DiscordAPIError: Unknown Interaction" error comes as soon as a button is used. I tried adding the message author's id to the custom ID of the buttons, but it still doesn't work for some reason and I am utterly confused.
From what I understand, I think it's unrelated to there being multiple button instances, but I can't think of any other reason. Here is the filter:
const filter = async (interaction) => {
if (interaction.user.id === message.author.id) return true;
await interaction.deferReply()
await interaction.editReply({ content: `**${interaction.user.username}**, You can't use this button!`, ephemeral: true})
return false
}
And the button click detector goes something like this:
GameCollect.on('collect', async ButtonInteraction => {
ButtonInteraction.deferUpdate()
// My code
})
I have tried:
Removing ButtonInteraction.deferUpdate() from all my click collectors.
Making the custom ids for the buttons have the message author's id at the end and making my code compatible with that change.
Smashing my head on my keyboard from confusion. Didn't work either... (satire)
If it's necessary, I can copy my code into a repl so that you can identify the problem easier, but I don't think the other portions would have any effect on this. I think it's just a piece of code/slight adjustment that I have to add for it to work with two people in the same channel.
EDIT: Alright, here is a source bin so that you can identify the problem better. Check the comments on the recent answer for some clarifications on my testing too. I will keep you updated if I find a solution.
DISCLAIMER: I still haven't found a way to make this work in over a month, and I am essentially just giving up on buttons. Thank you to everyone who helped though.
Try it:
Remove await interaction.deferReply() from filter
Create dynamic customIds like Math.random()
EDIT:
description: i had the same issue recently on my own bot, The problem is that when 2 messages that the have collectors(basically they're waiting for a interaction), if you call await interaction.deferReply() soon, it wil get messy, So here's the solution
First of all lets go for the filter function for checking the interaction, We need 2 things:
Checking the button id with the incoming interaction button id
Checking the user to be the real one
After all of these
delete deferReply from the filter func, Because when you didn't check the interactions, calling deferReply, will call both of messages or all of the messages that are waiting for interaction
add the deferReply to the function that is after filter
And don't remember to use random ids for components(Buttons, select menus and ...)
Instead of creating a collector on the text channel, you should create the collector on the actual message
const GameCollect = message.channel.createMessageComponentCollector({ filter, time: 60000 * 15 })
To
const GameCollect = <Message>.createMessageComponentCollector({ filter, time: 60000 * 15 })
Along with datetime prompt I am trying to pass extra suggestions but choices in prompt object is not working. Can anyone suggest a way to pass extra suggestions along with date time prompt.
Solution tried:
async flightBookingDate(step) {
if (destinationProvidedForFlightBooking) {
step.values.destination = "Delhi";
} else {
step.values.destination = step.result;
}
if (!step.values.flightBookingDate) {
await step.context.sendActivity("Below are the suggested date options?")
var reply = MessageFactory.suggestedActions(['Today', 'Tomorrow']);
await step.context.sendActivity(reply);
let prompData = {
prompt: "For when you want to book flight?",
retryPrompt: "No flights avaliable for today. Please enter different date"
}
return await step.prompt(DATETIME_PROMPT, prompData)
} else {
return await step.continueDialog()
}
}
after executing bot below is the output:
can someone help me with the solution?
The first thing to understand is that Teams does not support suggested actions.
A common alternative to suggested actions is cards. You can put actions in any of the cards Teams supports, but if you just need actions and no other features then I recommend using hero cards because they're the simplest. You can refer to sample 6 to see how to use cards. The code to create the activity containing your card might look something like this:
var reply = MessageFactory.attachment(CardFactory.heroCard(null, null, ['Today', 'Tomorrow']));
If you want to have your buttons show up as a card in Teams but as suggested actions in Emulator then you can use ChoiceFactory.forChannel but you'll still need a way to keep the suggested actions from disappearing when the next message is sent. The way to do this would be to include the suggested actions as part of the prompt instead of the activity sent before the prompt.
if (!step.values.flightBookingDate) {
let promptData = {
prompt: ChoiceFactory.forChannel(step.context, ['Today', 'Tomorrow'], "For what day do you want to book the flight? Below are the suggested date options:"),
retryPrompt: ChoiceFactory.forChannel(step.context, ['Today', 'Tomorrow'], "No flights avaliable for that day. Please enter a different date. Below are the suggested date options:")
};
return await step.prompt(DATETIME_PROMPT, promptData);
I have a form with a drop down select input field.
this select field's options should ideally be up to date with the backend.
I have a script in the backend which updates a list of options every 5 minutes.
As an initial trivial solution I've added an 'on focus' event listener to the select filed which fires a fetch request and updates the drop down with the returned list.
This off course introduces a small delay, and is overall unpleasant to use when I have menus that react in dependence to the first drop down.
Is there a way to run a script on the frontend background that'll fetch the options to a variable and update the drop down every few minutes?
(I'm using vanilla JS)
Thanks a lot in advance!
You could do something like this to update every 10 seconds:
function getOptions(){
setInterval(async function(){
const response = await fetch(/*whatever you want to fetch*/);
const options = await response.text();
/*use the data here*/
}, 10000);
}
You could also do a function which fires itself when it is done fetching the data using fetch().then() which I think is to be preferred here.
fetch() can use .then(), .catch() and .finally() so you could run the function again at .finally().
I'm working with puppeteer at the moment to create a web-crawler and face the following problem:
The site I'm trying to scrape information off of uses Tabs. It renders all of them at once and sets the display-property of all but one tab to 'none' so only one tab is visible.
The following code always gets me the first flight row, which can be hidden depending on the date that the crawler is asking for.
const flightData = await page.$eval('.available-flights .available-flight.row', (elements) => {
// code to handle rows
}
There doesn't seem to be an additional parameter you can pass with .$eval() like you can in
.waitForSelector('.selector', {hidden: false})
Am I following the wrong idea?
Is there a way to only select the shown element and work with that data?
const flightData = await page.$eval('.available-flights .available-flight.row:not([style*="display:none"]):not([style*="display: none"])', (elements) => {
// code to handle rows
}
Does the trick :)
I am trying to create a chat like app using Firestore and trying to list all messages in chat and updates it every time when a message is added.
I tried this way first to implement it.
mounted() {
docRef.collection('messages').orderBy('timestamp', 'desc').limit(1).onSnapshot((querySnapShot) => {
querySnapShot.forEach((doc) => {
if (!doc.metadata.hasPendingWrites) {
this.messages.push(doc.data())
}
})
})
}
This way seems efficient because this way only gets the latest message in collection. However, I found out this way has a problem. When a user refreshs a page, the user can't get the past messages.
So I changed it like this.
mounted() {
docRef.collection('messages').orderBy('timestamp', 'asc').onSnapshot((querySnapShot) => {
querySnapShot.docChanges().forEach((change) => {
if (change.type === 'added') {
this.messages.push(change.doc.data())
}
})
})
}
This way works as I expected. But this way needs a lot of requests because every time the collection is changed I need to read all the documents in the collection.
What is the efficient way to list messages in chat?
I thought it works if I get all the current messages first and set the listener for new messages but the listener is triggered immediately after I enter the page even though there is no change in the collection and read the latest message twice.
I ended up just using a flag to check whether the initial trigger is done. I don't know if this is smart way but this works.
// Get all current messages
docRef.collection('messages').orderBy('timestamp', 'asc').get().then((querySnapShot) => {
querySnapShot.forEach((doc) => {
this.messages.push(doc.data())
})
})
// Update for new messages (Skip the initial loading)
docRef.collection('messages').orderBy('timestamp', 'desc').limit(1).onSnapshot((querySnapShot) => {
querySnapShot.forEach((doc) => {
if (!doc.metadata.hasPendingWrites && this.isInitialDone) {
this.messages.push(doc.data())
}
this.isInitialDone = true
})
})
You are going to need to either set an appropriate limit, or add a filter to determine which ones you want.
If you want to go with a size limit, just change limit(1) in your first query to the number you actually want.
If you want to go with a filter, you should probably use the timestamp you already have in place in order to determine how far back in the past you would like to go.
Those are your two options for limiting the size of the results. You can use either one or both together, but there is no other way of limiting the size of the result set.