I am attempting to retrieve JSON from an external source - Open Weather Map and I'm using the Javascript Fetch API to generate the request. I should be returned the current weather at a location via JSON that I can parse from the Open Weather Map API. When I'm debugging my applications and I have breakpoints on the fetch statement the request is sent and I receive a prompt response from the API. After I remove the breakpoints I receive the following message from the Firefox Developer Edition console TypeError: NetworkError when attempting to fetch resource., the Google Chrome console doe not log an error but I can see that a new network request was not generated.
const submitButton = document.querySelector('.submit-zip-button');
const APIKEY = 'REDACTED';
function parseWeather(currentWeather){
let currentTemp = currentWeather.main.temp;
let currentWeatherDescription = currentWeather.weather[0].description;
let weatherIconCode = currentWeather.weather[0].icon;
}
function getWeather(zipNum){
let weatherData = new Object();
fetch(`https://api.openweathermap.org/data/2.5/weather?zip=${zipNum},us&units=imperial&appid=${APIKEY}`)
.then(response => response.json())
.then(data => {
console.log(data);
weatherData = data;
parseWeather(weatherData);
});
}
submitButton.addEventListener('click', function(event){
event.stopPropagation();
let zipcode = document.querySelector('#zipcode-input');
zipcode = Number(zipcode.value);
getWeather(zipcode);
Does anyone have any tips or material I can read to better understand what's going on? I've read the Fetch MDN Page and used their example code as the basis of my fetch function.
I see a fetch, but no async / await, or any other way of waiting for the request to finish.
JS won't wait for your request to finish, it will just continue running. By the moment the request is finished, the execution of that code will be long gone.
There is a similar post right here and also a more detailed explanation of why and how to await a request right here.
If you make breakpoints, you will slow down the execution of the program, and at that moment it will be able to finish the request, having enough time. That's why in the debug mode it works. In production, it will take no time to await anything, until you tell him to.
Related
In my sveltekit app I make AJAX calls to my api endpoints. For example:
+page.svelte
<script>
async function get_card() {
const url = '/api/card/?category=' + $page.params.slug;
const response = await fetch(url, {
method: 'GET',
})
const card = await response.json();
return card;
}
</script>
In the browser javascript console I get this warning:
Loading /api/card/?category=Neurology using `window.fetch`.
For best results, use the `fetch` that is passed to your `load`
function: https://kit.svelte.dev/docs/load#making-fetch-requests
But as far as I can tell, that fetch function is only accessible to me on the server, and I do not see a way to use it in a script that may run on the client (such as +page.svelte). I tried passing the function as part of the data object from load:
+layout.server.js
export const load = async ({ fetch, locals }) => {
return {
email: locals.user.email,
group: locals.user.group,
fetch: fetch
}
}
But, not surprisingly, that does not work since the function is not serializable.
Am I Doing It Wrong™, or should I just ignore the warning?
fetch is originally a browser API and SvelteKit defines it on the server as well, if it does not exist. The warning is there to tell you that you are creating another round trip to the server (one for the page and one for the data) when you possibly could have loaded the data on the server so it could be transmitted as part of the page (during server-side rendering).
If the code of your function is not executed right away, then this is a false positive (recent issue on this). I.e. if the data should be requested at a significantly later point, there is no way to bundle the request with the page.
(You are definitely not meant to pass on the fetch of load, you are supposed to use it to get the data.)
Im using the puppeteer package , in order to scrap a web page data that is fetched by clicking a button in this page
this are the presetting that I'm using:
const puppeteer = require('puppeteer-extra')
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
puppeteer.use(StealthPlugin())
// Add adblocker plugin to block all ads and trackers (saves bandwidth)
const AdblockerPlugin = require('puppeteer-extra-plugin-adblocker')
puppeteer.use(AdblockerPlugin({ blockTrackers: true }))
those setting are made in order that I will not be detected as a robot.
here what I'm doing :
(basically , creating a request by clicking a button , then this request return a json with a data that fill up some text info in a label , then I'm reading the data from that label
here's how im clicking the button :
const box = await btn.boundingBox();
const x = box.x + (box.width/2);
const y = box.y + (box.height/2);
console.log(x, y);
page.mouse.move(x,y,{step:1});
page.mouse.click(x,y)
await page.waitForTimeout(4000);
then afterwards ---> I'm getting the data from the data:
const [result] = await page.$x('//*[#id="content"]/div[1]/div[1]/div/div[2]/div');
// const txt = await result.evaluate.toString
let value = await page.evaluate(el => el.textContent, result);
console.log(value);
console.log('done?');
await browser.close();
const dic = {};
dic['status'] = 200;
dic['data'] = {"message": value};
response.send(dic);
I'm also using the 'on' method in order to see if the im getting a response from the action of clicking the button , like so:
await page.on('response', async response =>{
try {
console.succ(await response.json());
} catch (error) {
//
// console.error(error);
}
});
and it sure get one.
the problem is ---> that when I'm deploying it to the firebase cloud functions server,
firebase deploy --only functions
and then triggering the function -->
I'm getting a json that look like this :
{ success: false, message: 'You look like a robot.' }
But when deploying the same code to my local host like so
firebase serve --only functions
and then triggering the function -->
I'm not detected as a robot
and getting the json with a successful result --> and with that data that the clicking of a button supposed to fetch.
this is so weird , I'm trying to think that there's a connections between the firebase cloud functions and reCAPTCHA , because both are a google services
but, its not seem's reasonable for it to be true .
that being said, what could be the reason for this?
all that change is the environment that the code runs from.
do you have any idea why this is happening ?
and how to solve it of course .
Since your function runs properly locally, it's almost certainly not the function itself.
Sites take a variety of different approaches to detect bots, one of which is blocking traffic from known data centers like Google Cloud's. Using a residential IP proxy like those provided by BrightData will probably circumvent this.
I'm facing the same issue while using Puppeteer in Firebase Cloud Functions.
I'm using a residential IP proxy with the following set of packages puppeteer-extra, puppeteer-extra-plugin-stealth, puppeteer-extra-plugin-anonymize-ua', and user-agents`.
On localhost, all is working as expected while running Puppeteer in firebase Cloud Functions I'm getting a 404 response from the requested URL. So there must be some difference.
I am trying to get the latitude and longitude of user location using google api
I have tried doing this using the fetch method in javascript but I do not get any response. There's no error or anything.
The documentation says to fetch it this way:
https://www.googleapis.com/geolocation/v1/geolocate?key=MY_API_KEY
How I fetch it:
const res = await fetch(https://www.googleapis.com/geolocation/v1/geolocate?key=MY_API_KEY)
const response = await res.json()
When I first did it like above I got an error because it returned nothing which I found out after console.logging res.text()
I do not know what I'm doing wrong In the documentation they equally said something about including Request body but it's optional, So I have no need to include it. Please do you have any ideas on what I'm doing wrong?
I believe you need to send this as a POST request according to the documentation you linked. By default fetch sends a GET request.
To do that you just need to pass in the request type to the fetch function
const response = await fetch(<URI>,{method:'POST'});
const res = await response.json()
console.log(res)
Is there any way to get responses to a specific client when another client has a different request at the same time to the same server?
This is code snippet for an exchange server. The given function is present in a library named "ccxt", this function "exchange.fetchMarkets()" has an API which requests to a third party server which is an exchange server like 'bitfinex', 'crex24', 'binance', etc. The issue I am facing is when one client is requesting for an exchange like 'crex24' at the same time when another client is requesting for different exchange like 'binance', they are getting the same response as the function calls for the last recent exchange.
I want it to give responses according to the client's requests independent of each other.
this one is controller function:
const ccxt = require("ccxt");
exports.fetchMarkets = function(req, res){
let API = req.params.exchangeId;
let exchange = new ccxt[API]();
if (exchange.has["fetchMarkets"]) {
try{
var markets = await exchange.fetchMarkets();
res.send(markets)
}catch (err) {
let error = String(err);
res.send({ failed: error });
}
}else{
res.send({loadMarkets : "not available"})
}
}
This is end point for the server request:
app.route('/markets/:exchangeId')
.get(exchange.fetchMarkets)
Here you can find the ccxt library: https://github.com/ccxt/ccxt/wiki/Manual and can be included in the project by "npm install ccxt"
I don't see why the code you mentioned wouldn't work the way you are expecting it to work. I created a small app and it is working as expected. You can check here
https://repl.it/repls/IllfatedStrangeRepo
I am hitting four different request with different ids and I am getting different response.
Hope it clear the doubts.
When I go into the Network tab and inside response, I do get the results of this API, but I cannot get it to output in the console.
This is the code:
const NAMEURL = "https://uzby.com/api.php"
// get data from api
function getDataFromApi(value, callback){
const QUERY = {
min:`${value}`,
max:`${value}`
}
$.getJSON(NAMEURL, QUERY, callback)
}
function renderResult(result){
return `${result}`;
}
// render results to page
function displayName(data){
console.log(data);
const results = renderResult(data);
$('.nameResult').html(results);
}
// wait for user to submit
function watchSubmit() {
$('.js-search-form').submit(event => {
event.preventDefault();
const queryTarget = $(event.currentTarget).find('#js-dropValue');
const thisquery = queryTarget.val();
getDataFromApi(thisquery, displayName);
});
}
// running the watch submit function waiting for click
$(watchSubmit);
I tried doing a console.log(getDataFromApi()) but I get undefined in console.
console.log(getDataFromApi()); would never work, its displayName() that you would have to console log.
Your code was not wrong, with the exception of what you were trying to console log, but even then its not going to work because uzby does not want to share its resources directly with your application.
So indeed that error you were getting, the CORS error is the problem.
You need to proxy that request through your own server, the one that loads your page. You will have to research documentation on your server side language proxy. A potential solution for your code could be like this:
xmlhttp.open(
'GET',
'https://cors-anywhere.herokuapp.com/https://example.com/api.php?' + param,
true
);