Develop Reference

concatenation of request url to web archive's base url when using fetch api

I want to run some script from the browser console for external sites that extracts the dom tree with some format, I did it right but when I wanted to write the result in my file system. this was not allowed due to security issues, so I wrote a backend script using Node.JS that writes into the file system using a post request from the console an reading from the file system while sending a get request on another endpoint.
this is a part of the code I run on the console in which I'm using the fetch API to send my data to the backend (dict)
let dict = generateDictionary(root);
const url = new URL("http://localhost:5000/write");
fetch(url, {
method: "POST",
body: JSON.stringify({ data: JSON.stringify(dict) }),
headers: {
"Content-Type": "application/json"
},
})
.then((response) => console.log(response))
.then((data) => console.log(data));
for most of different sites it works well like that
and this is a snippet from the dom tree after serialization
the problem appears when I use https://web.archive.org/ historical pages to send the same request like https://web.archive.org/web/20220201010705/https://www.reference.com/.
here as it appears on the snapshot it appends the localhost URL to the web archive's URL. I tried to modify different http headers to solve the problem but all of my trials have failed. what can I do to avoid this concatenation?

Cannot OAuth into Spotify API through CloudFlare Worker with Client Credentials Flow

I'm trying to get an Access Token from Spotify to use it further to access the Spotify API without need of redirect URL to an Auth confirm page.
I have a CloudFlare Worker setup that has the code for this but it keeps failing no matter what I try.
According to the Docs of the Client Credential Flow I'm supposed to make a POST Request to the Endpoint as follows:
const credentials = btoa(`${SPOTIFY_ID}:${SPOTIFY_SECRET}`);
const auth = await fetch("https://accounts.spotify.com/api/token", {
method: "POST",
headers: {
"Content-type": "application/x-www-form-urlencoded",
"Authorization": `Basic ${credentials}`,
},
body: JSON.stringify({
grant_type: "client_credentials",
}),
});
In this example SPOTIFY_ID and SPOTIFY_SECRET are stored into the Environment Variables of the Worker itself and are valid credentials of my Spotify APP. The btoa() method converts the credentials into Base64 as requested under the format of <cliend_id:client_secret> and the output is correct (tested decoding it successfully)
The Response I get from the POST Request is always the same:
{
"webSocket": null,
"url": "https://accounts.spotify.com/api/token",
"redirected": false,
"ok": false,
"headers": {},
"statusText": "Bad Request",
"status": 400,
"bodyUsed": false,
"body": {
"locked": false
}
}
Also tested to run the request with curl and everything works, it returns a valid token.
I saw on other topic related questions that this is supposed to be run server side because of the OAuth security convention, but the CloudFlare worker, even though runs on top of V8 should be considered server side as well, since it doesn't run in the browser.
I would like to understand better why it seems that CloudFlare Workers environment is considered client side, this also happens if I try to use the Spotify API Client for Node where it doesn't include the server side methods while compiling, making them none existent in the compiled worker (maybe because of Webpack bundling?)
Another idea is that the fetch() method is client-side by default, so a possible solution would be to use another library to make the POST request to Spotify but on Workers I don't know which one can be used.
Am I missing something here? This has been taking too much time to solve and I have no hint.

I've managed to make it work but I have no clue how and why.
Basically by reading more on OAuth questions I figured out a bypass.
The problem seems to be the body parameter that might get in conflict with the fetch() method.
As Kenton mentioned I tried to look for more error info and eventually I came across this with the old code:
{
"error": "unsupported_grant_type",
"error_description": "grant_type parameter is missing"
}
This was not printed before I changed the credentials to be fully parsed into the const instead of evaluating it under the Authorization key of the headers object.
The way to make it work inside CloudFlare Workers is to pass the body grant_type param as query string inside the URL like this:
const credentials = `Basic ${btoa(`${SPOTIFY_ID}:${SPOTIFY_SECRET}`)}`
const auth = await fetch("https://accounts.spotify.com/api/token?grant_type=client_credentials", {
method: "POST",
headers: {
"Content-type": "application/x-www-form-urlencoded",
Authorization: credentials,
},
});
What I get now is the same correct response with token as worked in curl
{
"access_token": "randomstringthatrepresentsmytokenifinallysolvedthisshitthanksgodiwasgoinginsane",
"expires_in": 3600,
"token_type": "Bearer"
}
If someone knows better why this works and the previous doesn't it's encouraged to comment to this message.

Avoid CORS preflight for Firebase callable function

I have a Firebase Callable Cloud Function which I call in my javascript app in the browser.
Because the request host is ...cloudfunctions.net and not my app domain this results in a CORS preflight OPTIONS request before the real POST request.
If this was a function with http trigger I'd avoid the extra time that the preflight takes by specifying my function as a rewrite in my hosting config and sending the request to the same domain my app is hosted on.
Is there any way to avoid this preflight with Firebase Callable Cloud Functions? Perhaps there's a way to proxy the request through Firebase Hosting, like you can with http Cloud Functions

After combing through Firebase docs and JS SDK source I've decided this is not possible without using/overriding private APIs.
The solution I've used is to replicate the JS SDK code but specifying a URL that goes via Firebase Hosting so it's on the same domain as my app.
Same Cloud Function, same app code, no CORS preflight 👍🏼
Create a normal Firebase Callable Cloud Function
Add a rewrite to firebase.json
{
...
"hosting": {
...
"rewrites": [
{
"source": "myFunction",
"function": "myFunction"
}
]
}
}
Instead of calling it with firebase.functions().httpsCallable('myFunction') send a POST request to your own new URL
const token = await firebase.auth().currentUser.getIdToken()
const response = await fetch(
'https://myapp.web.app/myFunction',
{
headers: {
'Content-Type': 'application/json',
Authorization: 'Bearer ' + token
},
method: 'post',
body: JSON.stringify({ data })
}
)
Now the URL is within your domain so no CORS issues

For users who have the CORS preflight problem with the firebase local emulator, if you're using Webpack or Vue (vue-cli), you can resolve the situation with a proxy:
my-firebase-initialize-file.js (client side)
firebase.initializeApp(...); // your config
firebase.functions().useFunctionsEmulator('http://localhost:1234/api'); // Client url + /api
webpack.config.js or vue.config.js (client side, at the root, next to package.json)
module.exports = {
devServer: {
proxy: {
"^/api": {
target: "http://localhost:5001", // Emulator's url
pathRewrite: {
"^/api": ""
},
ws: true,
changeOrigin: true
}
}
}
};
Now every http://localhost:1234/api will be proxied to http://localhost:5001, so
you don't need CORS preflight anymore.
Of course you need to adapt these local urls to your case.

Due to security, it is not possible to skip to preflighted request
Reference:
https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#Preflighted_requests
Preflighted request would skip when meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (for example, Connection, User-Agent, or any of the other headers with names defined in the Fetch spec as a “forbidden header name”), the only headers which are allowed to be manually set are those which the Fetch spec defines as being a “CORS-safelisted request-header”, which are:
Accept
Accept-Language
Content-Language
Content-Type (but note the additional requirements below)
DPR
Downlink
Save-Data
Viewport-Width
Width
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
No event listeners are registered on any XMLHttpRequestUpload object used in the request; these are accessed using the XMLHttpRequest.upload property.
No ReadableStream object is used in the request.

If you are using specific region on server-side, make sure to specify region at client-side as well. Hope this help.
Firebase Functions
exports.status = functions
.runWith({
memory: '128MB',
timeoutSeconds: 15
})
.region('asia-east2')
.https.onCall((data, context) => {
return { message: 'OK' };
});
Web client
let functions = firebase.functions('asia-east2');
let status = functions.httpsCallable('status');
status({}).then(res => console.log(res));

I had the exact same issue: I had a custom domain and I was getting a CORS error when using a callable function.
I was looking through Firebase and saw a dropdown that was unfamiliar to me. I selected my hosting application, deleted and redeployed the function, and it was working fine without any CORS problems.
Firebase hosting setup

How do I fix CORS issue in Fetch API

I'm building a front-end only basic Weather App using reactjs. For API requests I'm using Fetch API.
In my app, I'm getting the current location from a simple API I found
and it gives the location as a JSON object. But when I request it through Fetch API, I'm getting this error.
Failed to load http://ip-api.com/json: Request header field Access-Control-Allow-Origin is not allowed by Access-Control-Allow-Headers in preflight response.
So I searched through and found multiple solutions to fix this.
Enabling CORS in Chrome solves the error but when I deploy the app on heroku, how can I access it through a mobile device without running into the same CORS issue.
I found an proxy API which enables the CORS requests. But as this is a location request, this gives me the location of the proxy server. So it's not a solution.
I've gone through this Stackoverflow question and added the headers to the header in my http request but it doesn't solve the problem. (Still it gives the same error).
So how can I solve the issue permanently ? What's the best solution I can use to solve the CORS issue for http requests in Fetch API ?

if you are making a post, put or patch request, you have to stringify your data with body: JSON.stringify(data)
fetch(URL,
{
method: "POST",
body: JSON.stringify(data),
mode: 'cors',
headers: {
'Content-Type': 'application/json',
}
}
).then(response => response.json())
.then(data => {
....
})
.catch((err) => {
....
})
});

To the countless future visitors:
If my original answer doesn't help you, you may have been looking for:
Trying to use fetch and pass in mode: no-cors
What is an opaque response, and what purpose does it serve?
Regarding the issue faced by the OP...
That API appears to be permissive, responding with Access-Control-Allow-Origin:*
I haven't figured out what is causing your problem, but I don't think it is simply the fetch API.
This worked fine for me in both Firefox and Chrome...
fetch('http://ip-api.com/json')
.then( response => response.json() )
.then( data => console.log(data) )

You should use the proxy solution, but pass it the IP of the client instead of the proxy. Here is an example URL format for the API you specified, using the IP of WikiMedia:
http://ip-api.com/json/208.80.152.201

pouchdb remote db replication "unauthorised" Cookie credentials ignored in electron app

Issue
I have an electron app, I managed to login to couchdb with cookie auth.
I also manage to retrieve the correct session. I used the fetch api so far to handle the requests.
For cookie storage in electron I use electron-cookies and this is working, if I do another request with credentials: 'include' and 'withCredentials' : true options (after login in) it makes the request with the cookie (Cookie:AuthSession=...).
Whenever a user is created in my app a corresponding database is created too with the new user being the admin of that database. So far so good.
This setup works to 100% in postman.
However, if I try to sync to the remoteDB with PouchDB this does not work, I get an unauthorized error.
My guess is that the authentification cookie is somehow ignored by PouchDB.
I tried way:
remoteDB = new PouchDB('http://somecorrecturl/userDBName');
and also like this:
remoteDB = new PouchDB('http://somecorrecturl/userDBName', {
ajax: {
method: 'POST',
mode: 'cors',
headers: {
'Content-Type': 'application/json',
'withCredentials' : true
},
credentials: 'include',
withCredentials: true,
}
});
without success.
( I also don’t see a request going out in the Network tab, this could be normal due to the nature of electron(?) tho’ )
Is there some special option I have to include? Using the fetch api for example I had to add credentials: 'include' before it would use the session cookie.
Is cookie auth even possible with pouchdb?
Or is there a way to sync a pouchdb with a couchdb "natively" without using the pouchdb api?
Doing the exact same thing with the same auth-cookie but using postman works. So my bet is that the cookie is ignored by pouchdb.
Info
Environment: Node.js / Electron
Platform: Chromium / Electron
Adapter: PouchDB
Server: CouchDB
Thank you for your help!