I am trying to play an audio using Google Text-To-Speech. Therefore I need to post a request to their endpoint with the Referer and the User-Agent properly set. This call should return an MP3 that I can play.
However, I get Refused to set unsafe header errors. This is my code. How can I do this?
$.ajax({
url: 'http://translate.google.com/translate_tts?ie=UTF-8&q=Hello&tl=en&client=t',
beforeSend: function(xhr) {
xhr.setRequestHeader("Referer", "http://translate.google.com/");
xhr.setRequestHeader("User-Agent", "stagefright/1.2 (Linux;Android 5.0)");
}, success: function(data){
el.mp3 = new Audio(data);
el.mp3.play();
}
});
You can't. It is impossible.
The specification requires that the browser abort the setRequestHeader method if you try to set the Referer header (it used to be that User-Agent was also forbidden but that has changed)..
If you need to set Referer manually then you'll need to make the request from your server and not your visitor's browser.
(That said, if you need to be deceptive about the user agent or referer then you are probably trying to use the service in a fashion that the owner of it does not want, so you should respect that and stop trying).
Note that while jQuery wraps XHR, the same rules apply to fetch.
Empty Origin and Referer headers with GET XMLHttpRequest from <iframe>
Well actually, it is possible; at least for ordinary web pages.
The trick consists in injecting an XMLHttpRequest
function into an empty <iframe>.
The origin of an empty <iframe> happens to be about://blank, which results in empty Origin and Referer HTTP headers.
HTML:
<iframe id="iframe"></iframe>
JavaScript:
const iframe = document.getElementById('iframe');
const iframeWin = iframe.contentWindow || iframe;
const iframeDoc = iframe.contentDocument || iframeWin.document;
let script = iframeDoc.createElement('SCRIPT');
script.append(`function sendWithoutOrigin(url) {
var request = new XMLHttpRequest();
request.open('GET', url);
request.onreadystatechange = function() {
if(request.readyState === XMLHttpRequest.DONE) {
if(request.status === 200) {
console.log('GET succeeded.');
}
else {
console.warn('GET failed.');
}
}
}
request.send();
}`);
iframeDoc.documentElement.appendChild(script);
JavaScript evocation:
var url = 'https://api.serivce.net/';
url += '?api_key=' + api_write_key;
url += '&field1=' + value;
iframeWin.sendWithoutOrigin(url);
Having the possibility of sending empty Origin and Referer HTTP headers is important to safeguard privacy when using third-party API services. There are instances where the originating domain name may reveal sensitive personal information; like being suggestive of a certain medical condition for example. Think in terms of https://hypochondriasis-support.org :-D
The code was tested by inspecting the requests in a .har file, saved from the Network tab in the F12 Developer View in Vivaldi.
No attempt in setting the User-Agent header was made. Please, comment if this also works.
There are some header, which browser doesn't allow programmer to set its value in any of the javascript framework (like jQuery, Angular, etc.) or XMLHttpRequest ; while making AJAX request. These are called the forbidden headers: Forbidden Header
Related
My AJAX function:
function ajaxQuery(url, method, param, async, onsuccess, onfailure) {
var xmlHttpRequest = new XMLHttpRequest();
var callback = function(r) { r.status==200 ? (typeof(onsuccess)=='function' && onsuccess(r)) : (typeof(onfailure)=='function' && onfailure(r)); };
if(async) { xmlHttpRequest.onreadystatechange = function() { if(xmlHttpRequest.readyState==4) { callback(xmlHttpRequest); } } }
xmlHttpRequest.open(method, url, async);
xmlHttpRequest.setRequestHeader('X-REQUESTED-WITH', 'XMLHttpRequest');
xmlHttpRequest.withCredentials = true;
if(method == 'POST') { xmlHttpRequest.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded'); }
xmlHttpRequest.send(param);
if(!async) { callback(xmlHttpRequest); }
}
Function call:
ajaxQuery('http://example.net/index.php', 'GET', null, true, function(r) {
tmp.innerHTML = r.responseText;
nlt = [].map.call(tmp.querySelectorAll('.nlt'), function(x) { return x.textContent; });
});
Headers set in PHP:
header('Access-Control-Allow-Origin: https://example.com');
header('Access-Control-Allow-Origin: https://www.example.com');
header('Access-Control-Allow-Origin: http://example.net');
header('Access-Control-Allow-Methods: GET, OPTIONS');
header('Access-Control-Allow-Credentials: true');
if(!preg_match('%https?:\/\/(www\.)?example\.com%', $_SERVER['HTTP_REFERER']) && !preg_match('%https?:\/\/example\.net%', $_SERVER['HTTP_REFERER'])) { die('No way!'); }
I am calling the userscript from a page that uses https, and my domain uses http. When I try AJAX through http, I get (Firefox) Blocked loading mixed active content. If I switch the query URL to https, the error changes to Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource, even though my PHP script clearly allows for requests from the external site. What am I missing?
In this particular example, my site is "http://example.net" and the external site is "https://www.example.com"
It is impossible to get an external resource through AJAX, JSONP or iFrames if the protocols don't match, at least in Firefox and Chromium, due to stupid "mixed content" restrictions. My website is running over http, and the website for which the userscript has been written has enforced https (meaning trying to request its pages through http automatically redirects to https, so I can't even work around the restriction by opting in for http).
The Access-Control-Allow-Origin header should be the same value as the Origin header as long as you want to allow it.
So. you want to multiple domains. I'm recommend you using 'regex'
I'm trying to load a cross-domain HTML page using AJAX but unless the dataType is "jsonp" I can't get a response. However using jsonp the browser is expecting a script mime type but is receiving "text/html".
My code for the request is:
$.ajax({
type: "GET",
url: "http://saskatchewan.univ-ubs.fr:8080/SASStoredProcess/do?_username=DARTIES3-2012&_password=P#ssw0rd&_program=%2FUtilisateurs%2FDARTIES3-2012%2FMon+dossier%2Fanalyse_dc&annee=2012&ind=V&_action=execute",
dataType: "jsonp",
}).success( function( data ) {
$( 'div.ajax-field' ).html( data );
});
Is there any way of avoiding using jsonp for the request? I've already tried using the crossDomain parameter but it didn't work.
If not is there any way of receiving the html content in jsonp? Currently the console is saying "unexpected <" in the jsonp reply.
jQuery Ajax Notes
Due to browser security restrictions, most Ajax requests are subject to the same origin policy; the request can not successfully retrieve data from a different domain, subdomain, port, or protocol.
Script and JSONP requests are not subject to the same origin policy restrictions.
There are some ways to overcome the cross-domain barrier:
CORS Proxy Alternatives
Ways to circumvent the same-origin policy
Breaking The Cross Domain Barrier
There are some plugins that help with cross-domain requests:
Cross Domain AJAX Request with YQL and jQuery
Cross-domain requests with jQuery.ajax
Heads up!
The best way to overcome this problem, is by creating your own proxy in the back-end, so that your proxy will point to the services in other domains, because in the back-end not exists the same origin policy restriction. But if you can't do that in back-end, then pay attention to the following tips.
**Warning!**
Using third-party proxies is not a secure practice, because they can keep track of your data, so it can be used with public information, but never with private data.
The code examples shown below use jQuery.get() and jQuery.getJSON(), both are shorthand methods of jQuery.ajax()
CORS Anywhere
2021 Update
Public demo server (cors-anywhere.herokuapp.com) will be very limited by January 2021, 31st
The demo server of CORS Anywhere (cors-anywhere.herokuapp.com) is meant to be a demo of this project. But abuse has become so common that the platform where the demo is hosted (Heroku) has asked me to shut down the server, despite efforts to counter the abuse. Downtime becomes increasingly frequent due to abuse and its popularity.
To counter this, I will make the following changes:
The rate limit will decrease from 200 per hour to 50 per hour.
By January 31st, 2021, cors-anywhere.herokuapp.com will stop serving as an open proxy.
From February 1st. 2021, cors-anywhere.herokuapp.com will only serve requests after the visitor has completed a challenge: The user (developer) must visit a page at cors-anywhere.herokuapp.com to temporarily unlock the demo for their browser. This allows developers to try out the functionality, to help with deciding on self-hosting or looking for alternatives.
CORS Anywhere is a node.js proxy which adds CORS headers to the proxied request.
To use the API, just prefix the URL with the API URL. (Supports https: see github repository)
If you want to automatically enable cross-domain requests when needed, use the following snippet:
$.ajaxPrefilter( function (options) {
if (options.crossDomain && jQuery.support.cors) {
var http = (window.location.protocol === 'http:' ? 'http:' : 'https:');
options.url = http + '//cors-anywhere.herokuapp.com/' + options.url;
//options.url = "http://cors.corsproxy.io/url=" + options.url;
}
});
$.get(
'http://en.wikipedia.org/wiki/Cross-origin_resource_sharing',
function (response) {
console.log("> ", response);
$("#viewer").html(response);
});
Whatever Origin
Whatever Origin is a cross domain jsonp access. This is an open source alternative to anyorigin.com.
To fetch the data from google.com, you can use this snippet:
// It is good specify the charset you expect.
// You can use the charset you want instead of utf-8.
// See details for scriptCharset and contentType options:
// http://api.jquery.com/jQuery.ajax/#jQuery-ajax-settings
$.ajaxSetup({
scriptCharset: "utf-8", //or "ISO-8859-1"
contentType: "application/json; charset=utf-8"
});
$.getJSON('http://whateverorigin.org/get?url=' +
encodeURIComponent('http://google.com') + '&callback=?',
function (data) {
console.log("> ", data);
//If the expected response is text/plain
$("#viewer").html(data.contents);
//If the expected response is JSON
//var response = $.parseJSON(data.contents);
});
CORS Proxy
CORS Proxy is a simple node.js proxy to enable CORS request for any website.
It allows javascript code on your site to access resources on other domains that would normally be blocked due to the same-origin policy.
CORS-Proxy gr2m (archived)
CORS-Proxy rmadhuram
How does it work?
CORS Proxy takes advantage of Cross-Origin Resource Sharing, which is a feature that was added along with HTML 5. Servers can specify that they want browsers to allow other websites to request resources they host. CORS Proxy is simply an HTTP Proxy that adds a header to responses saying "anyone can request this".
This is another way to achieve the goal (see www.corsproxy.com). All you have to do is strip http:// and www. from the URL being proxied, and prepend the URL with www.corsproxy.com/
$.get(
'http://www.corsproxy.com/' +
'en.wikipedia.org/wiki/Cross-origin_resource_sharing',
function (response) {
console.log("> ", response);
$("#viewer").html(response);
});
The http://www.corsproxy.com/ domain now appears to be an unsafe/suspicious site. NOT RECOMMENDED TO USE.
CORS proxy browser
Recently I found this one, it involves various security oriented Cross Origin Remote Sharing utilities. But it is a black-box with Flash as backend.
You can see it in action here: CORS proxy browser
Get the source code on GitHub: koto/cors-proxy-browser
You can use Ajax-cross-origin a jQuery plugin.
With this plugin you use jQuery.ajax() cross domain. It uses Google services to achieve this:
The AJAX Cross Origin plugin use Google Apps Script as a proxy jSON
getter where jSONP is not implemented. When you set the crossOrigin
option to true, the plugin replace the original url with the Google
Apps Script address and send it as encoded url parameter. The Google
Apps Script use Google Servers resources to get the remote data, and
return it back to the client as JSONP.
It is very simple to use:
$.ajax({
crossOrigin: true,
url: url,
success: function(data) {
console.log(data);
}
});
You can read more here:
http://www.ajax-cross-origin.com/
If the external site doesn't support JSONP or CORS, your only option is to use a proxy.
Build a script on your server that requests that content, then use jQuery ajax to hit the script on your server.
Just put this in the header of your PHP Page and it ill work without API:
header('Access-Control-Allow-Origin: *'); //allow everybody
or
header('Access-Control-Allow-Origin: http://codesheet.org'); //allow just one domain
or
$http_origin = $_SERVER['HTTP_ORIGIN']; //allow multiple domains
$allowed_domains = array(
'http://codesheet.org',
'http://stackoverflow.com'
);
if (in_array($http_origin, $allowed_domains))
{
header("Access-Control-Allow-Origin: $http_origin");
}
I'm posting this in case someone faces the same problem I am facing right now. I've got a Zebra thermal printer, equipped with the ZebraNet print server, which offers a HTML-based user interface for editing multiple settings, seeing the printer's current status, etc. I need to get the status of the printer, which is displayed in one of those html pages, offered by the ZebraNet server and, for example, alert() a message to the user in the browser. This means that I have to get that html page in Javascript first. Although the printer is within the LAN of the user's PC, that Same Origin Policy is still staying firmly in my way. I tried JSONP, but the server returns html and I haven't found a way to modify its functionality (if I could, I would have already set the magic header Access-control-allow-origin: *). So I decided to write a small console app in C#. It has to be run as Admin to work properly, otherwise it trolls :D an exception. Here is some code:
// Create a listener.
HttpListener listener = new HttpListener();
// Add the prefixes.
//foreach (string s in prefixes)
//{
// listener.Prefixes.Add(s);
//}
listener.Prefixes.Add("http://*:1234/"); // accept connections from everywhere,
//because the printer is accessible only within the LAN (no portforwarding)
listener.Start();
Console.WriteLine("Listening...");
// Note: The GetContext method blocks while waiting for a request.
HttpListenerContext context;
string urlForRequest = "";
HttpWebRequest requestForPage = null;
HttpWebResponse responseForPage = null;
string responseForPageAsString = "";
while (true)
{
context = listener.GetContext();
HttpListenerRequest request = context.Request;
urlForRequest = request.RawUrl.Substring(1, request.RawUrl.Length - 1); // remove the slash, which separates the portNumber from the arg sent
Console.WriteLine(urlForRequest);
//Request for the html page:
requestForPage = (HttpWebRequest)WebRequest.Create(urlForRequest);
responseForPage = (HttpWebResponse)requestForPage.GetResponse();
responseForPageAsString = new StreamReader(responseForPage.GetResponseStream()).ReadToEnd();
// Obtain a response object.
HttpListenerResponse response = context.Response;
// Send back the response.
byte[] buffer = System.Text.Encoding.UTF8.GetBytes(responseForPageAsString);
// Get a response stream and write the response to it.
response.ContentLength64 = buffer.Length;
response.AddHeader("Access-Control-Allow-Origin", "*"); // the magic header in action ;-D
System.IO.Stream output = response.OutputStream;
output.Write(buffer, 0, buffer.Length);
// You must close the output stream.
output.Close();
//listener.Stop();
All the user needs to do is run that console app as Admin. I know it is way too ... frustrating and complicated, but it is sort of a workaround to the Domain Policy problem in case you cannot modify the server in any way.
edit: from js I make a simple ajax call:
$.ajax({
type: 'POST',
url: 'http://LAN_IP:1234/http://google.com',
success: function (data) {
console.log("Success: " + data);
},
error: function (e) {
alert("Error: " + e);
console.log("Error: " + e);
}
});
The html of the requested page is returned and stored in the data variable.
To get the data form external site by passing using a local proxy as suggested by jherax you can create a php page that fetches the content for you from respective external url and than send a get request to that php page.
var req = new XMLHttpRequest();
req.open('GET', 'http://localhost/get_url_content.php',false);
if(req.status == 200) {
alert(req.responseText);
}
as a php proxy you can use https://github.com/cowboy/php-simple-proxy
Your URL doesn't work these days, but your code can be updated with this working solution:
var url = "http://saskatchewan.univ-ubs.fr:8080/SASStoredProcess/do?_username=DARTIES3-2012&_password=P#ssw0rd&_program=%2FUtilisateurs%2FDARTIES3-2012%2FMon+dossier%2Fanalyse_dc&annee=2012&ind=V&_action=execute";
url = 'https://google.com'; // TEST URL
$.get("https://images"+~~(Math.random()*33)+"-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=" + encodeURI(url), function(data) {
$('div.ajax-field').html(data);
});
<div class="ajax-field"></div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
You need CORS proxy which proxies your request from your browser to requested service with appropriate CORS headers. List of such services are in code snippet below. You can also run provided code snippet to see ping to such services from your location.
$('li').each(function() {
var self = this;
ping($(this).text()).then(function(delta) {
console.log($(self).text(), delta, ' ms');
});
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<script src="https://cdn.rawgit.com/jdfreder/pingjs/c2190a3649759f2bd8569a72ae2b597b2546c871/ping.js"></script>
<ul>
<li>https://crossorigin.me/</li>
<li>https://cors-anywhere.herokuapp.com/</li>
<li>http://cors.io/</li>
<li>https://cors.5apps.com/?uri=</li>
<li>http://whateverorigin.org/get?url=</li>
<li>https://anyorigin.com/get?url=</li>
<li>http://corsproxy.nodester.com/?src=</li>
<li>https://jsonp.afeld.me/?url=</li>
<li>http://benalman.com/code/projects/php-simple-proxy/ba-simple-proxy.php?url=</li>
</ul>
Figured it out.
Used this instead.
$('.div_class').load('http://en.wikipedia.org/wiki/Cross-origin_resource_sharing #toctitle');
$scope.readFile = function(url){
var doc = new XMLHttpRequest();
doc.onreadystatechange = function() {
if (doc.readyState == XMLHttpRequest.DONE) {
if (doc.responseText == undefined)
return;
$scope.loglines += doc.responseText;
}
}
doc.open("GET", url, true);
doc.send();
}
this is the code i used to get data from same origin using angularJS. how could i impore this to get data fron cross domain
To get data from cross domain, you can't do anything on client side, however you can set CORS on server side so that server allows you to send a cross origin request.
One other bad solution is to disable chrome same origin policy.
Disable same origin policy in Chrome
I thought I could catch an error in send like this
try {
xhr.send();
} catch(e) {
// fix-me: With the
// bookmarklet on a https page
// you can't even send a HEAD
// request due to security
// restrictions. Check for
// this case here.
console.log("xhr.send, e=", e, method, window.location.href, url)
debugger;
}
console.log("I am here now");
However I never get to that console.log statement in the catch block after xhr.send.
In the console I instead get a message like this.
Mixed Content: The page at 'about:blank' was loaded over HTTPS,
but requested an insecure XMLHttpRequest endpoint 'http://m.org/'.
This request has been blocked; the content must be served over HTTPS.
I am here now.
Is it supposed to work this way? (I am using Google Chrome.)
Is there any way to find out that there was an error? (Except looking in the console. ;-) )
UPDATE
#giuscri added the very good question if I did consider that this is async. I actually missed that it could be, but it is not. A bit surprisingly. ;-)
Please see the this example. It contains this code:
var url = "http://nowhere.org/";
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function() {
console.log("onreadystatechance, readyState=", xhr.readyState);
};
xhr.onprogress = function(event) {
console.log("onprogress, readyState=", xhr.readyState);
console.log("onprogress, event=", event);
};
xhr.onerror = function(event) {
console.log("onerror, readyState=", xhr.readyState);
console.log("onerror, event=", event);
};
var method = "HEAD";
xhr.open(method, url, true);
try {
xhr.send();
} catch(e) {
console.log("xhr.send, e=", e, method, window.location.href, url);
}
console.log("After send");
When you run this page from https:// (as in the link above) the onerror function is not run. If you run the same example from file:// then onerror is run.
Connecting from HTTPS to HTTP URIs drops the security given by the underlying encryption. Web browsers blocks such requests until explicitly allowed by the user in order to prevent data leakage over plaintext connections. Further, there is also a change in origin (scheme, domain, port).
I allowed Mixed Content for the page you linked and I got the error about the different origin in console. Looks like the code works.
By the way, support for synchronous requests using XMLHttpRequest is deprecated, because it blocks user interaction until the request completes.
I'm attempting to send a XMLHttpRequest to a paste site. I'm sending an object containing all the fields that the api requires, but I keep getting this issue. I have read over the issue, and I thought:
httpReq.setRequestHeader('Access-Control-Allow-Headers', '*');
Would fix it,but it didn't. Does anyone have any information on this error and/or how I can fix it?
Here is my code:
(function () {
'use strict';
var httpReq = new XMLHttpRequest();
var url = 'http://paste.ee/api';
var fields = 'key=public&description=test&paste=this is a test paste&format=JSON';
var fields2 = {key: 'public', description: 'test', paste: 'this is a test paste', format: 'JSON'};
httpReq.open('POST', url, true);
console.log('good');
httpReq.setRequestHeader('Access-Control-Allow-Headers', '*');
httpReq.setRequestHeader('Content-type', 'application/ecmascript');
httpReq.setRequestHeader('Access-Control-Allow-Origin', '*');
console.log('ok');
httpReq.onreadystatechange = function () {
console.log('test');
if (httpReq.readyState === 4 && httpReq.status === 'success') {
console.log('test');
alert(httpReq.responseText);
}
};
httpReq.send(fields2);
}());
And here is the exact console output:
good
ok
Failed to load resource: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://127.0.0.1:40217' is therefore not allowed access. http://paste.ee/api
XMLHttpRequest cannot load http://paste.ee/api. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://127.0.0.1:40217' is therefore not allowed access. index.html:1
test
Here is the console output when I test it locally on a regular Chromium browser:
good
ok
XMLHttpRequest cannot load http://paste.ee/api. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access. index.html:1
test
I think you've missed the point of access control.
A quick recap on why CORS exists:
Since JS code from a website can execute XHR, that site could potentially send requests to other sites, masquerading as you and exploiting the trust those sites have in you(e.g. if you have logged in, a malicious site could attempt to extract information or execute actions you never wanted) - this is called a CSRF attack. To prevent that, web browsers have very stringent limitations on what XHR you can send - you are generally limited to just your domain, and so on.
Now, sometimes it's useful for a site to allow other sites to contact it - sites that provide APIs or services, like the one you're trying to access, would be prime candidates. CORS was developed to allow site A(e.g. paste.ee) to say "I trust site B, so you can send XHR from it to me". This is specified by site A sending "Access-Control-Allow-Origin" headers in its responses.
In your specific case, it seems that paste.ee doesn't bother to use CORS. Your best bet is to contact the site owner and find out why, if you want to use paste.ee with a browser script. Alternatively, you could try using an extension(those should have higher XHR privileges).
I've gotten same problem.
The servers logs showed:
DEBUG: <-- origin: null
I've investigated that and it occurred that this is not populated when I've been calling from file from local drive. When I've copied file to the server and used it from server - the request worked perfectly fine
function cors() {
var xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) {
document.getElementById("emo").innerHTML = alert(this.responseText);
}
};
xhttp.withCredentials = true;
xhttp.open("GET", "http://owasp-class.lab:4444/api/get_info", true);
xhttp.setRequestHeader('Content-Type', 'application/x-www-form-urlencode');
xhttp.send();
}