We have a node.js app that displays content on partner websites. Our partners include some javascript code in their HTML templates, pointing to our server, and we then use the URL of the page which loads our code to figure out what content to show. Being a third-party script, we mostly rely on document.referrer to get the URL of the including page, but sometimes we are included in an iframe, which complicates things. We are looking to make this code more robust by writing a comprehensive test suite.
Is there a good way to test this type of functionality on the server? We’d need to emulate a browser with javascript support, and it would need to report the referrer like a normal browser would. We’d also need to be able to test cross-origin issues on the server, by simulating domain names, ports etc.
It would be preferable to use node.js for this, but that’s not a requirement.
Related
Apologies if this is a roundabout way of asking this question, but I am a little confused about how the web and javascript work.
What I want to do: execute javascript on all pages of a list of urls I have found. (Specifically use jquery to pull info from them)
Problem I can't execute Javascript on these pages because they aren't mine and don't have the Access-Control-Allow-Origin header. So I can't load them (with AJAX) in order to use JQuery on them.
BUT Google Chrome can both load pages and execute javascript on them (with their developer's console). So if I wanted too, I could go to each page, open the developers console, and pull the information from there. If there's nothing stopping Chrome from accessing these, then why am I stopped? And, is there a way around this?
Thank you, and I hope my description makes sense. I've been researching this for a while but have found nothing that explains how seemingly inconsistent CORS is.
I could go to each page, open the developers console, and pull the information from there. If there's nothing stopping Chrome from accessing these, then why am I stopped?
You're not stopped. You, the human at the keyboard, can do exactly as you say, by visiting each page as a top-level page.
What is stopped -- happily -- is any and all scripts on the Web you happen to run having the same level of visibility that you do. Based on your cookies and your network topology, you have a unique view into the Web. You can see your home router's control interface (on 192.168.1.1 or similar). You can see any local web server you're running on 127.0.0.1. No one else can see these. If the same-origin policy were not in place, then any script that you loaded on the Web could inspect these.
And, is there a way around this?
If you have some scripts that you trust absolutely (hopefully a significant subset of "all scripts that exist on the Web") that you want to be able to bypass the same-origin policy and see your full, cross-domain view of the Web, you could load them as an extension, which can act with elevated permissions beyond the abilities of normal web pages. (See How does Same Origin Policy apply to browser extensions?)
I'm going to assume that you are looking to grab data from these pages that aren't yours and store it somewhere. I have done this before with curl using php. If you are looking to display these sites for users to interact in a different way, but starting from a page that is yours, you may be able to render these pages by grabbing the source html using curl and rendering it as a sort of proxy.
I've used this tutorial for something similar https://www.youtube.com/watch?v=_kQN-3aNCeI . Hopefully this gives you a start. I think you should be a little more detailed in your question though to get more help.
So I'm playing a game online on my laptop and it is a pretty simple html5 game but the optimization is nonexistant. The game with a few circles on my screen is using 85% of my cpu.
Logically, I started profiling the game and was trying to figure out if I could optimize it for my laptop. Now, what I'm wondering is how could I run the game but with a tweaked version of the JS script that is running.
If I try to save the page and run it from my laptop it of course has CORS issues so I can not access the server.
Can I somehow change the script a bit which is executing in my browser but while still staying "inside the webpage" which is running so that I may normally make XHR requests to the server?
Also, although this is not in the title of the question, can I somehow proxy the XHR request to my script but while not breaking the CORS rule?
Why is it so different if I run the same thing from my browser and from a saved HTML on my desktop? I have the same IP and am doing the same thing but from the url it "feels like" I'm running it from somewhere else. Can I somehow imitate that I'm running "from the webpage" but instead run it from a modified saved html?
You could proxy, given there isn't a cross domain protection mechanism or some sort of logging in (which complicates stuff).
What you could very well do is use a browser extension which allows you to add CSS, HTML and JavaScript.
I'm not entirely savvy on extensions so I'm not sure you can modify existing code but I'm guessing that if you can add arbitrary JS code you may very well replace the script tag containing the game for a similar personally modified script based on it. It's worth a try...
Link to getting started with chrome extensions
Update:
If you're set on doing it, proxying ammounts to requesting an URL with your application and do something with the page (html) instead of the original source. I assume you want to change the page and serve it to your browser.
With this in mind you will need the following, I dont know C# so you'll have to google around for libraries and utilities:
a way to request URLs (see link at bottom)
a way to modify the page, you need a DOM crawler
a way to start said process and serve it to your browser by hitting your own URL, meaning you need some sort of web server
I found the following question specifically on proxying with C#
This has been asked a lot of times already: I need to get the full file path via a web-page. The use case is an application running on the same machine as the browser (i.e. the application starts a local HTTP server and fires up the browser.) File-paths are of course valid and the same for both client/server now. The use case it that the user selects a file and then the server process does some computation on it, and the input files are typically large (read: several GiB in size.)
The easiest thing would be to directly access the path using , but for security reasons, this is disabled. I'm looking now for the least intrusive workaround to this problem. The target browser in question is Chrome. I'm fine if the user has to click "accept" once on some security warning, as long as I can ensure that it won't appear again.
Do I have to write an extension, NSPlugin, can I use some special header magic to mark my page as "local", is there some security setting I can set? The less the client has to do the better, and I would prefer some "click here to allow access ..." solution above everything else. Can I directly install an extension from the server process that would do this (after the user clicks accept?)
Is it possible to do this with a Java applet/Flash? That would be the easiest solution, and clients are guaranteed to have Flash installed (as it is bundled in Chrome...)
You can create Java applet for tasks like this and self-sign it. User will have to allow it to run, but then you will be able to access applet's function that will return file path string via Javascript.
Clearly file io on the client's system is forbidden from JavaScript. If this wasn't the case it would be absolutely trivial to hack every web browser that visits your website.
Battlefiled 3 is controlled though the browser. To do this EA wrote a browser extension for the top three browsers. But that's resource intensive. If you just care about chrome you can use an addon, and for that i suggest using the NPAPI.
And as MOleYArd said, Java is a good solution and probably more common than an extension or addon.
I apologize if this has been asked before. I searched but did not find anything. It is a well-known limitation of AJAX requests (such as jQuery $.get) that they have to be within the same domain for security reasons. And it is a well-known workaround for this problem to use iframes to pull down some arbitrary HTML from another website and then you can inspect the contents of this HTML using javascript which communicates between the iframe and the parent page.
However, this doesn't work on the iPhone. In some tests I have found that iframes in the Safari iPhone browser only show content if it is content from the same site. Otherwise, they show a blank content area.
Is there any way around this? Are there other alternatives to using iframes that would allow me to pull the HTML from a different domain's page into javascript on my page?
Edit:
One answer mentioned JSONP. This doesn't help me because from what I understand JSONP requires support on the server I'm requesting data from, which isn't the case.
That same answer mentioned creating a proxy script on my server and loading data through there. Unfortunately this also doesn't work in my case. The site I'm trying to request data from requires user login. And I don't want my server to have to know the user's credentials. I was hoping to use something client-side so that my app wouldn't have to know the user's credentials at the other site.
I'm prepared to accept that there is no way to accomplish what I want to do on the iPhone. I just wanted to confirm it.
You generally can NOT inspect the contents of an iframe from another domain via JavaScript. The most common answers are to use JSONP or have your original server host a proxy script to retrieve the inner contents for you.
Given your revisions, without modification or support from the secondary site, you are definitely not going to be able to do what you want via the iPhone's browser.
"In some tests I have found that iframes in the Safari iPhone browser only show content if it is content from the same site"
I found the same thing. Is this documented somewhere? Is there a workaround? This sounds like broken web standards to me, and I am wondering if there is a solution.
Basically, what I'm trying to do is simply make a small script that accesses finds the most recent post in a forum and pulls some text or an image out of it. I have this working in python, using the htmllib module and some regex. But, the script still isn't very convenient as is, it would be much nicer if I could somehow put it into an HTML document. It appears that simply embedding Python scripts is not possible, so I'm looking to see if theres a similar feature like python's htmllib that can be used to access some other webpage and extract some information from it.
(Essentially, if I could get this script going in the form of an html document, I could just open one html document, rather than navigate to several different pages to get the information I want to check)
I'm pretty sure that javascript doesn't have the functionality I need, but I was wondering about other languages such as jQuery, or even something like AJAX?
As Greg mentions, an Ajax solution will not work "out of the box" when trying to load from remote servers.
If, however, you are trying to load from the same server, it should be fairly straightforward. I'm presenting this answer to show how this could be done using jQuery in just a few lines of code.
<div id="placeholder">Please wait, loading...</div>
<script type="text/javascript" src="/path/to/jquery.js">
</script>
<script type="text/javascript>
$(document).ready(function() {
$('#placeholder').load('/path/to/my/locally-served/page.html');
});
</script>
If you are trying to load a resource from a different server than the one you're on, one way around the security limitations would be to offer a proxy script, which could fetch the remote content on the server, and make it seem like it's coming from your own domain.
Here are the docs on jQuery's load method : http://docs.jquery.com/Ajax/load
There is one other nice feature to note, which is partial-page-loading. For example, lets say your remote page is a full HTML document, but you only want the content of a single div in that page. You can pass a selector to the load method, as in my example above, and this will further simplify your task. For example,
$('#placeholder').load('/path/to/my/locally-served/page.html #someTargetDiv');
Best of luck!-Mike
There are two general approaches:
Modify your Python code so that it runs as a CGI (or WSGI or whatever) module and generate the page of interest by running some server side code.
Use Javascript with jQuery to load the content of interest by running some client side code.
The difference between these two approaches is where the third party server sees the requests coming from. In the first case, it's from your web server. In the second case, it's from the browser of the user accessing your page.
Some browsers may not handle loading content from third party servers very gracefully (that is, they might pop up warning boxes or something).
You can embed Python. The most straightforward way would be to use the cgi module. If the script will be run often and you're using Apache it would be more efficient to use mod_python or mod_wsgi. You could even use a Python framework like Django and code the entire site in Python.
You could also code this in Javascript, but it would be much trickier. There's a lot of security concerns with cross-site requests (ah, the unsafe internet) and so it tends to be a tricky domain when you try to do it through the browser.