I am trying to create a reverse-proxy web application using flask. I don't really know it is called reverse-proxy exactly but my webapp gets url from ../proxy/<url> and using requests and bs4 in python scrape the website and do some change in <a href="</test>"> to <a href="../proxy/<domain>/test"> and same for <form> and for other links also in the website for making sure that next request to the page is through proxy but some page that uses JavaScript to load data doesn't goes through ../proxy, they just send request like a normal website.
Now here is my question,
Is there any way to create an environment like things to make every link go through proxy or what could be the other best way to do such reverse-proxy things in flask? How to make sure that each and every time that website connect to network then connect through the proxy?
Edit: Idea:
Making browser log like things using JavaScript, but while any link want to connect to internet then manipulate it into /proxy/<url>.
CORS: I don't know what to say but make CORS or using CORS to block request to other domain.
Related
CONTEXT
In a Meteor app, I'm using a reverse-proxy to load the content of a third party site into an iframe. This allows me to inject CORS headers into the site, bypassing the cross-origin policy to allow me to make changes to the site. In particular, I'm using the http-proxy npm package: https://github.com/nodejitsu/node-http-proxy
PROBLEM
Since each user of my app should be able to load a different site into an iframe and make changes to it, I am wondering if it I need to:
Create a separate server each time I make a request for a new website, or
I can have a single proxy server, through which all website requests are passed
What would this look like - would I be loading proxyaddress/siteUrl into each user's iframe?
Is there a way with javascript on a page to tamper http requests done by other scripts (on same page)? The other scripts can be from external domains.
Let's say on a page X a script loaded from an external domain performs an http get like GET http://www.example.com?foo=bar is it possible that a previously loaded script in the same page X can capture this request, and tamper with it so it becomes GET http://www.example.com?foo=qux?
In jquery i can achieve this by wrapping the ajax get post methods. But is this possible for plain javascript, no frameworks, working across all page and client's http requests?
It seems to me that for this to be done, the script must be able to override something very deep in the core. If i have to guess i would say it's not possible by design and because of security. What do you think?
p.s. no proxies, no external tools.
No you can't capture the HTTP request going out of your app, once it has gone out of your app.
However, there is one work around possible
if you want to alter some parameters before it goes of your app,
All the other request from your website are AJAX requests
They are invoking your custom method, say customAjax() which can
alter the parameters of the actual request that will go out.
I have a PHP app that renders HTML pages for a social media application that I'm creating. Then, JavaScript initializes and makes things interactive. The PHP side of things logs into a separate webservice with curl.
Now, I can't figure out a way to share the session started in PHP with JavaScript, so when I make a AJAX request in JavaScript to the data server, its authenticated.
Is there a way to share a PHP session with JavaScript? Or to share authentication initially created with PHP with JavaScript?
I would say it sounds like there is something wrong with your architecture. In my opinion, the web server itself, should be the only peer providing data to the client/browser. It's a two party conversation only.
When trying to hit a third-party server from the browser, you violate the browsers Same-Origin Policy, unless you specifically allow CORS by explicitly setting various request and response headers. - and you would only do so in very special situations.
The best solution might be to create proxy services at the web server, that can be hit directly (locally) by the browser. The web server can then (acting as controller) forward the data-request to the data server (model) and finally return the response to the browser (view).
You can read out the session cookie set by PHP (SID I guess) through JavaScript containing the session ID.
When you make a query, use
http://example.com/?sid=SessionID
Is there a way to send data packets from an active Python script to a webpage currently running JavaScript?
The specific usage I'm looking for is to give the ability for the webpage, using JavaScript, to tell the Python script information about the current state of the webpage, then for the Python script to interpret that data and then send data back to the webpage, which the JavaScript then uses to decide which function to execute.
This is for a video game bot (legally), so it would need to happen in real time. I'm fairly proficient in Python and web requests, but I'm just getting into JavaScript, so hopefully a solution for this wouldn't be too complex in terms of Javascript.
EDIT: One way I was thinking to accomplish this would be to have Javascript write to a file that the Python script could also read and write to, but a quick google search says that JavaScript is very limited in terms of file I/O. Would there be a way to accomplish this?
For security reasons, javascript in a browser is usually restricted to only communicate with the site it was loaded from.
Given that, that's an AJAX call, a very standard thing to do.
You can make HTTP requests using the XMLHttpRequest API, which Jquery abstracts with $.ajax and $.get. You can also use the lower level Websockets network API:
https://developer.mozilla.org/en-US/docs/WebSockets
Note that the XMLHttpRequest API will only allow requests to the same server, OR requests that return an appropriate Access-Control-Allow-Origin header.
It sounds like the Javascript is only going to send information, not receive any. In that case, you're in luck. I'm guessing you are also running the Javascript and the Python on the same machine.
Run a Python webserver on the machine the browser is running on. Here's a simple example:
http://webpy.org/install
Once visiting http://127.0.0.1:8080/ in your browser gives the message Hello World!, you can start adding more addresses to your website, for example http://127.0.0.1:8080/report_data, http://127.0.0.1:8080/report_event etc.
Your Javascript can then make AJAX requests using jQuery.ajax or XMLHTTPRequest, to the address http://127.0.0.1:8080/report_data, and pass the information as GET parameters.
I need to offer a web service that my clients can use on their web sites with AJAX. They are not able to call my web service because of XSS preventions. The clients can not make a proxy to access my web service.
I am trying to make a javascript library on my server that they could include in their site, which would in turn call the web service on the server. Somehow it does not seem to work.
The server is located at Google App Engine.
So the question is: How can I make a javascript library on my server that uses backend on that server and remote users can use it? Much like google maps js API works?
You should use Cross Origin Resource Sharing instead, just set CORS http headers for your web service.
Access-Control-Allow-Origin: http://clientsite.com http://client.website.com
Same origin policy is dependant on document origin therefore providing a JavaScript library will not help.
Two possibilities:
have your javascript library create an iframe pointed at your server. Communicate between the code running in that iframe and the 3rd-party site via the best crosspage communication for the browser you're on. Google's Closure library has a class called CrossPageChannel that works very well for this. Put the bulk of your logic in the iframe. This can be nice because it'll prevent the 3rd-party site from doing anything that isn't well-defined by the messages you pass across the iframe boundary.
use JSONP to get data from your server and keep all the logic in the javascript library.