I tried to scrape a website just for learning purposes. I encountered serious form security: I tried to enter the form value using
element.value = "some text". The value appeared inside the input field but when I hit submit button it disappeared and the form got submitted with no value for the input field. My question is,
what type of security is this?
what i can do bypass it?
I tried various solutions such as using jquery to send keys and billiteRange library also tried to dispatch keyboardEvent but nothing works for me. I have been trying for the last 5 days but I've had no luck with this. Please guide me I want to do this.
To my knowledge, there is (currently) no browser side security on a javascript/html level. The best thing a website can do on that level to protect a form from any injected javascript code, is to hide a key (maybe an anti-csrf key) and use it when submitting the form.
On any other case (when there is no key or when you can access and read the key) you can just re-create the hole form and submit it, or even just fake the request to the server. Thus, there is no point in developing codes to protect forms, unless you want to hide the way you generate data that you will send to the server (but, I don't consider this a protection).
In your case, it seems that the javascript code that handles the submission of the form unintentionally or intentionally blocks your attempts to submit the form. This may be due to a validation function that does not fire (most probably). For example, onchange event does not fire when you change the value of an input.
You can see all the events on the form and its inputs from the developer tools of your browser and figure out how the webpage handles the form, but the easiest way to fool the webpage is to fake the hole request.
Nice Answer by GramThanos you helped me a lot it's really a nice explanation, helped me to break it.
Just
use
var event = new Event('eventName', {bubbles: 'true'});
element.dispatchEvent(event);
Related
So I recently signed up with anti-captcha and have been testing with the https://github.com/ad-m/python-anticaptcha/blob/master/examples/recaptcha_selenium.py script.
I cannot get past a reCAPTCHA that has no submit button (hidden or visible) nor a clear way to submit for verification. I've used the anti-captcha firefox plugin, so I know it can be passed. But I am stuck at the point of doing this manually myself.
I thought this was going to be a helpful answer, but it doesn't go into depth. I am able to get the job.get_solution_response() token and enter it into the required textfield, but I cannot submit the "form."
Does anyone have success with this? I am also looking to do this in a headless version of the browser. Would a solution be different based on headless vs non-headless?
BTW, realtor(dot)com is the website I am having trouble with. If I am not aloud to post this site, please let me know so I can remove it.
I went back into the source code of the site and found they are using a function as a callback for verification...
I saw that they are using a function called solvedCaptcha and injecting a variable name payload. So here is how I solved it:
NOTE: Make sure the driver is currently on the reCAPTCHA page.
driver.execute_async_script("var payload = '<enter the job.get_solution_response() here>'; solvedCaptcha(payload);")
This async script then calls the page's verification and it reloads the blocked window.
NOTE: This CAPTCHA did not have a submit button and was not placed in a form. So using .submit() on an item within the driver would not work.
try to sendKeys of \n or Keys.Enter to the textfield.
This was my solution in my projects.
Also this is generic one
This is going to be a difficult one to explain but basically I have an angular application with a login form which runs a function on submit which sends an ajax request to the server to do the login.
Now, I'm not using ng-submit but hijacking the normal submit attribute like this:
<form my-form submit="controllername.doSubmit()">
I then have an angular directive called "my-form" which uses {require: 'form'} in it's definition object and then does this in the postLink function:
element.bind('submit', function(event)
{
// Removed for brevity
scope.$apply(scope.submit);
});
So, basically this form submit stuff was written a long time ago and does a lot of other stuff like triggering form validation and stuff by default so I don't wnt to rewrite any of this or go back to using ng-submit. Aside form anything else I have a few big apps using this code which would need to change a lot.
Anyway, it all works fine on the surface but if I fill in the log in form and then do some other stuff (including filling in other forms set up the same way) and then leave my laptop for a few days and come back to the page, somehow all the form data has been added into the URL bar, after the ?? and before the # including the password in plain view!
Not sure why this doesn't happen straight away, ony after un-sleeping the PC, and not always. The other weird thing is that the names of the parameters are not the original ones (email, password) but the names of the parameters of the first form currently on the page (actionStatus, required), so Chrome is obviously getting very confused.
My instinct tells me that when the form is submitted, the formData is being stored somewhere for later because I'm not cancelling the default action of the form correctly when I'm running my javascript function and because it's a single page application that formData never leaves the memory. It's then thinking it's gone to a new page and putting that data in the URl, but it's getting the names wrong because the forms on the page has changed.
Sorry, I can't provide more code, just a fairly wooly description but I don't know what else to say really, it's all very strange.
I finally have an answer to this and the answer is totally unexpected.
I use a plugin called browserSync to sync multiple browsers I'm testing on and it had a feature called form syncing switched on.
I had chrome open and when I opened up another browser (typically in the morning when the computer first woke up) I had to log in and because they were on different pages it was copying my login details from Firefox into the first form it found in my existing chrome window. It was also putting the contents of Firefox's login form into chrome's URL.
Crazy.
Before submitting a form, i use javascript code (surrounded in PHP) in order to make locally some controls but sometimes javascript may not be enabled client-side.
The fact is that I have to check by pattern/regex each control of the form for example checking email, phone number,.. format so that user cannot enter anything haphazardly. Therefore, if javascript is not enabled, the form must not be submitted, even if all field are fulfilled out.
Therefore my question is to know if there is a tag or function which allow to perform what i want to?
Thank for your help
JavaScript runs client-side.
That means that users have FULL CONTROL over it.
Then, if they want to disable it, you can't do anything about it.
The only thing you should do is be sure that users with JS disabled will be able to submit the form too.
If you use JS to validate the form, be aware that users have FULL CONTROL over it, so they can send the form if they want, even if your code says that it's invalid.
The right way to do it is:
Be sure users without javascript can send the form
Implement client-side validation for users with javascript activation. This way they will have a better user experience (because can know if the data is invalid immediately) and is less server intensive (your server will have to validate less invalid forms).
ALWAYS validate the submited form server-side. Data coming from a client is always UNTRUSTED, even if you think you have validated it.
In all of the sample code I have seen, it appears that the only function of onbeforeunload is to serve up an alert dialog box prior to the person leaving the page. Is that the only thing that can be triggered by the event or is it possible to do something else, like an unobtrusive function that sends off partial form data?
I am trying to capture abandoned shopping carts in Yahoo! Small Business and unfortunately I do not have access to any server side scripting, so I'm forced to work client-side only.
I was also thinking of doing an ajax posting of data after the email field was changed, and then comparing the list of all forms partially submitted against completed carts to determine which were incomplete.
You can save the partial form data in localStorage. Then, when another page is loaded, you could check for the presence of that data and AJAX it to the server, deleting it from localStorage on success. Or you might be able to just use that data in JavaScript, without involving the server, but that depends on your setup.
<body onbeforeunload="return ('You will lose all your data')" onunload="alert('You have gone away!')">
</body>
Onbeforeunload uses for alert box. Onunload for anything else.
You can technically fire off an ajax event, but there is no guarantee that it will complete before the page is actually reloaded.
Internet Explorer (with default settings, which I generally assume will be in effect on the desktops of the Great Unwashed) seems to dislike the idea of accepting attachment content in an HTTP response if the corresponding request wasn't made directly from a user action (like a "click" handler, or a native form submit). There are probably more details and nuances, but that's the basic behavior that's frustrating me.
It seems to me that this situation is common: the user interface in front of some downloadable content — say, a prepared PDF report — allows for some options and inputs to be used in the creation of the content. Now, as with all forms that allow the user to stipulate how an application does something, it's possible that the input will be erroneous. Not always, but sometimes.
Thus there's a dilemma. If the client tries to do something fancy, like run an AJAX transaction to let the server vet the form contents, and then resubmit to get the download, IE won't like that. It won't like it because the actual HTTP transaction that carries the attachment back will happen not in the original user-action event handler, but in the AJAX completion callback. Worse, since the IE security bar seems to think that the solution to all one's problems is to simply reload the outer page from its original URL, its invitation to the user to go ahead and download the suspicious content won't even work.
The other option is to just have the form fire away. The server checks the parameters, and if there's anything wrong it responds with the form-container page, peppered appropriately with error messages. If the form contents are OK, it generates the content and ships it back in the HTTP response as an attached file. In this case (I think), IE is happy because the content was apparently directly requested by the user (which is, by the way, a ridiculously flimsy way to tell good content from bad content). This is great, but the problem now is that the client environment (that is, the code on my page) can't tell that the download worked, so the form is still just sitting there. If my form is in some sort of dialog, then I really need to close that up when the operation is complete — really, that's one of the motivations for doing it the AJAX way.
It seems to me that the only thing to do is equip the form dialogs with messaging that says something like, "Close this when your download begins." That really seems lame to me because it's an example of a "please push this button for me" interface: ideally, my own code should be able to push the buutton when it's appropriate. A key thing that I don't know is whether there's any way for client code to detect that form submission has resulted in an attachment download. I've never heard of a way to detect that, but that'd break the impasse for me.
I take it you're submitting the form with a different target window; hence the form staying in place.
There are several options.
Keep the submit button disabled and do ongoing validation in the background, polling the form for changes to fields and then firing off the validation request for a field as it changes. When the form is in a valid state, enable the button; when it isn't, disable the button. This isn't perfect, as there will tend to be a delay, but it may be good enough for whatever you're doing.
Do basic validation that doesn't require round-trips to the server in a handler for the form's submit event, then submit the form and remove it (or possibly just hide it). If the further validation on the server detects a problem, it can return a page that uses JavaScript to tell the original window to re-display the form.
Use a session cookie and a unique form ID (the current time from new Date().getTime() would do); when the form is submitted, disable its submit button but keep it visible until the response comes back. Make the response set a session cookie with that ID indicating success/failure. Have the window containing the form poll for the cookie every second or so and act on the result when it sees it. (I've never done this last one; not immediately seeing why it wouldn't work.)
I expect there are about a dozen other ways to skin this cat, but those are three that came to mind.
(Edit) If you're not submitting to a different target, you might want to go ahead and do that -- to a hidden iframe on the same page. That (possibly combined with the above or other answers) might help you get the user experience you're looking for.
There's a whole number of really good reasons IE does this, and I'm sure it's not something anyone would argue with - so the main objective is to get around it somehow to make things better for your users.
Sometimes its worth re-thinking how things are done. Perhaps disable the button, use javascript to check when all the fields are filled out, and fire off an ajax request once they are. If the ajax was successful, enable the button. This is but one suggestion, I'm sure there will be more...
Edit: more...
Do simple submission (non-AJAX), and if the checks fail, send a page back rather than an attachment. The page sent back could contain all the information originally submitted (plus whatever error message to the user) so the user doesn't need to fill out the entire form again. And I'm also sure there will be more ideas...
Edit: more...
I'm sure you've seen this type of thing before - and yes, it is an extra click (not ideal, but not hard).... an "if your download fails, click here" -> in this case, do it as you want to do it, but add a new link/button to the page when the AJAX returns, so if the download failed, they can submit the already validated form from a "direct user action". And I'm sure I'll think of more (or someone else will).....
I have been fighting a similar issue for a while. In my case, posting to a hidden iframe didn't work if my web app was embedded in an iframe on another site (third party cookie issues) unless our site was added to the Trusted Sites list.
I have found that I could break up the download into POST and GET sequence. The post returns a short lived GUID that can be used in a GET request to initiate the download. The POST can do the form validation as well as return the GUID in a successful response. Once the client has the GUID, you can set the src property of a hidden iframe element to the download URL. The browser sees the 'Content-Disposition': 'attachement' header and gives the user a download ribbon to download the file.
So far it appears to work in all the latest browsers. Unfortunately it requires you to modify you server side API for downloading the file.