I need to mimic the behavior of a browser. For Say I need to acess the DOM properties of a webpage (ex. Document.cookie or window.onblur etc) without loading a webpage in an actual browser and without interacting (ex. clicking button, putting mouse over a link etc) in the browser.
Infact, I am trying to do some thing where I have an imaginary browser Object BROWSER. So, I can do :
BROWSER g = BROWSER.load('google.com');
g.document.cookie();
g.window.onblur();
I guess this is known as 'browser instrumentation'. How Can I do it ? Any ideas.. ?
Your best bet is probably Selenium (http://seleniumhq.org/). Although it does use a real browser (which is sort of necessary if you want to test things in a real browser environment), it will allow you to completely automate/control that browser to make it do whatever you want. Using it you can write code like this:
# Pseudo-code to search for Selenium on Google
browser.open('www.google.com')
browser.findElementByCSS('#search').value('Selenium')
browser.findElementByCSS('#submitButton').click()
which sounds like what you are trying to do.
Is this something you're looking for? It's called PhantomJS
From the site:
Full web stack, No browser required
PhantomJS is a headless WebKit with JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.
Run functional tests with frameworks such as Jasmine, QUnit or CasperJS.
Related
I'm currently working on personal automation project and I want to add 'minimize app' feature.
For example: When I click button, I want to minimize the current browser. I already have window.close(); but this will close the app definitly. I found things like window.minimalize(); or window.minimize();, but none of them worked for me. So is there a way to minimize app?
Edited: It does not have to be only client-side JS, it can be used as terminal based nodejs app. Ex: I type minimize Google Chrome, and it will minimize it.
One Google search for how to minimize browser gives the following result:
There is no way to minimize the browser window within javascript. No, there isn't. However, depending on what you're doing and which browsers you're targeting, you could play around with the blur and focus events of the window to achieve similar effect.
Alternatively, if such control is required, you could always port your code to Electron.js or Neutrino.js, which were made to let you create desktop applications using JavaScript.
Maybe if you post some code and give us a train of thought to follow, we could work something out.
I don't have a lot of hope for this one, but I have to ask. I am hoping, for didactic purposes, to come up with some means by which a student could load a simple javascript program into a browser, and have it interact with them in the old-fashioned command line manner, where it prints a line and then reads a line of input. This works fine if you use prompt(), but the fact that it creates popups is aesthetically annoying, and today's browsers cheerfully volunteer to stifle the scripts which overuse it. The problem is, prompt() appears to be the only way browser Javascript has of actually pausing a script to wait for input. If we avoid it, that throws us immediately into having to deal with a real-time GUI event input model.
I've been looking for a way to fake it -- to set up some kind of environment in which it is possible for a Javascript method to wait for input and then return when it's given. The best possibility I've got so far is to connect it to a Java applet, but the java applet brand is kind of poisoned now and I doubt people would want to install the plugin. Could there be another way? Worker threads? A browser add-on? Some server-side trick? Does anyone have an idea?
This question is now years old... I wonder if the addition of promises and async/await to Javascript makes this any more possible now?
...I sort of got it to work. If the read-eval-print loop is in an async function and uses await for the line that reads input, you can indeed make an old fashioned linear read-eval-print loop in an event-driven browser environment. But I don't think you can make the main thread wait on input without an async function, except with the grandfathered prompt method (which they're now getting ready to deprecate).
The technical term for what you want is a REPL: a Read-Evaluate-Print-Loop. There are many REPLs out there.
The Best Browser-Based Solution
Use Google Chrome's inbuilt console! It's a Javascript REPL that can also let you interact with a web page. You can access it by using Ctrl+Shift+I on Windows and Unix inside Chrome, or the equivalent command on a Mac (Google it!).
To load a file and play with it, all your students need to do is create a directory structure like this:
project
|--> index.html
|--> javascript.js
make sure index.html has a script tag that points to javascript.js, and then open index.html with Chrome. Voila! You have loaded a Javascript file, and can now play around with it in Chrome.
It's the best solution because you get the full power of the DOM, can do REPL stuff, and is virtually painless - everyone uses Chrome, and your students can even go home and mess around with it completely.
A Browser-Based Alternative
Rather than reinvent the wheel, you can also use Repl.it.
It's a browser-based Javascript REPL website that supports inserting an arbitrary Javascript program, and interacting with its contents. This is the closest you'll probably get to meeting your requirements - it'll be unable to interact with the DOM (obviously), but it'll more than work.
Non-Browser-Based Alternative
If the requirement for a web-based solution can be relaxed, simply using Node.js' inbuilt REPL on a terminal can be more than sufficient.
You could install Node on your lab's computers, and have people play with Javascript in that capacity. There'll be no DOM, but you could certainly have them write functions and algorithms to solve simple problems. Plus, it'd be a good way to introduce them to the fact that Javascript is no longer client-side-bound.
The fact that you'll be interacting with Javascript in an actual terminal, rather than an emulated one in the browser, is another neat bonus.
If I've Misunderstood...
Some of the question comments make it sound like you want a way to be able to interact with terminal utilities using Javascript from a browser. If that is the case, this is impossible.
There is no way for Javascript to evaluate, parse or do anything on the command line that isn't written using Javascript. You cannot expect the equivalent of a bash ls using a browser-based solution - that's because browsers don't have access to your underlying filesystem, which is a good thing. You cannot run sed, awk, grep, etc. for the same reason - Unix utilities are inaccessible to a browser. There are ways to run Unix utilities using Node, of course, but then you will be teaching them how to use Node, rather than how to play with the Unix console.
If all you want, however, is a way to SSH from a browser into a common environment, there are certainly browser-based ways to do that. FireSSH is a Firefox plugin (now also ported to Chrome) that lets people SSH into a common server. They can then do ls, vim, etc. and have it run in the server, with the results piped back to their browser screen. You'll have to think carefully about security in this case, of course, but I think simply giving people user permissions for this server should more than suffice.
Note that FireSSH doesn't use Javascript to parse or do anything - all it is doing is relaying commands you type to a server, having the server execute those commands remotely, and then piping the results back to your screen.
Alternatives to Prompt
I added this after understanding OP's requirements in more detail.
This is a question that has been asked before. I am fond of library solutions, and in 2013, iocream.js was developed for just this sort of browser functionality. You can embed it in a page, and use the jin function to assign values.
If going with a Node.js solution, by far the best approach is to make use of the prompt library. I personally find it very useful for embedding within Node.js applications.
SpiderMonkey is Mozilla's C++-based Javascript engine, and supports a function called readline(). Unfortunately, there doesn't appear to be a mainstream browser implementation.
I want to use Mechanize to simulate browsing to a web page with active JavaScript, including DOM Events and AJAX, and so far I've found no way to do that.
I looked at some Python client browsers that support JavaScript like Spynner and Zope, and none of them really work for me. Spynner crashes PyQt all the time, and Zope doesn't support JavaScript as it seems.
Is there a way to simulate browsing with Python only (no extra processes) like WATIR or libraries that manipulate Firefox or Internet Explorer while supporting Javascript fully as if actually browsing the page?
I've played with this new alternative to Mechanize (which I love) called Phantom JS.
It is a full web kit browser like Safari or Chrome but is headless and scriptable. You script it with javascript, not python (as far as I know at least).
There are some example scripts to get you started. It's a lot like using Firebug. I've only spent a few min using it but I found I was quite productive right from the start.
From http://wwwsearch.sourceforge.net/mechanize/faq.html#general
If you come across this in a page you want to automate, you have four options. Here they are, roughly in order of simplicity.
Figure out what the JavaScript is doing and emulate it in your Python code: for example, by manually adding cookies to your CookieJar instance, calling methods on HTMLForms, calling urlopen, etc. See above re forms.
Use Java’s HtmlUnit or HttpUnit from Jython, since they know some JavaScript.
Instead of using mechanize, automate a browser instead. For example use MS Internet Explorer via its COM automation interfaces, using the Python for Windows extensions, aka pywin32, aka win32all (e.g. simple function, pamie; pywin32 chapter from the O’Reilly book) or ctypes (example). This kind of thing may also come in useful on Windows for cases where the automation API is lacking. For Firefox, there is PyXPCOM.
Get ambitious and automatically delegate the work to an appropriate interpreter (Mozilla’s JavaScript interpreter, for instance). This is what HtmlUnit and httpunit do. I did a spike along these lines some years ago, but I think it would (still) be quite a lot of work to do well.
Basically if you want something that deals with javascript then you need a real javascript engine, these invariably involve automating a real browser (I'm including headless ones in this).
Java’s HtmlUnit doesn't do a very good job as it doesn't use a javascript engine from an actual browser. Phantom JS sounds ideal (as newz2000 points out) however I find that when manipulating pages with javascript it can be very difficult to debug your script if you can't actually see the page you're dealing with.
This leads to solutions such as Selenium Webdriver which has a full python API to automate various browsers, however you must run a java jar and it actually launches the browser, so not the pure python solution you're after (but I think this is as close as you can get).
You can use Selenium with Python. You can then scrape JavaScript-generated content as well as manipulate the page with additional JavaScript (as well as Python).
# In your virtualenv: pip install selenium
from selenium import webdriver
# Launch Firefox GUI
browser = webdriver.Firefox()
# Alternatively, you can drive PhantomJS without a GUI
# With Node.js installed: `npm install -g phantomjs`
# browser = webdriver.PhantomJS()
# Fetch a webpage
browser.get('http://example.com')
# If you need the whole HTML document
# just like inspecting the rendered page with the console
html = browser.page_source
# Get an element, even if it was created with JS
button = browser.find_element_by_css_selector('div.some-class > \
input.the-submit-button')
# Click on something
button.click()
# Execute some JavaScript (assumes jQuery is loaded on the page)
browser.execute_script("$('html, body').animate({ scrollTop: 500 }, 50);")
You can run the code in a Python REPL and use autocomplete to discover the methods available on browser or whatever element you have selected. Or do something like print(dir(browser)) to see what is available.
An example how to use PyV8, to run JS on a DOM with python can be found here:
https://github.com/buffer/thug
This should be fairly easy to make it run together with mechanize.
I have a computer running Windows and connected with many devices, such as a printer, an RFID reader, etc.
I want all my applications run as HTML, CSS, and JavaScript, so I need to access these devices through JavaScript. But I don't know if it is possible to add custom JavaScript API to browser (such as Internet Explorer or Firefox), or what work are needed to make this possible.
You can use a browser's basic printing abilities to control a printer and may be able to find a keyboard wedge RFID reader, both of which would allow you to use just basic JavaScript code to operate them. However, you more than likely will need an ActiveX control (COM component) that you can instantiate from a browser script to be able to have any real control over the devices.
Unfortunately, you may have to write those controls yourself. You might be able to do it in C# with some judicious use of P/Invoke, however.
Another possibility is to write a browser plug-in that would act as a helper. I believe browser plug-ins, such as ones for Firefox, have more free reign access to your physical machine. It could accept commands from your website to be executed outside of the browser's JavaScript sandbox. Also I believe you can write browser plug-ins for Firefox in JavaScript.
I have a JavaScript application that an user interacts with. I need to save the appearance of the interface at the current time, crop out the part that I need (or only shot the part that I need by specifying the div), and send it back to the server.
Clearly any external services would not be able to do this, I need a JavaScript (or Flash) script that can save the screen appearance. Is this possible?
Also, as the comment below says, I cannot use ActiveX.
Google is doing this in Google+ and a talented developer reverse engineered it and produced http://html2canvas.hertzen.com/ . To work in IE you'll need a canvas support library such as http://excanvas.sourceforge.net/
I think using JavaScript, you won't be able to due to the security restrictions. Flash, possibly.
It's impossible in pure JavaScript, without using ActiveX.
It is impossible using JavaScript (nor Flash). It depends on your constraints, and there are some workarounds.
You can take advantage of browser extensions (such as a Firefox add-on), but I guess it does not fit your requierments.
The best option I can think of is to construct the DOM tree on the client side, and then post it to remote server.
On the server side nothing really holds you from doing generally anything. Using WebKit or even launching Internet Explorer or Firefox, you can create the snapshot server-side.
It's far from elegant, but possible.