Develop Reference

python : disable download popup when using firefox with selenium

I have script that using selenium and firefox to automating download action.
The problem is whenever I run script I always get pop up from firefox keep asking what kinds of action I would like to do, even though I set download path in firefox preference. I checked files and folders to create master mimeTypes.rdf for all users, but I couldn't find mine.(I'm using ubuntu). I found ~/.mozilla/firefox but there was no file for directory of my profile name nor any file has an extension like .rdf
here is the criminal's pic that making me crazy
firefox download popup
below is what I've done to disable the popup.
profile = FirefoxProfile()
profile.set_preference("browser.download.panel.shown", False)
profile.set_preference("browser.helperApps.neverAsk.openFile", 'application/zip')
profile.set_preference("browser.helperApps.neverAsk.saveToDisk", 'application/zip')
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.download.dir", "/home/i-06/Downloads")
driver = webdriver.Firefox(firefox_profile=profile)

I have spent many hours trying to suppress that "save or open" pop-up that appears when downloading a file using the firefox driver with selenium (python 3.x). None of the many suggestions involving various values for profile.set_preference worked for me. Maybe I missed something.
Still, I finally got it working by the other method that is recommended : using an existing firefox profile.
You can tweak your default (or custom) profile to the file save behaviour you want. Type the following in the firefox address bar and make changes here :
about:preferences#applications
Then the only setting up you need to do to download the file into your current working directory is :
from selenium import webdriver
fp = webdriver.FirefoxProfile(<your firefox profile directory>)
fp.set_preference("browser.download.folderList",2)
fp.set_preference("browser.download.dir", os.getcwd())
driver = webdriver.Firefox(firefox_profile=fp)
If you have a typical ubuntu setup, you can find your default firefox profile dir by viewing ~/.mozilla/firefox/profile.ini
In that .ini file, look for Path under [Profile0]

I doubt you need to define both. Remove the below line from your code
profile.set_preference("browser.helperApps.neverAsk.openFile", 'application/zip')
Also sometime the MIME type of zip file can be different based on the server. It could be any of below
application/octet-stream
multipart/x-zip
application/zip
application/zip-compressed
application/x-zip-compressed
So in Network tab check what is the content type you are getting and add that to your profile to make sure the dialog doesn't come

I removed profile.set_preference("browser.helperApps.neverAsk.openFile", 'application/zip') as Tarun Lalwani suggest and it still work. But my problem was that I put application/mp4 instead of video/mp4. You could check MIME type here.

Using selenium to download a file via window.open

I'm trying to scrape a webpage where clicking a link results in a new window popping open that immediately downloads a csv. I haven't been able to figure out the format of the url since it's fairly dense javascript (and one function is called via the onClick property while another is called as part of the href property. I have not worked with Selenium before, so I was hoping to confirm before getting started that what I want to do is possible. I had read somewhere that downloading files via new popup windows is not necessarily something I can do with Selenium.
Any advice would be greatly appreciated. A this is possible would be very helpful as would as here's how you'd do it even sketched in broad detail. Thanks much!
To be clear, my difficulties primarily stem from the fact that I can't figure out how the URL to download the file is generated. Even looking at the Google chrome network calls, I am not seeing where it is, and it would probably take me many hours to track this down, so I am looking for a solution that relies on clicking specific text in the browser rather than disentangling the cumbersome machinery behind the scenes.

Here's how I download files using Firefox webdriver. It's essentially creating a browser profile so that the default download location for certain file types are set. You can then verify if the file exists at that location.
import os
from selenium import webdriver
browser_profile = webdriver.FirefoxProfile()
# add the file_formats to download
file_formats = ','.join(["text/plain",
"application/pdf",
"application/x-pdf",
"application/force-download"])
preferences = {
"browser.download.folderList": 2,
"browser.download.manager.showWhenStarting": False,
"browser.download.dir": os.getcwd(), # will download to current directory
"browser.download.alertOnEXEOpen": False,
"browser.helperApps.neverAsk.saveToDisk": file_formats,
"browser.download.manager.focusWhenStarting": False,
"browser.helperApps.alwaysAsk.force": False,
"browser.download.manager.showAlertOnComplete": False,
"browser.download.manager.useWindow": False,
"services.sync.prefs.sync.browser.download.manager.showWhenStarting": False,
"pdfjs.disabled": True
}
for pref, val in preferences.items():
browser_profile.set_preference(pref, val)
browser_binary = webdriver.firefox.firefox_binary.FirefoxBinary()
browser = webdriver.Firefox(firefox_binary=browser_binary,
firefox_profile=browser_profile)
# set the file name that will be saved as when you download is complete
file_name = 'ABC.txt'
# goto the link to download the file from it will be automatically
# downloaded to the current directory
file_url = 'http://yourfiledownloadurl.com'
browser.get(file_url)
# verify if the expected file name exists in the current directory
path = os.path.join(os.getcwd(), file_name)
assert os.path.isfile(path)

How to write file in Google Chrome App without prompting?

I am fumbling around with the free Chrome Dev Editor on my Chromebook. I am trying to use the fileSystem to read and write .txt files. It is all very wrapped up, not at all like in C. I can no more tell if I am even allowed to do something, let alone where the proper place is to find out how.
I think the files I can see using the Files thingy are in the sandbox that I am allowed to play in (meaning, folders that are accessible by the app?) The root is called Downloads. Sure enough, if I use all the dot calls and callback arguments for the read, as in the examples at developer.chrome.com/apps/filesystem, it works. But I have to have a prompt
every time for both reads and writes.
A little more Googling came up with this trick: (I think it was here in stackoverflow, in fact) a chrome.runtime call, getPackagedDirectoryEntry, that seems to give me a handle to the folder of my app. Great! That's all I need to not have to go through the prompting. For the readfile, anyway.
But then trying to apply the same trick to the writefile did not work. In fact, it did nothing discernible. No errors, no complaints. Nothing. Even though the write file with prompting works fine (so presumably I have the permissions and Blob construction right.) What to do?
Here is my code:
function test(){
// Samsung 303C Chromebook - Chrome Dev Editor - /Downloads/Daily/main.js
// prompted write
chrome.fileSystem.chooseEntry({type:'saveFile'},function(a){
a.createWriter(function(b){
b.write(new Blob(["Programming fun"],{type:'text/plain'}));
},function(e){trace.innerText = 'error is ' + e;});
});
// unprompted read
chrome.runtime.getPackageDirectoryEntry(function(a){
a.getFile('text.txt',{},function(b){
b.file(function(c){
var d = new FileReader();
d.onloadend = function(){trace.innerText = this.result;};
d.readAsText(c);
});
});
});
// unprompted write - why not?
chrome.runtime.getPackageDirectoryEntry(function(a){
a.getFile('new.txt',{create:true},function(b){
b.createWriter(function(c){
c.write(new Blob(["Miss Manners fan"],{type:'text/plain'}));
},function(e){trace.innerText = 'error is ' + e;});
});
});
}

To be fair, Filesystem API is a big mess of callbacks and it's not unreasonable to get drowned in it.
It's not currently documented, but chrome.runtime.getPackageDirectoryEntry returns a read-only DirectoryEntry, and there is no way to make it writable (it's specifically blacklisted).
You probably don't see an error, because it fails at the getFile stage, for which you don't have an error handler.
Unfortunately, for a Chrome App the only option to write out to a real filesystem is to prompt the user. However, you can retain the entry and ask only once.
If you don't need to write out to the real filesystem but need only internal storage, HTML Filesystem API can help you (yes, it's marked as abandoned, but Chrome maintains it since chrome.fileSystem is built on it).
Extensions additionally have access to chrome.downloads API that enables writing to (but not reading) the Downloads folder.
P.S. What you see in Files app is your "real" local filesystem in ChromeOS + mounted cloud filesystems (e.g. Google Drive)

You can use the basic web Filesystem API. First, add the "unlimitedStorage" permission. Then, copy the packaged files to the sandboxed filesystem, like this:
chrome.runtime.getPackageDirectoryEntry(function(package) {
package.getMetadata(function(metadata) {
webkitRequestFileSystem(PERSISTENT, metadata.size, function(filesystem) {
package.copyTo(filesystem.root)
})
})
})

How run a windows console for JavaScript

I would like to have a console window (a command line) on Windows 7 which will allow me to play with JavaScript just like a python console.
Update:
It's important to have a file access from within the console (or script run through it).

You can use Node.js's REPL. To do so follow this steps:
Download and Install Node.js.
Call Node.js from the Start Menu / Start Screen or directly node.exe installation path (e.g C:\Program Files\nodejs\node.exe).
Enjoy!
You may want to add the installation path to your PATH enviroment variable for ease of use.
Note: to leave node.js press Ctrl + C twice.
To access the local files, you will need the File System module. This is an example of usage:
var fs = require("fs");
fs.readFile(
"C:\\test.txt",
function(err, data)
{
if (!err)
console.log(data.toString());
}
);
This will output the contents of the file C:\test.txt to the console.
Note: An unhandled exception will cause node.js to "crash".

You can just use the developer tools.
For example, in Chrome, press F12. This will bring up the developer tools. The last option on the menubar is console. This will allow you to create JS variables and functions and to interact with DOM elements on the current page

It's possible thanks to Mozilla Rhino JavaScript Engine.
To create a console window for JS:
1) Download Mozilla Rhino JavaScript Engine binary.
2) Extract: js.jar.
3) Create a script to run the console window (e.g. rihno_console.bat):
java -cp js.jar org.mozilla.javascript.tools.shell.Main
For more information about usage (for instance, and global functions inside this console) visit the Rhino Shell web page.

Just like I informed another user with the same question as yours who was faced with the same need, check out DeskJS (https://deskjs.wordpress.com). It's a portable Windows console application that lets you run pure JavaScript code and even load any existing JS files. It supports even the basic JS popup boxes implemented in browsers. You can save your commands as JS files that can be run on startup or by dragging-and-dropping them on the app. Plus there's so much more to it like you can create a build system for Sublime Text that can run JS files via cmd, it supports themes for customizing the entire console and snippets which let you save short snippets of JavaScript code for later use. Improvements are still being made on the app together with other native APIs being included. Hope this helps you as it did for me.

Debug Javascript in netbeans 7.4 inside PHP project

I want to debug javascript code inside my php project in netbeans. I have read on several occasions that this should be possible in the new netbeans 7.4 version, for example here and here, but i cannot get it to work. I have installed de debugger connector for chrome and php debugging works just fine but when i try to set a breakpoint in a .js file it says:
unresolved breakpoint,
debugger is not attached to tab with id....
i understand that the link from the netbeans page is for an html 5 application but i thought this debugging would also be enabled in php projects. Am i doing something wrong?
i know i can debug with firebug or chrome itself but i would like to do it all in one place in my netbeans IDE...
thanks in advance

The unresolved breakpoint usually mean that for instance you set it in file that is not loaded in Chrome's tab right now (or for some reason, IDE cannot match URL of JS file and local JS file). The mixed debugging works only in Embdded Browser or in Chrome with NetBeans connector (you can see the usually yellow bar in your page saying "NetBeans connector is debugging this page" and you can debug PHP and JS at the same time.
Have a look here, although it is about Java EE projects, it is very similar to PHP projects
Updated answer:
One issue I remember (and plain Chrome Dev Tools has it as well) is that if you have JavaScript file attached to HTML/PHP with dynamic parameter to prevent browser from caching, e.g. , where "673612" changes each time a file is loaded. If that's your case, try to remove this dynamic attribute. I think that e.g. Sencha or ExtJS use this feature which "breaks" debuggers.

I had a similar problem : javascript breakpoints were broken, while everything else was working fine (for example php breakpoints were okay).
The reason was that in the run configuration properties I changed the Project URL to something that was not the host anymore, but a subfolder managed by a url rewriting rule.

johanvs is correct, but my reputation is not enough to +1.
Suppose a NetBeans project contains many files in different folders:
/var/www/index.html
/var/www/config.html
/usr/doc/readme.txt
/usr/doc/license.txt
Since "index.html" is not in the project root folder but under "/var/www", NetBeans does not know "http://127.0.0.1/index.html" is corresponding to "/var/www/index.html". To solve, verify below settings in NetBeans -> File -> Project Properties:
Sources -> Web Root
"var/www"
Run Configuration -> Project URL
"http://127.0.0.1/"
Run Configuration -> Index File -> Browse
"index.html"
Run Configuration -> Remote Connection -> Manage -> Initial Directory
"/"
Run Configuration -> Upload Directory
(empty)