I have a Python program that uses BeautifulSoup to extract some data from a website.
In Google Chrome, there is this option called the Developers Console; it is used to execute javascript commands live right on the webpage.
Is there a way, or a work-around-of-a-way to execute javascript commands remotely using Python? Like casting a webpage to an object and running a javascript command in the background (without launching a new Chrome window)?
You can use Native Messaging API to communicate between a shell script and the browser.
You can also launch Chrome or Chromium with --headless flag set. See also puppeteer.
Related
How to run javascript with node js without opening google chrome from different proxies.
Can i get a sample code.
eg project: https://github.com/huytd/agar.io-clone
eg2project: https://drive.google.com/drive/folders/19v9lxRrRqrMp6n3k5rbDRljGVdLYELQc?usp=sharing
I don't want to use the project i link to
briefly: I'm trying to find a code that goes to Facebook.com/nick and clicks the send message button
puppeteer: puppeteer is no-gui-on-gui chromium browser. and nodejs
I'm making a Native Messaging extension in firefox, I have some question:
Background-script (addded by background key) and content-script (added by content-script key). What's different between them and can I use both of them with 1 JS file? If not, I have to send page-script->content-script->background-script->native-app?
Which's the best function I need to use to send message from page-script to native app? I followed this example or tried with this function but it doesn't work.
Background-script can be considered as long term running, since Firefox started, to Firefox process quit. While content scripts runs in web page context, ends when web page closed. Content scripts can be used to access/modify web page, while background script can't. For detail, see documentation of background scripts, content scripts.
You need Native Messaging to send message from extension to native app. In extension, use chrome.runtime.connectNative to connect to native app. It will return a chrome.runtime.Port object. Then use Port.postMessage to communicate with native app.
Reference: Full Documentation of Firefox WebExtensions
Is it possible to run an external exe (under windows) using native messaging from a HTML application (Javascript) ?
Note:
I have a npapi plugin that runs an exe installed in the client machine. now npapi is deprecated, so I need another way to run my external exe from my website.
You are asking if there is a way for an html page to launch an executable on the client's machine? No, that is not possible. It'd be a huge security hole if it were.
What are you trying to accomplish?
Selemium uses browser to open the page and get content. But on my azure server(command line based) I could not have chrome or firefox. So what is the alternative to use python based selenium code on azure server.
I looked at http://phantomjs.org/ a headless browser. But I guess it is javascript so I would require to convert python code in JS.
Is there any other better alternative?
code snippet:
driver = webdriver.Chrome()
def getVideoTrend(self, item):
driver.get(item['link'])
element = WebDriverWait(driver, 20).until(lambda driver: driver.find_elements_by_class_name('yvp-main'))
self.yahoo_video_trend = []
for s in driver.find_elements_by_class_name('yvp-main'):
print "Processing link - ", item['link']
trend = item
trend['video_link'] = s.find_element_by_tag_name('video').get_attribute('src')
print s.find_element_by_tag_name('video').get_attribute('src')
self.yahoo_video_trend.append(trend)
Try using requests for your browsing needs and BeautifulSoup4 for parsing
So what is the alternative to use python based selenium code on azure
server.
May I know whether you are using Azure Web App? Per my understanding, Azure Web App does not allow us to install the custom software on the server. So in this case you may want to use a virtual machine instead. Windows based virtual machines have GUI pre-enabled, so you only need to use remote desktop to login to the server and install a browser (if you don’t want to use the default IE). For a Linux server, you can install a desktop and then install a browser as well. Please refer to http://blogs.technet.com/b/uktechnet/archive/2013/11/12/running-a-remote-desktop-on-a-windows-azure-linux-vm.aspx to see if this instruction is helpful.
I am developing a chrome extension for company internal use, what i want to do is open folder in explorer (design folder at internal server) when someone add the project id in a textbox and click the button,
ex:
'Y:\design\' + siteNumber
Can i do it using javascript?
Thanks!
Nalinda
No, javascript doesn't have access to the local machine's files/folders.
Sort of. You would need to write a stand-alone application in a native language (like C++ or Java) that could execute the opening of the folder, then have the Chrome extension communicate with it through the NPAPI Plugin. You could keep it all in Javascript by creating a Node.js program that executes the open /path/to/folder command (or your OS's equivalent). Either way, that native program would have to be pre-installed on the user's machine. But this doesn't seem like it should be a problem if it is an internal project.