I'm building a chrome extension and I want to run a python script which is in my PC on click of a button from an extension (which is basically HTML). The python script uses selenium web-driver to scrape data from a website and store it in another log file.
You basically use nativeMessaging. It allows you to create a communication bridge between your extension and an external process (such as python).
The way nativeMessaging works is by installing a host on your machine, and communicates to and from Chrome extension through stdin and stdout. For example:
Host in Python
This is how you write your nativeMessaging host in python, I have included the full example of this from the docs, but made it easier to understand with less code.
host.py
This is basically an echo server, respects stdin and stdout, makes sure it sends it as binary stream.
#!/usr/bin/env python
import struct
import sys
import os, msvcrt
# Set the I/O to O_BINARY to avoid modifications from input/output streams.
msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY)
msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)
# Helper function that sends a message to the webapp.
def send_message(message):
# Write message size.
sys.stdout.write(struct.pack('I', len(message)))
# Write the message itself.
sys.stdout.write(message)
sys.stdout.flush()
# Thread that reads messages from the webapp.
def read_thread_func():
message_number = 0
while 1:
# Read the message length (first 4 bytes).
text_length_bytes = sys.stdin.read(4)
if len(text_length_bytes) == 0:
sys.exit(0)
# Unpack message length as 4 byte integer.
text_length = struct.unpack('i', text_length_bytes)[0]
# Read the text (JSON object) of the message.
text = sys.stdin.read(text_length).decode('utf-8')
send_message('{"echo": %s}' % text)
def Main():
read_thread_func()
sys.exit(0)
if __name__ == '__main__':
Main()
host.json
This defines the communication python host, make sure the extension guid is the guid of your extension.
{
"name": "com.google.chrome.example.echo",
"description": "Chrome Native Messaging API Example Host",
"path": "host.bat",
"type": "stdio",
"allowed_origins": [
"chrome-extension://knldjmfmopnpolahpmmgbagdohdnhkik/"
]
}
host.bat
This runs the python executable.
#echo off
python "%~dp0/host.py" %*
install_host.bat
You run this once, to register your host in your OS.
REG ADD "HKCU\Software\Google\Chrome\NativeMessagingHosts\com.google.chrome.example.echo" /ve /t REG_SZ /d "%~dp0host.json" /f
Chrome Extension
manifest.json
Add the permissions for nativeMessing
{
"permissions": [
"nativeMessaging"
]
}
communication.js
In order to connect to the python host, you need to do the following:
const hostName = "com.google.chrome.example.echo";
let port = chrome.runtime.connectNative(hostName);
port.onMessage.addListener(onNativeMessage);
port.onDisconnect.addListener(onDisconnected);
To send a message to your python host, just send a json object to the port.
const message = {"text": "Hello World"};
if (port) {
port.postMessage(message);
}
To know the error when it disconnects:
function onDisconnected() {
port = null;
console.error(`Failed to connect: "${chrome.runtime.lastError.message}"`);
}
This full example is in the docs, I just renamed some stuff for clarity, available for Windows/Unix https://chromium.googlesource.com/chromium/src/+/master/chrome/common/extensions/docs/examples/api/nativeMessaging
Related
Here is a small websockets client and server POC.
It sends a single hard-coded message string from the (Python) server to the Javascript client page.
The question is, how to send further, ad-hoc messages? From the server to the client.
Tiny HTML client page with embedded Javascript:
<!DOCTYPE html>
<html lang="en">
<body> See console for messages </body>
<script>
# Create websocket
const socket = new WebSocket('ws://localhost:8000');
# Add listener to receive server messages
socket.addEventListener('open', function (event) {
socket.send('Connection Established');
});
# Add message to browser console
socket.addEventListener('message', function (event) {
console.log(event.data);
});
</script>
</html>
Here is the Python server code:
import asyncio
import websockets
import time
# Create handler for each connection
async def handler(websocket, path):
await websocket.send("message from websockets server")
# Start websocket server
start_server = websockets.serve(handler, "localhost", 8000)
# Start async code
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
This successfully sends a hard-coded message from server to client.
You can see the message in the browser console.
At this point the websocket is open.
The main application (not shown) now needs to send messages.
These will be dynamic messages, not hard-coded.
How can we send later, dynamic messages from the server?
After the code here runs?
I would like to put the socket into a global variable and call a send method but this is not possible because the server runs a continuous loop.
You can insert further messages into the Python server code like this:
import asyncio
import datetime
from typing import Iterator
import websockets
import random
websocket_connections = set()
sock_port = 8000
sock_url = 'localhost'
global_socket = lambda: None
async def register(websocket):
print('register event received')
websocket_connections.add(websocket) # Add this client's socket
global_socket = websocket
async def poll_log():
await asyncio.sleep(0.3) # Settle
while True:
await asyncio.sleep(0.3) # Slow things down
# Send a dynamic message to the client after random delay
r = random.randint(1, 10)
if (r == 5): # Only send 10% of the time
a_msg = "srv -> cli: " + str(random.randint(1,10000))
print("sending msg: " + a_msg)
websockets.broadcast(websocket_connections, a_msg) # Send to all connected clients
async def main():
sock_server = websockets.serve(register, sock_url, sock_port)
await asyncio.sleep(0.3) # Start up time
async with sock_server: await poll_log()
if __name__ == "__main__":
print("Websockets server starting up ...")
asyncio.run(main())
There is a very helpful example of a complete, full-duplex Websockets application here.
That example is part of the Websockets 10.4 documentation.
It's very helpful as a reference and to understand how websockets are used.
Introduction
I am learning from JS (after messing up with Java for a bit) and I stumbled upon sockets in javascript.
Problem
I've made a python server which is listening, then I loaded the JS extension to Chrome but the server does not get any message. Where I did wrong or what I am missing?
manifest.Json (maybe it's relevant as I couldn't do this .js as "background")
{
"manifest_version": 2,
"name": "Server-Test-Ext",
"version": "0.1",
"content_scripts": [
{
"matches": ["<all_urls>"],
"js": ["socket.js", "client.js"]
}]
}
client.js ( example from the Socket.io website)
var socket = new io.Socket();
socket.connect('https://localhost:8080');
socket.on('connect', function(){
// connected!
});
socket.on('message', function(msg){
// message coming
});
socket.send('Hello world!');
server.py
import socket
HOST = '127.0.0.1'
PORT = 8080
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((HOST, PORT))
s.listen()
conn, addr = s.accept()
with conn:
print('Connected by', addr)
while True:
data = conn.recv(1024)
if not data:
break
conn.sendall(data)
The server creates successfully the tcp (You can see that by typing on the CMD netstat -an))
Notes:
- I am using Visual Studio Code (cool text editor, still learning it)
- I am new to the subject (self-taught)
My expectation was to receive the message on the python server, which is running before the ext. Obviously.
Thanks for the interest in this question.
you need to declare socket permission to your manifest.js.
details:
https://developer.chrome.com/apps/manifest/sockets
I'm following the instructions in the documentation site, but I got stuck in the echo example, the websocket is created correctly and it's connected to the server but when I send anything to the server I'm not getting any response (In the example says I should see an alert window with the same message that I send into the socket but I don't, although I've changed the alert for a console.log but still), what I'm doing wrong?
In settings.py:
INSTALLED_APPS = {
...
'channels',
'myapp',
...
}
...
# Channels settings
CHANNEL_LAYERS = {
"default": {
"BACKEND": "asgiref.inmemory.ChannelLayer",
"ROUTING": "myapp.routing.channel_routing",
},
}
In routing.py:
from channels.routing import route
from myapp.consumers import *
channel_routing = [
route("websocket.receive", ws_receive),
]
In consumers.py:
def ws_receive(message):
# ASGI WebSocket packet-received and send-packet message types
# both have a "text" key for their textual data.
message.reply_channel.send({
"text": message.content['text'],
})
In asgi.py
import os
from channels.asgi import get_channel_layer
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")
channel_layer = get_channel_layer()
Then I run: python manage.py runserver, and in my browser I go to the server url and in the console I put the following:
socket = new WebSocket("ws://" + window.location.host + "/chat/");
socket.onmessage = function(e) {
alert(e.data);
}
socket.onopen = function() {
socket.send("hello world");
}
Again, at this point I should see an alert window (or the console.log message) but I get nothing.
The requests that I made have a status of pending (Although I read here and the first comment says it's normal)
And the server output looks like this:
Every time that I've tried to send something through the websocket in the browser, the server just print CONNECT but no log from the js console is showing.
Edit: I've tested websockets in my browser against echo.websocket.org and I got the answer as expected:
I changed to an older version of twisted and it fixed it. Hth
Ok, I'm writing a pyqt software to generate a webpage. Due to some security issues with Chrome and other things, I need a webserver to test the webpage.
So I thought to create a button called run, that you can click or press f5 to start a server, and open the browser to the page. The snippet of code that this button calls, simplified (there is some code to do things, including changing current directory and such), looks like this:
import sys
import webbrowser
from SimpleHTTPServer import SimpleHTTPRequestHandler as HandlerClass
from BaseHTTPServer import HTTPServer as ServerClass
Protocol = 'HTTP/1.0'
port = 8080
ip = '127.0.0.1'
new = 2 #goes to new tab
url = "http://"+ip+":{0}".format(port)
serverAddress = (ip,port)
HandlerClass.protocol = Protocol
httpd = ServerClass(serverAddress, HandlerClass)
sa = httpd.socket.getsockname()
webbrowser.open(url,new=new)
httpd.serve_forever()
Ok, the problem is as serve_forever is called, it can be expected to serve forever. Is there a way to kill the server after browser is closed?
Edit: I understand many people recommend using threads but I can't find a way to detect that the browser has closed or killing the thread in system monitor (I'm on Ubuntu) while testing.
Edit2: ok, I've read webbrowser.py, it doesn't seem to return any sort of process identifier...
Edit3: I'm reflecting on this, maybe the correct approach would be checking if someone is accessing the server, and if not, then kill it... This way I can detect if the tab was closed... Problem is the only way I can think uses a dummy page with this power that loads whatever page to test inside, which seems too hackish...
It seems if I can find a way of doing this, maybe through error responses...I can do a webserver in a subprocess that has a while and exits by itself like the one here: https://docs.python.org/2/library/basehttpserver.html#more-examples
import sys
#from threading import Thread
import webbrowser
import BaseHTTPServer
import SimpleHTTPServer
serverClass=BaseHTTPServer.HTTPServer
handlerClass=SimpleHTTPServer.SimpleHTTPRequestHandler
Protocol = "HTTP/1.0"
port = 8080
ip = '127.0.0.1'
new = 2 #2 goes to new tab, 0 same and 1 window.
url = "http://"+ip+":{0}".format(port)
handlerClass.protocol = Protocol
httpd = serverClass((ip,port), handlerClass)
sa = httpd.socket.getsockname()
print("\n---\nServing HTTP on {0}, port {1}\n---\n".format(sa[0],sa[1]) )
browserOk = webbrowser.open(url,new=new)
def runWhileTrue():
while True:
#print(vars(httpd))
httpd.handle_request()
runWhileTrue()
Right now I'm thinking about using a timer like a watchdog, if the server is not used more then a period, it get's killed... But I think this is an awful solution... I wanted the browser to ping for it for some time while the tab is opened...maybe, don't know if optimal, looking this code right now : SimpleHTTPServer and SocketServer .
Thinking maybe if the server could understand a message from the website it could break loop. The tab closure could be detected in javascript like here : Browser/tab close detection using javascript (or any other language). Don't know how to communicate this to the server.
EditFinal:
In the javascript code of the webpage, I've inserted:
window.addEventListener('unload', function (e) { e.preventDefault(); jsonLevelGet("http://127.0.0.1:8081/exit.json"); }, false);
Then, the python code is this server.py:
import sys
from threading import Thread
import webbrowser
import BaseHTTPServer
import SimpleHTTPServer
serverClass=BaseHTTPServer.HTTPServer
handlerClass=SimpleHTTPServer.SimpleHTTPRequestHandler
Protocol = "HTTP/1.0"
port = 8080
ip = '127.0.0.1'
admIp = ip
admPort = 8081
new = 2 #2 goes to new tab, 0 same and 1 window.
url = "http://"+ip+":{0}".format(port)
handlerClass.protocol = Protocol
httpdGame = serverClass((ip,port), handlerClass)
httpdAdm = serverClass((admIp,admPort), handlerClass)
sa = httpdGame.socket.getsockname()
sb = httpdAdm.socket.getsockname()
print("\n---\nServing HTTP on {0}, port {1}\n---\n".format(sa[0],sa[1]) )
print("\n---\nAdm HTTP listening on {0}, port {1}\n---\n".format(sb[0],sb[1]) )
browserOk = webbrowser.open(url,new=new)
def runGameServer():
httpdGame.serve_forever()
print("\nrunGameServer stopped\n")
httpdAdm.shutdown()
return
def runAdmServer():
httpdAdm.handle_request()
httpdGame.shutdown()
print("\nrunAdmServer stopped\n")
return
gameServerThread = Thread(target=runGameServer)
gameServerThread.daemon = True
admServerThread = Thread(target=runAdmServer)
gameServerThread.start()
admServerThread.start()
admServerThread.join()
It works! When the tab is closed, the server.py code exits! Thanks #st.never!
As you said, you could detect (in Javascript, in the browser) that the window is being closed, and send one last request to the server to shut it down.
If you don't want to inspect all the requests searching for the "poweroff request", you can instead have your server listen on two different ports (probably on different threads). For example, the "main" server listens on port 8080 with the current behaviour, and a separate instance listens on port 8081. Then you can simply shut down the server whenever any request reaches port 8081.
I'm trying to use a Socket connection to read a file on a remote website. So far, my code:
conn = new Socket;
if( conn.open( 'example.com:80' ) ) {
conn.write( 'GET /indesign-page/ HTTP/1.0' + "\n\n" );
reply = conn.read(999999);
conn.close();
} else {
alert( 'Problem connecting to server' );
}
The socket connects to example.com fine, but the request comes across as this:
GET http://localhost/indesign-page/ HTTP/1.0
when it should be this:
GET http://example.com/indesign-page/ HTTP/1.0
I've tried changing the conn.write parameters to 'GET http://example.com/indesign-page/ ...', but then it comes across as:
GET http://localhosthttp://example.com/indesign-page/ HTTP/1.0
The webserver requires that the host be set correctly to serve correctly.
You need to set the "Host" header.
conn.write( 'GET /indesign-page/ HTTP/1.0' + "Host: example.com\r\n" + "\n\n" );
Because conn.open( 'example.com:80' ) means find example.com's server ip then connect that ip address at 80 port, so the web server does not know that you had resolved example.com before connected to it.
Do you need to use a manual socket object? On Adobe's Community Site there's a mention to this already created FTP Script where you could call a GET or PUT to a file on a FTP server.
Otherwise which OS are you using? If you'll always be on a Mac, you could shell out to an AppleScript command and place the file anywhere you'd like:
var command = 'do shell script "curl http://localhost/indesign-page"';
var response = app.doScript(command, ScriptLanguage.APPLESCRIPT_LANGUAGE);
The nice thing about the AppleScript is that you can execute the command manually using the AppleScript Editor (or Script Editor if you're earlier than 10.6).