Develop Reference

How to get the contents of the body tag of the page through the browser extension?

Why is variable body empty? how to get body tag content?
I am writing code for google chrome extension. I plan to use the
extension for personal use only. For web scraping.
To finally analyze the text. I want to play with text.
background.js
chrome.tabs.onUpdated.addListener(function(tabId, changeInfo, tab) {
if(document.readyState == "complete"){
var url = changeInfo.url;
// I want to save the url to a file(url.txt)
var body = document.body.innerText;
// why is body empty?????
var pattern = /[A-Z].*?\./g;
var result = text.match(pattern);
result.forEach(myFunction);
function myFunction(item) {
text += item+"\n";
}
// I want to save the text to a file(collection.txt)
}
});
manifest.json
{
"name": "Parser",
"version": "1",
"manifest_version": 2,
"background": {
"scripts":["background.js"]
},
"permissions": [
"tabs",
"activeTab",
"storage",
"http://*/*",
"https://*/*"
]
}

Based on the manifest.json you posted it looks like you are running your code as a background script.
Background scripts don't have direct access to the loaded page content - this is why the body is empty in your code.
Instead, you will need to use a content script to access the page content and then message that data across to your background script for processing.
Here is an example setup using a background and content script that should let you retrieve and process the page content when a tab loads (not tested, but should point you in the right direction).
Thanks to the ResourceOverride extension which I used as a reference for writing the example below.
background.js
// background.js
chrome.runtime.onMessage.addListener(function(message, sender){
if (!message || typeof message !== 'object' || !sender.tab){
// Ignore messages that weren't sent by our content script.
return;
}
switch (message.action){
case 'receiveBodyText': {
processBodyText(sender.tab, message.bodyText);
break;
}
}
});
function processBodyText(tab, bodyText){
var url = tab.url;
// I want to save the url to a file(url.txt)
// TODO: Process your bodyText
var pattern = /[A-Z].*?\./g;
var result = text.match(pattern);
result.forEach(myFunction);
function myFunction(item) {
text += item+"\n";
}
// I want to save the text to a file(collection.txt)
}
content.js
// content.js
window.addEventListener('load', function(){
let bodyText = document.body.innerText;
chrome.runtime.sendMessage({
action: 'receiveBodyText',
bodyText: bodyText
});
});
manifest.json
// manifest.json
{
"name": "Parser",
"version": "1",
"manifest_version": 2,
"background": {
"scripts":["background.js"]
},
"content_scripts": [{
"matches" : [
"http://*/*",
"https://*/*"
],
"js": ["content.js"]
}],
"permissions": [
"tabs",
"activeTab",
"storage",
"http://*/*",
"https://*/*"
]
}
Information and docs
A note on WebExtension API differences between firefox and chrome:
Chrome uses the chrome namespace, Firefox uses the future-standard browser namespace.
So code written in Chrome would use: chrome.tabs.onUpdated(...)
and the equivalent in Firefox would be: browser.tabs.onUpdated(...)
Be aware of that when reading the docs and reading example extensions.
Background scripts
do not have access to the loaded page
have full access to the WebExtensions API
Content scripts
have full access to the loaded page
have only limited access to the WebExtensions API
Chrome content scripts docs
MDN content scripts docs
Docs on communication between content scripts and background scripts
WebExtensions API
Chrome WebExtensions API reference
MDN WebExtensions API reference
other useful links
MDN WebExtensions example Github repository
MDN "Anatomy of a WebExtension"
MDN detailed browser WebExtensions support tables
ResourceOverride extension - this is a fairly complex extension that uses both background and content scripts. I this used as a reference/example to better understand how extensions are written.

Porting WebExtension from Chrome to Firefox?

I made a working Chrome extension that is not packaged and is just a directory on my computer. I found out that I should be able to port this to Firefox rather easily.
I followed the "Porting a Google Chrome extension" guide on MDN and found that my manifest file is perfect.
I then followed the instructions on how to perform "Temporary Installation in Firefox" of the extension.
However, when I click on any file inside the directory, nothing happens. The extension doesn't load. Any advice? I know the extension works in Chrome fine and loads without error.
manifest.json:
{
"manifest_version": 2,
"name": "ER",
"description": "P",
"version": "1.0",
"browser_action": {
"default_icon": "icon.png",
"default_popup": "popup.html"
},
"background": {
"scripts": [ "background.js" ]
},
"content_scripts": [
{
"matches": [ "SiteIwant" ],
"js": [ "ChromeFormFill.js" ],
"run_at": "document_idle"
}
],
"permissions": [
"*://*/*",
"cookies",
"activeTab",
"tabs",
"https://ajax.googleapis.com/"
],
"externally_connectable": {
"matches": ["SiteIwant"]
}
}
ChromeFormFill.js:
// JavaScript source c
console.log("inside content");
console.log(chrome.runtime.id);
document.getElementById("ID").value = chrome.runtime.id.toString();
Background.js
chrome.runtime.onMessage.addListener(
function (request, sender, sendResponse) {
if (request.data === "info") {
console.log("Recieving Info");
return true;
}
});
chrome.tabs.create(
{
url: 'myUrl'
active: true
}, function (tab) {
chrome.tabs.executeScript(tab.id, { file: 'Run.js', runAt: "document_idle" });
});
Run.js will just alert('hi').
It just won't do anything when I try to load it on Firefox; nothing will happen.

Issues:
In manifest.json:
externally_connectable is not supported:1
Firefox does not support externally_connectable. You can follow bug 1319168 for more information. There is, currently, no expected time when this will be implemented.
You will need to communicate between the code on your site and the WebExtension using a different method. The way to do so is to inject a content script and communicate between the site's code and the content script. The common ways to do this are CustomEvent() or window.postMessage(). My preference is CustomEvent().
Using window.postMessage() is like yelling your message outside and hoping that either nobody else is listening, or that they know that they should ignore the message. Other people's code that is also using window.postMessage() must have been written to ignore your messages. You have to write your code to ignore any potential messages from other code. If either of those were not done, then your code or the other code can malfunction.
Using CustomEvent() is like talking directly to someone in a room. Other people could be listening, but they need to know about the room's existence in order to do so, and specifically choose to be listening to your conversation. Custom events are only received by code that is listening for the event type which you have specified, which could be any valid identifier you choose. This makes it much less likely that interference will happen by mistake. You can also choose to use multiple different event types to mean different things, or just use one event type and have a defined format for your messages that allows discriminating between any possible types of messages you need.
matches value needs to be valid (assumed to be intentionally redacted):
You have two lines (one with a trailing ,, one without; both syntactically correct):
"matches": ["SiteIwant"]
"SiteIwant" needs to be a valid match pattern. I'm assuming that this was changed away from something valid to obfuscate the site that you are working with. I used:
"matches": [ "*://*.mozilla.org/*" ]
In Background.js:
The lines:
url: 'myUrl'
active: true
need to be:
url: 'myUrl',
active: true
[Note the , after 'myUrl'.] In addition, myUrl needs to be a valid URL. I used:
url: 'http://www.mozilla.org/',
A Firefox 48 bug (now long fixed):
Your line:
chrome.tabs.executeScript(tab.id, { file: 'Run.js', runAt: "document_idle" });
In Firefox 48 this line is executed prior to the tab being available. This is a bug. It is fixed in Firefox Developer Edition and Nightly. You will need one of those to test/continue development.
Issues in ChromeFormFill.js:
Another Firefox 48 bug (now long fixed):
chrome.runtime.id is undefined. This is fixed in Developer Edition and Nightly.
Potential content script issue:
I'm going to assume your HTML has an element with an ID = 'ID'. If not, your document.getElementById("ID") will be null. You don't check to see if the returned value is valid.
Running your example code
Once all those errors were resolved, and running under Firefox Nightly, or Developer Edition, it worked fine. However, you didn't have anything that relied on being externally_connectable, which won't function.
agaggi noticed that I had forgotten to include this issue in the original version of my answer.

Develop a chrome extension to count <input> tags of currently accessed webpage

Hey I'm new to javascript and chrome extension developing. I'm trying to develop a chrome extension which use browser action to count the number of tags in currently active tab of the Google Chrome browser.I can use getElementsByTagName.lenght method to calculate the number of tags and I know that I can use console API to access the DOM of a webpage. But I have no idea how to call that API from my javascript file.Do you guys know anything regarding this ?

To get access to the current page DOM you need to write a content script.
1. Specify the content script in manifest.json
"content_scripts": [
{
"matches": ["http://www.google.com/*"],
"css": ["mystyles.css"],
"js": ["jquery.js", "myscript.js"]
}
]
If you need to inject the script sometimes use Programmatic Injection by specifying permissions field:
{
"name": "My extension",
...
"permissions": [
"activeTab"
],
...
}
2.I would prefer the latter in this case.In popup.js add the code:
function printResult(result){
//or You can print the result using innerHTML
console.log(result);
}
chrome.tabs.executeScript(null, { file: 'content.js' },function(result){
printResult(result);
});
3.In content script u have access to current page/active tab DOM.
var result=document.getElementsByTagName("input").length;

Chrome extension: load and execute external script

I have trouble loading and executing external js-script into my chrome extension. Looks the same as this question, but I still cant't figure out why it doesn't work in my case.
The idea is that I want to have in my content script some default function which should parse a web-page content. And for some specific web-pages I want to load and use specific parsers, so I try to load proper js-script for a wep-page, and this script shoud extend functionality of default parser.
By now I try only execute code from external script, but have such error: Unchecked runtime.lastError while running tabs.executeScript: No source code or file specified at Object.callback
This is my manifest.json:
{
"name": "Extension name",
"version": "1.2",
"description": "My chrome extension",
"browser_action": {
"default_popup": "popup.html",
},
"content_scripts": [{
"css": [
"style.css"
],
"js": [
"bower_components/jquery/dist/jquery.js",
"bower_components/bootstrap/dist/js/bootstrap.js",
"content.js"
],
"matches": ["*://*/*"]
}],
"web_accessible_resources": [
"frame.html",
"logo-48.png"
],
"icons": {
"16": "logo-16.png",
"48": "logo-48.png",
"128": "logo-128.png"
},
"permissions": [
"tabs",
"storage",
"http://*/",
"https://*/"
],
"manifest_version": 2
}
This is popup.html
<!doctype html>
<html>
<head>
<title>Title</title>
<script src="popup.js"></script>
</head>
<body>
<ul>
<li>Some link</li>
</ul>
</body>
</html>
And in popup.js i execute scrip like this:
chrome.tabs.query({active: true, currentWindow: true}, function(tabs) {
chrome.tabs.executeScript(tabs[0].id, {file: "http://127.0.0.1:8000/static/plugin/somesite.js"});
});
What am I dong wrong, did I miss something? Or should I use another approach to solve the issue?

Running scripts from external sources like you try is forbidden by google chrome and will block or even not publish your extension. All scripts must be in the extension. But there is a solution,
from google chrome doc:
The restriction against resources loaded over HTTP applies only to
those resources which are directly executed. You're still free, for
example, to make XMLHTTPRequest connections to any origin you like;
the default policy doesn't restrict connect-src or any of the other
CSP directives in any way.
If you need badly an external source, you can do a XML HTTP request and use the eval to the content. Here is a part of code from google doc:
var xhr = new XMLHttpRequest();
xhr.open("GET", "http://127.0.0.1:8000/static/plugin/somesite.js", true);
xhr.onreadystatechange = function() {
if (xhr.readyState == 4) {
// WARNING! Might be evaluating an evil script!
var resp = eval("(" + xhr.responseText + ")");
// Or this if it's work
chrome.tabs.executeScript(tabs[0].id, {code: xhr.responseText});
}
}
xhr.send();
or you can use some library, $.get() with jquery or $http with angularjs.
if you add eval in your code you must add in manifest.json this:
"content_security_policy": "script-src 'self' 'unsafe-eval'; object-src 'self'"`,

As per the discussion here: https://groups.google.com/a/chromium.org/forum/#!topic/chromium-extensions/LIH7LGXeQHo,
Running scripts from external sources may cause your extension to be unpublished or blocked.
Just providing another approach, you could make an ajax call to the content script then call chrome.tabs.executeScript
chrome.tabs.query({active: true, currentWindow: true}, function(tabs) {
$.get("http://127.0.0.1:8000/static/plugin/somesite.js", function(result) {
chrome.tabs.executeScript(tabs[0].id, {code: result});
}, "text");
});

The 'file' option for executeScript only relates to files embedded in your extension. I know, the documentation isn't clear on that and while it might work with URLs, it sounds like a hack, not a feature. In order to load scripts from external sources into your active page, you usually have to execute a script that creates a script tag inside the DOM of the loaded document there.
I feel I have answered parts of this question before here: Why is chrome.tabs.executeScript() necessary to change the current website DOM and how can I use jQuery to achieve the same effect?
To summarize:
1) in order to have access to web pages from the extension, you need to add permissions for it:
"permissions" : [
"tabs",
[...]
"http://*/*",
"https://*/*" ],
2) you need to execute some sort of code that creates the DOM script element that loads what you need:
chrome.tabs.executeScript(tab.id, {
code : 'var script=document.createElement(\'script\');' +
'script.onload=function() { /*do something in the page after the script was loaded*/ };' +
'script.src=\'http://127.0.0.1:8000/static/plugin/somesite.js\';' +
'document.body.appendChild(script);'
}, function (returnedValue) {
// do something in the extension context after the code was executed
});
Take a look at the remex function in the link above, which I think solves a lot of the ugliness of javascript written as a string as here.

Reloading a Chrome extension content script execution environment

I’d like to mimic the effect of window.location.reload(), but only for the “isolated world” which my content script is running in. That is, remove all existing JS, particularly callbacks and event bindings. Is there a nice way to do this?
Note: chrome.runtime.reload() doesn’t work for this; it has the effect of reloading the extension and the background script, but it does not reload existing content scripts until the user refreshes.

As far as I can tell, there's no automatic way to re-inject content scripts, for example during an extension update. What you can do is to find all tabs whose url matches the pattern you need, and programmatically re-inject the content scripts using chrome.tabs.executeScript.
Note that this method requires to add a permission for the same URL pattern as the one used by your content script.
Manifest.json:
"content_scripts":
[
{
"matches": [ "http://*.google.com/*" ],
"js": [ "content_script.js" ]
}
],
"permissions":
[
"tabs", "http://*.google.com/*"
]
Background.js:
chrome.runtime.reload();
chrome.tabs.query({ url: "http://*.google.com/*" }, function(tabs)
{
for(var i = 0; i < tabs.length; i++)
{
chrome.tabs.executeScript(tabs[i].id, { file: "content_script.js" }, function() {});
}
});