Getting url and content on EVERY page from Chrome extension - javascript

First of all, all the answers will be much appriciated, as I'm running out of my own ideas.
I have a "simple" problem to tackle. I wanna do a chrome extension, which would basically track the websites you're visiting (with content included). To clarify, we have full consent of the users to do this. :-)
What I did so far, isn't working well (I've tried to use many of the hooks provided by chrome API, ie chrome.webNavigation.onCommitted, chrome.webRequest.onBeforeRequest, chrome.webNavigation.onCompleted, chrome.webRequest.onBeforeRedirect, chrome.history.onVisited).
The best option so far is to use chrome.history.onVisited, but then again, I've no idea how to get website content from it - I've tried to execute content script which returns document.documentElement.outerHTML, but in order to do so I need to know tabId, which isn't available there.
Basically, something is messed up in chrome API or my thinking - probably the latter :)
I want to point out, that it almost works, but not on all cases. It's hard for me to debug which cases fail though.
Thanks a lot for all the tips I can get!

Based on your clarification in a comment I could suggest you to use content scripts and message passing.
At the moment when the user press "start recording" button you can - from the background page - execute content script which will intersect content of the pages already opened and then will send it back to the background page for further processing (save possibly). You can query for all tabs in any window using chrome.tabs.query function:
chrome.tabs.query({}, function(tabs){
console.log(tabs); //tabs array contain all tabs in all windows.
});
Iterating over this array you can execute script in all tabs using it's tab.id:
tabs.forEach(function(tab){
chrome.tabs.executeScript(tab.id, { 'file': 'cs.js' })
});
Then you can attach event handler to listen up for changes in tabs:
chrome.tabs.onUpdated.addListener(function(change){
if(change.status === 'complete'){
//inject content script
}
});
Second scenario is to add content script to every page user visits adding proper manifest entry:
...
"content_scripts": [
{
"matches": ["<all_urls>"],
"js": ["cs.js"]
}
],
...
After cs.js load you can call background page and ask it if the extension should capture page content (using messaging passing). If yes - do the same as above, if not, just do nothing.
It is the easiest and most convenient way of doing what you need.

Related

Can you only use either a background page or a background script in a chrome extension?

In my chrome extension I want to have a background script for obvious reasons. However I now also want to have a background page in which I like right here discribed load some html (Chrome extension: loading a hidden page (without iframe)) using an iFrame which I can interact with using a content script. But when I'm trying to load both the background script and the background page like so:
...
"background":{
"scripts": ["background_script.js"]
},
"background": {
"page": "iFrameBackground.html",
"persistent": true
},
...
and then try to send a message from my content script to the background script I get this error:
Unchecked runtime.lastError: Could not establish connection. Receiving end does not exist.
So either I'm missing something entirely here or are you really just able to use one of the two?
Thank you :)
Problems
Only one declaration method can be used.
Your json is invalid as it has two identical keys so Chrome won't even install it.
Solution
Choose just one method. Any method.
You can access DOM in your background script/page regardless of the way it's declared.
The result of each method is identical: they both create a background page. An extension can have just one background page where any amount of background scripts are loaded - just like in any other web page. The scripts method exists just for convenience: it simply generates a dummy HTML page as you can see in devtools inspector for the background page.

How to create a Chrome extension that adds extra information to a specific site?

I play card games on a board game platform called BoardGameArena.
Now I'd like to have a card tracker-helper that would inform me about the remaining cards for one of the games called "Papayoo". The entire log is available while playing the game so should be a pretty straightforward Chrome extension, right?
All of the relevant action happens on the right side of the page:
<div id="right-side-second-part">
<div id='logs_wrap'>
<div id='logs'>
<div id='seemorelogs'><a id='seemorelogs_btn'href='#' onclick='return false'></a></div>
</div>
</div>
</div>
It's a bit weird that I see an empty template like above if I do "View Page Source" of "Save Page As", but the <div id='logs_wrap'> is being populated during the game, and I can see the entries appear using the "Inspect" functionality. But maybe that's expected for the fields that are being populated dynamically (I'm not a web developer).
Anyway, here is my hello world attempt for a Chrome extension:
manifest.json
{
"name":"Papayoo Helper",
"description":"Papayoo game helper for the BGA platform",
"version":"1",
"manifest_version":2,
"content_scripts": [
{
"matches": ["http://boardgamearena.com/*/papayoo*"],
"js": ["tracker.js"]
}
]
}
an example URL of the game is https://boardgamearena.com/7/papayoo?table=122356466 so I assume it should be matched by my rule.
tracker.js
document.getElementById("right-side-second-part").innerHtml = "<div id='tracker'>Hello world!</div>" + this.innerHtml;
and it should prepend my "Hello world!" to the existing element.
I load the extension by visiting chrome://extensions/ and selecting "Load unpacked".
It doesn't work as nothing is displayed, which leads to the following questions:
How to debug the behavior and see why it doesn't get loaded?
How to adjust it to update automatically (or at least frequently)?
I guess the logic will be simply to iterate elements within <div id='logs'> once the extension itself is working.
Thanks!
matches
It should be https not http
This is a Single Page Application site which doesn't reload pages from server when you navigate so you need to run the content script on the entire site and then watch for the changed URL path explicitly in your code.
It should also include the subdomains like en so we'll add *. to match both the main site and its subdomains.
To detect the change in URL see this answer
Result:
"matches": ["https://*.boardgamearena.com/*"]
innerHtml
innerHtml should be innerHTML
Don't assign innerHTML of another site using = or += as it removes all the event listeners attached from JavaScript thus breaking its functionality. Instead use insertAdjacentHTML or explicitly build DOM via document.createElement or some js library.
To debug, use the built-in devtools, there are many tutorials. First make sure the content scripts are listed in devtools UI (in this case they won't be for reasons explained above) then set a breakpoint in the code somewhere and reload the tab, see what happens when it triggers.
To repeatedly do something you can use timers e.g. setInterval and setTimeout. Another approach is to use MutationObserver to react to site's changes.

Open options page from content-script on button click?

I am trying to open my web-extension options page from an injected button via a content-script onto a page. Here is the setup:
manifest setting:
"options_ui": {
"page": "options/options.html",
"open_in_tab":true
},
"web_accessible_resources": ["icons/icon.png", "icons/icon64.png","options/options.html"]
content-script.js:
Settings
What am I missing here? Also, I know moz-extension: might not be the best option for cross-browser operation but not sure what should be the correct namesapce?
EDIT:
I am using a fixed id in manifest as :
"applications": {
"gecko": {
"id": "{adacfr40-acra-e2e1-8ccb-e01fd0e08bde}"
}
},
Your content script, as shown, is actually a chunk of HTML and not JavaScript, as expected. So it wouldn't work. Perhaps that's not your actual code, just what you create?
But suppose you do adjust the content script to add a button and have a listener attached (out of scope for this question, I think). How to open the options page?
The canonical way is to call browser.runtime.openOptionsPage(), but that API can't be called from a content script.
Two options:
Stick with openOptionsPage(). In that case, you need a background (event) page that listens to Messaging, and then signal from the content script that you want the options page open.
The advantage of this approach is that you don't need to make the options page web-accessible.
If you insist on directly opening the page from the content script / a tag, you can, but you'll need to get the dynamically-allocated UUID for your extension instance.
The URL for your options page is not fixed, but should be obtained with browser.runtime.getURL("options.html"), and that URL should be used in the link creation.
This method requires declaring it as web-accessible.

How to show insecure content in iframe

In short , I'm developing a google chrome extension , when I add any url starting with http:// to source attribute to an iframe, I get a message like :
[blocked] The page at 'https://www.facebook.com/' was loaded over
HTTPS, but ran insecure content from 'http://youtu.be/m0QxDjRdIq4':
this content should also be loaded over HTTPS.
and I don't see the content in the iframe !
so how can I overcome this ?
what I want to achieve is that : I hide facebook adds , and in its place I added an iframe instead, I detect when the mouse is hovering over a link contained in a post, then I want to show the link's content in an iframe.
What are my possible alternatives? I don't need to enable showing insecure content in chrome because it is a chrome extension that I will publish!
It seems that the security limit is strict, so we need a way to work around that.
What if you could load the page using other means than an <iframe> and insert it into the page afterwards? There are multiple ways to do that, ranging from more practical to less realistic.
You can use the Chrome captureVisibleTab API to generate a screenshot of a website as an image, exactly what you need. It sounds like you need a visible tab to use this API, but you can actually specify any Chrome window as a target and you can create Chrome windows unfocused and hidden behind the edge of the screen.
If captureVisibleTab provides trouble in step 2, there is also pageCapture API to get an entire page as a single content object.
You can also use a server to create screenshots. Serve a simple application over HTTPS that uses PhantomJS to create a screenshot. An advantage of this approach is your server is likely to be much faster at screenshot generation. The disadvantage is you need to pay for the server.
You could also use xhr in your extension background process (which is not limited by the security limitation) to get the HTML. This wouldn't get any resources, but that could be a beneficial thing if you want a very quick if inaccurate screenshot. Just load HTML, parse and detect links to stylesheets, download them and inject those stylesheets into the HTML as <style> tags.
The resulting HTML can be injected to the <iframe> manually. You could even inject scripts and images this way, but that would be harder and less useful, since you need a quick screenshot of how the page looks like.
I think using built-in Chrome functionality for screenshots is the best bet, if only you can make the user experience good enough.
First and stupid way: change http in link on https. But youtube and I think many other sites don't allow to show their content in iframes. try it and you get Refused to display 'link' in a frame because it set 'X-Frame-Options' to 'SAMEORIGIN'.
Second and at least stupid way: remove protocol from link, like //youtu.be/m0QxDjRdIq4 and you get protocol, that on this page. But a situation similar to the previous.
Third way for youtube only: you can generate iframe with src like //www.youtube.com/embed/m0QxDjRdIq4 and user can see the video.
Fourth way, not for all sites: use site API's - not a best solution, but like a option.
Fifth way, but impossible (I think): try to get page's content with javascript and regenerate it in way, that you need.
Sixth way, needs powerfull server: create an service on your server, which will download pages and resend it to users. One problem - linear dependence server's power of requests.
Seventh way, I forgot that it's extension: you can open link in another tab/window, get it content, close tab/window and show content in tab that you need.
Eigth way, the best, I think: use YAHOO yql like this:
$.getJSON("https://query.yahooapis.com/v1/public/yql?q=select"
+"* from html where url='youtube.com/watch?v=m0QxDjRdIq4'"
+"&format=json&diagnostics=true&callback=?"
, function (data, textStatus, jqxhr) {
// process data
}
}
Demo on jsFiddle

Searching and highlighting text on current page for a Chrome Extension

How should I connect my pages to search and highlight text on current tab?
Currently I have:
manifest.json does defining content/backgr/event page do significant things,auto inject code etc?
popup.html essentially a shell for the search input which is used by search.js
search.js should this be in background/event/content/popup.html page?
What I still don't understand after reading:
What is a content page vs. background/event page?
I know one is constantly running vs injected, but that's as much as I got from the chrome extension manual, I still don't quite understand if the content script/page is seperate from the popup.html for example and what the difference between a script in the popup.html vs content page/script is.
What I know:
I know how to search for text on a page, and replace it or change its style etc. using JS.
I need to read up on the messaging API for Chrome Extensions.
I know I need to know how to use the messaging API, is it going to be required for page search and highlighting?
Summary:
I don't need a walk through or full answer, just a little help visualizing how Chrome extensions work, or at minimum how I should set mine up in relation to page interaction IE:
search.js content page injected >>>>> popup.html
and maybe a short bit about how injection works in chrome extensions(IE, do I only need to specify that it is content page in manifest.json to have it injected or is there more work to it)/expected behavior?
Apologies for the jumbled thoughts/question/possibly missing the things relevant to my questions while reading the manual.
I will start with making the purpose of each kind of page/script more clear.
First is the background page/script. The background script is where your extension lives. It isn't required, but in order to do most extension things, you need one. In it you can set up various event listeners and such depending on what you want it to do. It lives in its own little world and can only interact with other pages and scripts using the chrome.* apis. If you set it up as an event page it works exactly the same except that it unloads when not in use and loads back into memory when it has something to do.
Content scripts refer to injected Javascript and/or css. They are the primary tool used for interacting with web pages. They have very limited access to chrome.* apis, but they have full access to the DOM of the page they are injected into. We will come back to using them in a minute.
Now for Popup pages. Unlike the background script and content script, popups have both a HTML and JS portion. The HTML part is just like any other page, just small and as a overlay popup coming out from the icon. The script portion of it, however, can do all the things the background page does, except that it unloads whenever the popup is closed.
Now that the distinctions are more clear let's move on to what you want to do. It sounds like you want to open the popup, have the user enter text to search for in the current tab, then highlight that text on the page. Since you said that you already know how you plan on highlighting the text, I will leave that part to you.
First to set up our manifest file. For this particular action, we don't need a background script. What we do need is both the "tabs" and "activeTab" permissions. These will enable us to inject our script later. We also need to define the browser action with it's popup. Altogether it would look something like this:
"browser_action": {
"default_icon": "icon.png",
"default_popup": "popup.html"
},
"permissions": [
"tabs", "activeTab"
]
Now in our popup.html file, we can only have markup and css, no inline code at all. We will put it all in our js file and include it. Something like this should work:
<!DOCTYPE html>
<html>
<head>
<script src="popup.js"></script>
</head>
<body>
<input type="text" id="searchText">
<button id="searchButton">Search</button>
</body>
</html>
This is where we come back to the content script stuff. There are two ways to inject a content script, first is to define it in the manifest. This works best when you always want to inject it for a particular set of url's. Second, to use the chrome.tabs.executeScript method to inject it when we need to. That is what we will use.
window.onload = function(){
document.getElementById('searchButton').onclick = searchText;
};
function searchText(){
var search = document.getElementById('searchText').value;
if(search){
chrome.tabs.query({active:true,currentWindow:true},function(tabs){
chrome.tabs.executeScript(tabs[0].id,{file:search.js});
chrome.tabs.sendMessage(tabs[0].id,{method:'search',searchText:search});
});
}
}
With this, we have successfully injected our script and then send the search text to that script. Just make sure that the script is wrapped in a onMessage listener like this:
chrome.runtime.onMessage.addListener(function(message,sender,sendResponse){
// message.searchText is the text that was captured in the popup
// Search/Highlight code goes here
});
And that pretty much sums it up. With that, you should be able to get it working. If something is still not clear let me know and I will fix it.
I think what's confusing you is the non-existant concept of a "content page". There is no such thing. What you're likely referring to is a "content script". Let me explain the three main components of an extension:
Background Page
As you said, this is the persistent aspect of a Chrome Extension. Even though it can be HTML page it is never rendered. You simply use it to run JavaScript and other content that stays persistent. The only way to "refresh" the background page is to refresh the extension in the extension manager, or to re-install the extension.
This is most useful for saving information that should remain persistent, such as authentication credentials, or counters that should build up over time. Only use the background page when absolutely necessary, though, because it consumes resources as long as the user is running your extension.
You can add a background script like to manafest file like this:
"background": {
"scripts": [
"background.js"
]
},
Or like this:
"background": {
"page": "background.html"
},
Then simply add background.js to background.html via a typical tag.
Popup
This is what you see when you click the icon on the toolbar. It's simply a popup window with some HTML. It can contain HTML, JavaScript, CSS, and whatever you would put in a normal web page.
Not all extension need a popup window, but many do. For example, your highlight extension may not need a popup if all it's doing is highlighting text on a page. However, if you need to collect a search result (which seems likely) or provide the user with some settings or other UI then a popup is a good way to go about this.
You can add a popup to the manifest file like this:
"browser_action": {
"default_popup": "popup.html"
},
Content script
As I mentioned, this is not a "page" per se -- it a script, or set of scripts. A content script is what you use to infuse JavaScript into pages the user is browser. For example, a user goes to Facebook and a content script could change the background to red. This is almost certainly what you'll need to use to highlight text on a page. Simply infuse some JavaScript and any necessarily libraries to search the page or crawl the dom, and render changes to that page.
You can inject content scripts every time a user opens any URL like this:
"content_scripts": [
{
"matches" : [
"<all_urls>"
],
"js" : [
"content.js"
]
}
],
The above injects "content.js" into "all urls".
You'll also need to add this to the permissions:
"permissions": [
"<all_urls>",
]
You can even add JQuery to the list of content scripts. The nice thing about extensions is that the content scripts are sandboxed, so the version of JQuery you inject will not collide with JQuery on pages the user visits.

Categories

Resources