Develop Reference

Take screenshot from external website

I am developing a startpage where users can add links to the page by using a formular. They can add name, url, description and upload an image.
I want to automate the process of uploading an image, the image should be captured automatically. My script should take a screenshot of the website which the user entered in url. I know I can take screenshots of html elements by using html2canvas.
Approach 1
My first approach was to load the external website to an iframe, but this does not work because some pages are restricting this, e.g. even the iframe tutorial on w3schools.com does not work and I get Refused to display 'https://www.w3schools.com/' in a frame because it set 'X-Frame-Options' to 'sameorigin'.
HTML
<div id="capture" style="padding: 10px; color: black;">
<iframe src="https://www.w3schools.com"></iframe>
</div>
Approach 2
My next approach was to make a call to my webserver, which loads the target website and returns the html to the client. This works, but the target site is not getting rendered properly, e.g. images are not loading. (see screenshot below)
HTML
<div id="capture" style="padding: 10px; color: black;"></div>
JS
var testURL = "http://www.google.de";
$.ajax({
url: "http://server/ajax.php",
method: "POST",
data: { url: testURL},
success: function(response) {
$("#capture").html(response);
console.log(response);
html2canvas(document.querySelector("#capture")).then(
canvas => {
document.body.appendChild(canvas);
}
);
}
});
PHP
if (!empty($_POST['url'])) {
$url = filter_input(INPUT_POST, "url");
}
$c = curl_init($url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
//curl_setopt(... other options you want...)
$html = curl_exec($c);
if (curl_error($c))
die(curl_error($c));
// Get the status code
$status = curl_getinfo($c, CURLINFO_HTTP_CODE);
curl_close($c);
echo $html;
Is it possible to achieve this?
Update
I managed to load some pictures by changing my ajax, but they are not rendered by html2canvas.??
var testURL = "http://www.google.de";
$.ajax({
url: "http://server/ajax.php",
method: "POST",
data: { url: testURL},
success: function(response) {
response = response.replace(/href="\//g, 'href="'+testURL +"/");
response = response.replace(/src="\//g, 'src="'+testURL +"/");
response = response.replace(/content="\//g, 'content="'+testURL +"/");
$("#capture").html(response);
console.log(response);
html2canvas(document.querySelector("#capture")).then(
canvas => {
document.body.appendChild(canvas);
}
);
}
});
Result
Result Canvas

I love php, but for screenshots I found that using phantomjs provide the best results
Example file screenshot.js
var page = require('webpage').create();
page.open('https://stackoverflow.com/', function() {
page.render('out.png');
phantom.exit();
});
Then from the shell:
phantomjs screenshot.js
Or from php:
exec("phantomjs screenshot.js &");
The goal here is to generate the js file from php.
Result in a file called out.png in the same folder. This is a full height page screenshot.
Example output
We can also take good captures with Firefox from the command line. This require X anyway.
firefox -screenshot test.png http://www.google.de --window-size=1280,1000
Example output

Not in pure php. Nowadays major number of sites generates content dynamically with js. It can be rendered only by browsers, but good news - there is something called phantomjs - browser without UI. It can do job for You, even they have working example in their tutorials which I succesfully implemented few years ago with small knowledge of javascript.
There is alternative library called a nightmarejs - I know this only from friends opinion which says that it's simpler than phantom, but I won't guarantee to You that it won't be a nightmare - personally I hadn't use it.

It is possible, but if you want an screenshot you need something like a browser that render the page for you. The iframe approach go in that way. But iframe is the page itself. If you want a .jpg , .png or something like that, the best way in my opinion is using wkhtmltoimage. https://wkhtmltopdf.org/.
The idea is that you install Qt WebKit rendering engine in your server, just as you install a browser in your server, this render the page and save the final result in a file. When some user submit a url, you pass it as argument to wkhtmltopdf then you could have an image of that url. The basic use could be somethig like
wkhtmltoimage http://www.example1.com /var/www/pages/example1.jpg
you should run that statement in bash, from php could be:
<?php
exec('wkhtmltoimage http://www.example1.com /var/www/pages/example1.jpg');
?>
Keep in mind that wkhtmltoimage execute css, javascript.., everything. Just like browser.

PhoneGap load local page inside HTML

I tried to load a page inside my phonegape aplication with Jquery .load() but it doesn't work since it's not on a server and it's on local machine. When I'll upload the app on build. phonegape my page will still be included on the www folder. How can I load a page inside a div ?
This is not working
<script>
$(".form-control2").change(function(){
$(".plan_mese4").load("select_mese.html");
});
</script>
If I use http it's working but i don't need that since my file is local and not on a server.
edit*
Right now i have my app on desktop, i also compiled it with phonegap and .load() it's not working since the file is local. It only works if i put my file on a server and i load it from there but i don't want that since the request take more time.

thnx to Quentin i found that i can use the filepath to make an ajax call even if it's local. So i did it like this if anyone else needs it. I get the url location in my case file:///E:/rezerv.city/www/local.html?id=114&tip=5and i remove the local.html?id=114&tip=5.html part and i add what i need in my case select_mese.html.
$(".form-control2").change(function(){
var url =window.location.href;
url = url.substring(0, url.lastIndexOf("/") + 1);
$(".plan_mese4").load(url +'index.html');
});

in my app I'm use:
<div id="loadExternalURL" class="site"></div>
document.addEventListener("deviceready", onDeviceReady, false);
function onDeviceReady() {
$('#loadExternalURL').load('http://www.site.com.br');
$.ajax({
dataType:'html',
url:'http://www.site.com.br',
success:function(data) {
$('#ajax').html($(data).children());
}
});
}
It works for me.

Load a text file from the same directory

I'm working with Play Framework (using the port 9000). When I use Ajax indicating the URL to a txt I try to indicate it in the following way:
...
<script>
var fileContent;
$.ajax({
url : "http://localhost:9000/text.txt",
dataType: "text",
...
</script>
...
The text file "text.txt" is on the same folder as the html file that calls it. However, when I try to access it, I just can't in this way. I did set up Apache yesterday to test it and I did try typing
http://localhost/text.txt
in the web browser and it loads, but when I do that with Play, it does not allow me. Any ideas? Thanks!

how to load content of word document in textarea

I am new to jQuery and JavaScript. I am trying to read the content of a .doc file and display it in a textarea.
var url = "D:\\way2Jobs\\way2jobz\\WebContent\\pages\\Resumes\\";
var firstName = $("#first").val();
var extn=".doc";
jQuery.ajax({
url : url+firstName+extn,
dataType : "doc",
success : function(data){
alert(firstName);
document.getElementById("candResume").innerHTML = data;
}
});

You just can't (thanks god) make an ajax request to your local filesystem. It's a safety restriction and you can't bypass this.
1 - You have to at least use a webserver like apache
2 - You never will sucessfuly make a request to D:\
3 - You CAN request a DOC file and process it using javascript. But it's not that easy because DOC file isn't a plain text and javascript was not made for it. Maybe it's easier for you to do it using a server side language such as PHP or JAVA or even NodeJS if you are familiar with javascript.

Load page with Ajax and load new scripts if necessary

I have a page1 which has a wrapper with articles.
<div id="wrapper">
<article>...</article>
<article>...</article>
<article>...</article>
<article>...</article>
</div>
I am loading new articles from page2 with Ajax and appending them to the wrapper
$.ajax({
url : page_to_load,
success : function (data) {
$('article',data).appendTo('#wrapper');
}
});
But some of this new articles might need specific scripts that were not loaded in page1 but would be loaded if accessed directly page2.
So the problem is some of the content in new articles breaks as they are missing those scripts.
What is the best solution? I can think of checking the new loaded page scripts and compare them with the already loaded and download the new ones, but I have no idea of how to do this.
EDIT
I noticed if I set the dataType to 'html' I cant search for the scripts, although they are there:
$('script',data)//doesn't match anything
But if I do:
console.log(data);
I can see the whole page with <html> and <script> tags

There is no problem actually, if you append HTML to the Dom then script calls will be interpreted as if the page was loaded directly, just make sure you use the same parameters as the shorthand jquery $.POST method.
I actually do this all the time and the <script src=""> are called and interpreted correctly
Just make sure you're accessing the scripts from the right scope as if the html was hardcoded on page1.
If this doesn't work, then check with the web inspector if the scripts are loaded or not.

Working solution:
$.ajax({
url : page_to_load,
success : function (data) {
// load the scripts
var dom = $(data);
dom.filter('script').each(function(){
var scriptSrc = $(this).attr('src');
if(!$('script[src="'+ scriptSrc +'"]').length && scriptSrc !== undefined) {
var script = $("<script/>");
script.attr('src', scriptSrc);
$("head").append(script);
}
});
// load the articles
$('article',data).appendTo('#wrapper');
}
});

Not sure but maybe you could add a script in the AJAX call - I'm not sure of this because I haven't tried it:
Proxy.Scripts.Add(new ScriptReference(AddVersion("/Pages/Items/SearchList/SearchList.Grid.js" )));

This workflow should work :
After every page request :
Get all the script tags in the loaded page.
Loop over the scripts
If it's not loaded yet , load it.
Here is the code snippet :
$.ajax({
url : 'http://localhost',
success : function (data) {
$('article',data).appendTo('#wrapper');
$.each($.parseHTML(data,null,true),function(){
if($(this)[0].tagName == "SCRIPT"){
var src = $(this).attr('src');
if($('script[src="'+ src +'"]').length){
$.getScript(src);
}
});
}
});

Disable async.
$.ajax({
url : page_to_load,
async: false,
success : function (data) {
$('article',data).appendTo('#wrapper');
}
});

you can use RequireJS or HeadJs library for calling js files.
RequireJS is a JavaScript file and module loader and HeadJS, a small library for Responsive Design, Feature Detections & Resource Loading
HeadJs is great and useful, try it.