I have a limited hosting server. I want to read the filetree (all files and folders) and create a hyperlink to them on a basic html page.
For clarity, I'm using Keybase, am publicly sharing files, but want to list them on an index.html page, not use their site's "filetree"
https://keybase.pub/example_user (keybase filetree)
https://example_user.keybase.pub (the index.html file)
The html file is in the root directory and I want to display all the (pdf) files in /subdir (and their sub-directories)
This isn't a "real" webserver. I'm looking for something easy and simple like a FOR loop on load within html
Thanks.
What you are trying to do is known as "screen scraping". If you do some googling on the keywords "javascript screen scraping" you will find lots of information and examples.
Basically, You fire off an AJAX request to retrieve the content of a page, parse that content to obtain the data your looking for, and then display that data in your page.
I want to write a npm package to localize an html url.
1. using the html url download the html page
2. parse the html file, extract all the js, css and img files used in the html and local these resources.
3. If these js, css and img files using some external resources, localize these resources. For example, extract background image in the css.
The first and second requirements are easy to meet. But I have no idea about the last one.
I can parse the all the css files and localize the resources used in it. But how can I parse the js files?
For example:
If the js adds a 'script src = XXX' tag into the html dom, how can I extract the src?
I think I would try to use a headless browser to catch every network calls instead of trying to parse the code.
I didn't used it personally but PhantomJS seems to fit the bill.
It can be used to load a webpage then execute any script / css that would normally happen on the request and execute stuff once the page is loaded.
The network monitoring features are probably what you'll want to use.
Im using a array to display some images in a website:
var paintingImages;
paintingImages =
[
{
url: 'images/objects/ron.jpg',
alt: 'ron'
}
];
This js code is written in paintings.js and my main js code is written in the file main.js
I have made this website for a artist and I want to give him the opportunity to login and add pictures to his website. I'm not that good with php, but adding images to a ftp in a folder is no problem.
Because I'm using a array for retrieving the images, I need to be able to add items to the array.
This is the part where I'm stuck. I don't know how to edit a existing js file, so the next time I open the website, the items (images) will been shown.
Activexobject is not a option because it's only possible in IE.
In summary:
I need to add a item to a array
I need to save this into the file, so the next time the website opens, it will be shown
I'm not verry good with php, I prefer javascript
Can't use Activexobject because of the use of multiple browsers
You can not simply modify a javascript file sitting on the server from browser side javascript.
You need to implement some server side logic.
If you don't like PHP, but like JavaScript, check out NodeJS for example.
With Node you should be able to build some lightweight serverside logic to modify your json array file with additional images.
You would need to change your logic .. You need to send this data to server anyhow . and when you load your next time take it back from server and update your webpage accordingly. I would suggest use any javascript MVC framework like backbone, angular.js. it will help you.
Use a database.
Save it to a database server (You figure this out, could be via forms, xhr, websockets, etc.).
On page load, load the correct data for the current user.
Create the $userPaintings array (call it what you want) from the user data in PHP.
Then simply output a JS object in the page with PHP:
<script>
var paintingImages = <?php json_encode($userPaintings); ?>
</script>
I need to be able to save a page on my website to my harddrive, so that i can use it both online and offline. The thing is, the page uses references to javascript and CSS files outside it's own folder. It is very important that i can save the whole page as one .html file, so that all the javascript and CSS code from the external files are in that file as well.
Is there a way to do this?
This should be programatically
As you request, this can be done programmatically with lets say Python.
The pattern of the code would look like this:
Request the user to paste the url in a box
wget or curl the page and use regex to find out where the included codes are
OR: use a library like SGML to interact with the HTML tags directly
Put all the linked CSS, JS etc. files in a List
Fetch all the linked files their contents and put them in a List
Rebuild the HTML source code and strip out the and tags stuff
Now loop the linked files's contents in the tags like this:
newHeaderContent = ''
for content in linkedFilesArray:
newHeaderContent = newHeaderContent + content
newHTML = firstHTMLCode + newHeaderContent + lastHTMLCode
I want to make an app that can display on any webpage, just like how Disqus or IntenseDebate render on articles & web pages.
It will display a mini-ecommerce store front.
I'm not sure how to get started.
Is there any sample code, framework, or design pattern for these "widgets"?
For example, I'd like to display products.
Should I first create a webservice or RSS that lists all of them?
Or can one of these Ajax Scripts simply digest an XHTML webpage and display that?
thanks for any tips, I really appreciate it.
Basically you have two options - to use iframes to wrap your content or to use DOM injection style.
IFRAMES
Iframes are the easy ones - the host site includes an iframe where the url includes all the nessessary params.
<p>Check out this cool webstore:</p>
<iframe src="http://yourdomain.com/store?site_id=123"></iframe>
But this comes with a cost - there's no easy way to resize the iframe considering the content. You're pretty much fixed with initial dimensions. You can come up with some kind of cross frame script that measures the size of the iframe contents and forwards it to the host site that resizes the iframe based on the numbers from the script. But this is really hacky.
DOM injection
Second approach is to "inject" your own HTML directly to the host page. Host site loads a <script> tag from your server and the script includes all the information to add HTML to the page. There's two approaches - first one is to generate all the HTML in your server and use document.write to inject it.
<p>Check out this cool webstore:</p>
<script src="http://yourdomain.com/store?site_id=123"></script>
And the script source would be something like
document.write('<h1>Amazing products</h1>');
document.write('<ul>');
document.write('<li>green car</li>');
document.write('<li>blue van</li>');
document.write('</ul>');
This replaces the original <script> tag with the HTML inside document.write calls and the injected HTML comes part of the original page - so no resizing etc problems like with iframes.
<p>Check out this cool webstore:</p>
<h1>Amazing products</h1>
<ul>
<li>green car</li>
<li>blue van</li>
</ul>
Another approach for the same thing would be separating to data from the HTML. Included script would consist of two parts - the drawing logic and the data in serialized form (ie. JSON). This gives a lot of flexibility for the script compared to the kind of stiffy document.write approach. Instead of outpurring HTML directly to the page, you generate the needed DOM nodes on the fly and attach it to a specific element.
<p>Check out this cool webstore:</p>
<div id="store"></div>
<script src="http://yourdomain.com/store_data?site_id=123"></script>
<script src="http://yourdomain.com/generate_store"></script>
The first script consists of the data and the second one the drawing logic.
var store_data = [
{title: "green car", id:1},
{title: "blue van", id:2}
];
The script would be something like this
var store_elm = document.getElementById("store");
for(var i=0; i< store_data.length; i++){
var link = document.createElement("a");
link.href = "http://yourdomain.com/?id=" + store_elmi[i].id;
link.innerHTML = store_elmi[i].title;
store_elm.appendChild(link);
}
Though a bit more complicated than document.write, this approach is the most flexible of them all.
Sending data
If you want to send some kind of data back to your server then you can use script injection (you can't use AJAX since the same origin policy but there's no restrictions on script injection). This consists of putting all the data to the script url (remember, IE has the limit of 4kB for the URL length) and server responding with needed data.
var message = "message to the server";
var header = document.getElementsByTagName('head')[0];
var script_tag = document.createElement("script");
var url = "http://yourserver.com/msg";
script_tag.src = url+"?msg="+encodeURIComponent(message)+"&callback=alert";
header.appendChild(script_tag);
Now your server gets the request with GET params msg=message to the server and callback=alert does something with it, and responds with
<?
$l = strlen($_GET["msg"]);
echo $_GET["callback"].'("request was $l chars");';
?>
Which would make up
alert("request was 21 chars");
If you change the alert for some kind of your own function then you can pass messages around between the webpage and the server.
I haven't done much with either Disqus or IntenseDebate, but I do know how I would approach making such a widget. The actual displaying portion of the widget would be generated with JavaScript. So, say you had a div tag with an id of commerce_store. Your JavaScript code would search the document, when it is first loaded (or when an ajax request alters the page), and find if a commerce_store div exists. Upon finding such a container, it will auto-generate all the necessary HTML. If you don't already know how to do this, you can google 'dynamically adding elements in javascript'. I recommend making a custom JavaScript library for your widget. It doesn't need to be anything too crazy. Something like this:
window.onload = init(){
widget.load();
}
var widget = function(){
this.load = function(){
//search for the commerce_store div
//get some data from the sql database
var dat = ajax('actions/getData.php',{type:'get',params:{page:123}});
//display HTML for data
for (var i in dat){
this.addDatNode(dat[i]);
}
}
this.addDatNode = function(stuff){
//make a node
var n = document.createElement('div');
//style the node, and add content
n.innerHTML = stuff;
document.getElementById('commerce_store').appendNode(n);
}
}
Of course, you'll need to set up some type of AJAX framework to get database info and things. But that shouldn't be too hard.
For Disqus and IntenseDebate, I believe the comment forms and everything are all just HTML (generated through JavaScript). The actual 'plugin' portion of the script would be a background framework of either ASP, PHP, SQL, etc. The simplest way to do this, would probably just be some PHP and SQL code. The SQL would be used to store all the comments or sales info into a database, and the PHP would be used to manipulate the data. Something like this:
function addSale(){ //MySQL code here };
function deleteSale(){ //MySQL code here };
function editSale(){ //MySQL code here };
//...
And your big PHP file would have all of the actions your widget would ever need to do (in regards to altering the database. But, even with this big PHP file, you'll still need someway of calling individual functions with your ajax framework. Look back at the actions/getData.php request of the example JavaScript framework. Actions, refers to a folder with a bunch of PHP files, one for each method. For example, addSale.php:
include("../db_connect.php");
db_connect();
//make sure the user is logged in
include("../authenticate.php");
authenticate();
//Get any data that AJAX sent to us
var dat = $_GET['sale_num'];
//Run the method
include("../PHP_methods.php");
addSale(dat);
The reason you would want separate files for the PHP_methods and run files, is because you could potentially have more than one PHP_methods files. You could have three method API's, one for displaying content, one for requesting content, and one for altering content. Once you start reusing your methods more and more, its best to have them all in one place. Rewritten once, rewritten everywhere.
So, really, that's all you'd need for the widget. Of course, you would want to have a install script that sets up the commerce database and all. But the actual widget would just be a folder with the aforementioned script files:
install.php: gets the database set up
JavaScript library: to load the HTML content and forms and conduct ajax requests
CSS file: for styling the HTML content and forms
db_connect: a generic php script used connect to the database
authenticate: a php script to check if a user is logged in; this could vary, depending on whether you have your own user system, or are using gravitars/facebook/twitter/etc.
PHP_methods: a big php file with all the database manipulation methods you'd need
actions folder: a bunch of individual php files that call the necessary PHP methods; call each of these php files with AJAX
In theory, all you'd have to do would be copy that folder over to any website, and run the install.php to get it set up. Any page you want the widget to run on, you would simply include the .js file, and it will do all the work.
Of course, that's just how I would set it up. I assume that changes in programming languages, or setup specifics will vary. But, the basic idea holds similar for most website plugins.
Oh, and one more thing. If you were intending to sell the widget, it would be extremely difficult to try and secure all of those scripts from redistribution. Your best bet would be to have the PHP files on your own server. The client would need to have their own db_connect.php, that connects to their own database and all. But, the actual ajax requests would need to refer to the files on your remote server. The request would need to send the url of the valid db_connect, with some type of password or something. Actually, come to think of it, I don't think its possible to do remote server file sharing. You'd have to research it a bit more, 'cuz I certainly don't know how you'd do it.
I like the Azmisov's solution, but it has some disadvantages.
Sites might not want your code on their servers. It'd be much better if you would switch from AJAX to loading scripts (eg. jQuery's getJSON)
So I suggest:
Include jquery hosted on google and a short jquery code from your domain to the client sites. Nothing more.
Write the script with cross-domain calls to your server (through getJSON or getScript) so that everything is fetched directly and nothing has to be inctalled on the client's server. See here for examples, I wouldn't write anything better here. Adding content to the page is easy enough with jQuery to allow me not mentioning it here :) Just one command.
Distribute easily by providing two lines of <script src= ... ></script>