I'm currently trying to learn how to scrape diffrent site, so I got a couple on a list that I'm need to do. I'm not really sure how it works yet, but was wondering if I could use Firebug to find the specific part where I want my scraper to go to?
test-page
The link is to one of the pages I'm trying to scrape from. From this one am I going to get the abstract section.
If I then go in firebug and find the classes I'm looking for I se that I can get the path in X-path. Is this something I can use to setup my scraper or?
/html/body/div[2]/div[8]/div/div[2]/div/div[2]/div[1]/div/div[1] <-- Xpath to
And If I can use this to send my scraper there, how can some example look?
Help is much appreciated!
Related
Wondering if anyone can point me in the right direction on how I would be able to scrape the data from this website. I understand the data is filled in after the page is fully loaded, and have seen js libraries that can request the data be loaded, but can't remember the name of them. I prefer to code in python though if possible.
https://stockflare.com/stocks/BBY
I think this would work. Found it from another answer regarding react webpages. Can anyone confirm?
python-casperjs
It's not legal to scrape our data. :( Please drop me a message if you'd like to work with us. Shane at Stockflare dot com.
Shane
This question might not be suitable for this website and I am only asking this for information purposes..
So, please let me know if this is not suitable and I will delete it.
I have created a web application using PHP and Javascript.
what I want to do is to find a way to turn the entire thing into a small javascript code so I only give that piece of javascript code to the users and they can copy and paste it in their website in order to be able to use the application on their own website without being able to edit the contents of it!
could someone please advise on this?
Again, please let me know if this question is not suitable for this site and I will delete it.
There are two ways to go about this, make it available as an iframe widget (which could theoretically be injected through javascript) or make a cross-domain API with which the javascript would interface. From the sound of it going down the iframe route sounds most sensible, although it does come with a clickjacking vulnirability.
I have an app i'm developing with JQ Mobile and PhoneGap. Within this app, i want to include a 'Notes Taking' page and it has to do the following.
Make new notes and save them locally.
Recall them whenever i need to,
Edit them and delete them
i could not find a good example to follow, i tried this:
http://tutorialzine.com/2012/09/simple-note-taking-app-ajax/
but he lost me at the index php and how to implement that, i also tried this example:
http://miamicoder.com/2011/building-a-jquery-mobile-application-part-1/
but its so complex i got lost in all the code.
I'm a beginner at programming and my JavaScript is very very poor and i have no idea where to start and what kind of code is involved in making this part of the app.
Please help. Thank you.
Your first site (http://tutorialzine.com/2012/09/simple-note-taking-app-ajax/) will not work for you as the examples are embedding PHP into the html and that is not acceptable with phonegap apps.
Your second site is making it way more complicated than it needs to be for your limited scope.
Not going to do this for you but I will give you some advice and places to look.
Draw out what you want the user interface to look like and create that html
Create a load and save function in javascript to... well... load and save your notes. Take a look at the below references:
This will show you how to interact with the local database and store and retrieve info. Always a good thing to read the documentation.
This is a great resource for help all sorts of stuff. In this case, with SQL queries.
Test, test, test. With the above, you will be able to figure it out if you put your mind to it. Come back here and post specific questions with code samples of what you have tried if you get stuck.
I'm trying to make a private HTML document on my flash drive that when launched, will log me into a website. I've searched and searched and I have yet to find someone online with the same problem, thus I could not find any help. This would be for sites that log in using the POST method of course. I know it's possible to request a page, but is it possible to control the elements inside of it?
Any information is appreciated.
You probably want to make a userscript with scriptish or a make it as a bookmarklet.
I'd recommend something like KeePass to do this. Any website worth it's salt won't allow you to do a cross-domain POST.
I'm thinking about creating a webpage and I'm trying to brainstorm some ways to display them in the page.
If i wanted to get dirty and create everything myself, i think i could do it with html5, CSS3, and javascript/jquery. Just do some kind of page buttons with an image tag and maybe get into some more detailed stuff as it comes up (i dont know how i would do zooming and multiple pages).
But wahat i really want to know is if there is already some way to do this? I've looked around for a bit and cant seem to find any sort of plugin that would read a cbz file or display an set of images with the 'e-reader' type of tools in mind. Just wondering if anyone knows of anything?
Thanks
I used to use an online reader for a long time so I started an experiment to build one myself a while back: netcomix
It's open source so you can see if you find anything appealing in what I did. I figured I'd do all the real UI work client side with HTML, CSS, and JavaScript and the server was strictly responsible for acting as a service (for example, to supply a list of comics or a list of all the pages in a particular issue) and serving up the individual JPG/PNG/GIF files. That compartmentalized things nicely and I was very pleased with how jQuery BBQ gave me a history that I could back through even though I stayed on one page the whole time.
Now if I were to do the same experiment again, I'd use Backbone.js to give some structure to the client side and obviously it needs a lot of love because the server side really does nothing at the moment. Early versions were strictly hard coded although I started putting in some simple SQL stuff in there in the latest version. It's nothing more than an experiment though and should be treated as such. It's there for ideas and little else. If you find it interesting and want some more ideas contact me and I'll be happy to let you know all my wacky ideas for such a program.
I know this is an old question. But web technologies have gotten better in the last few years. There are several comic book readers that can work in the browser using pure HTML and JavaScript. I wrote one called: http://comic-book-reader.com .
If you want to see a very simple example of how to read CBR and CBZ files in the browser. You should check out http://workhorsy.github.io/uncompress.js/examples/simple/index.html which uses the JavaScript library https://github.com/workhorsy/uncompress.js