I've got a problem. I'm working with a food supplier and I need save the content of each order as html. Orders are listed on a single page as links, but this has 2 difficulties
Page uses authentication (need to log me in in advance)
This is the real problem: the page use a lot of javascript. Actually everything works without changing the web address so I can't use wget or rio gem (url not like www.fooddoe.com/order, www.fooddoe.com/order/1, etc. but always like www.fooddoe.com/suplierx).
I think firewatir would be a good option but the problem is than I need to save the page in a format similar to html (including images). Is it possible using firewatir? Are there other options in clojure or javascript?
Thanks so much!!
I had to read your question twice to understand what you mean.
From web address from example I assume this is yours supplier web page. So IMHO the easiest way is:
Look into source of web page to get an idea how it gets the data (99% for some kind of AJAX request).
Request goes to the server which responds to it.
Now there are two ways:
Get idea how the request is made and write and app to make such request and generate web page with it (more difficult, more general)
Contact your supplier and get original database (simpler but one-time solution)
And I think that this is not the question specific to any language.
Related
I am trying to set up a simple set up as follows:
Have a mobile app with a page consisting of 4 lines (4 html paragraph lines (I am using phonegap)).
I want to use a web page from which I will input the data for those 4 lines. This information is sent to a server and that server transfers this information to that app on that mobile phone. Now, those 4 lines on the mobile phone is filled with the new information.
Similarly user inputs information on another page consisting of 10 lines of li (list). This information is again sent to the server and to the web page where the information is displayed.
I can almost feel the "internet police guys" getting all hyped and ready to vote this question down. But please understand that I have been on this site and various forums desperate to find a tutorial to guide me to do this and not able to find.
I am trying to use ajax to perform this setup. Confused how I would be using the php file. Information such as password n username is going to go in that php file to connect to the server. But php is a server side script thus needs to sit at the public_html folder. How do I use the php file from my desktop? Write a separate javascript to access it?
It is the concept that is confusing me. I am familiar with html,js,php.
I would appreciate any guidance or maybe a link to a tutorial which would help me to do the concept I mentioned. Thanks for listening.
You will need to create an API using PHP. This API is uploaded to your server and is considered "RESTful". Google a tutorial for what fits your needs. You can set all sorts of rules in this API such as requiring any requests to have an ID or access token.
Since you are using PhoneGap, your HTML and JS files rest on the device, so you will need to allow permissions to your API from anywhere. For this you will have to speak to your host provider about unless you know how to configure it yourself (some providers restrict what you want to do by default as an extra security precaution against XSS attacks).
Next, you can either use jQuery, or you can write some AJAX calls by writing the JavaScript yourself.
The most efficient way for this to work is to send JSON objects to and from the API. You will include a "command" in the JSON when you are sending from your app. On the PHP side, you will retrieve this command and use the rest of the data included in your JSON object to process the request. Your API will need to encode a JSON object for return (such as a user's profile information).
Here is a basic PHP API tutorial to get you going that explains some of the features of a RESTful API: PHP API
Here is a simple AJAX function (you will probably want to make this much more modular): AJAX
As broad as your question is, it seems like the best/easiest thing for you to do will be for you to first create a PHP webpage that will access a SQL database to perform the record updating. Actually, this should serve all of your needs for your mobile users assuming you don't need push notifications for live data updates.
I am assuming, since you are using phone gap, that you are more comfortable with web languages. After you get the webpage fully operational, then you should start building your app based on that exact same SQL database. With mobile app development there are a lot more "what if's" (what if the phone rings, what if the app is running in the background, what if there is no cellular service, etc...)
It is always easier to start with what you know and build on that, rather than starting with a new development platform and troubleshooting as problems arise.
I have a wordpress site, and i'm not a php developer and not very eager to start either so I'm avoiding it like the plague, but I do have a requirement that requires a little bit of extra coding. I need to:
go to a different website,
download that page,
check for a certain phrase,
if phrase exists, extract a link from that page
and if link is returned I need to show that link on my wordpress site.
Currently, I have an asp.net page that does this and i'm hosting that page in an iframe on my wordpress site. but i'd like to do it without an iframe.
Question is, is there anyway for javascript to go to a different page (my asp.net page) and get a parameter (link) from it. If link is provided i will show certain content on wordpress site.
Or can javascript download a text file from the server? problem with that is i need a trigger to update the text file.
Any advice is appreciated.
Thanks.
What you should understand is that by "avoiding [PHP] like the plague" you're inadvertently avoiding the proper way of doing things. Javascript is a client-side language, and PHP is a server-side language. By asserting that you only want the load on the client's end (the kind of logic involved in what you want to do isn't exactly lightweight), you can potentially end up with a VERY slow webpage.
Not to mention, this type of situation is analogous to using a hammer to do a backhoe's job.
Either way, to answer your question, yes. You can use the jQuery Load method in tandem with Javascript's Match method.
What you should TRY to do, however, is make a CURL request using PHP, and then cache the page on your server. By doing this, you will limit the number of calls to a given page, and optimize load times.
Please consider the second option, even as an attempt in good practice. Good luck.
Use ajax and connect to a different page (on your server) which is written in server-sided language (like asp.net, as you said), that connects to the remote website.
More about Ajax
Is there a method or is it even possible to get a products details by using a URL. Let's say I paste a URL of a product from a store like Walmart Or bestbuy, would it be possible to write something to retrieve the product info (price, name, info, etc..) does this exist? Or would this have to be something site specific that I can write for each specific store?
One solution I see is to parse the HTML code of the page the URL redirects to using for example Tika, but I'm not sure the e-commerce website in question will like that very much :) Maybe you could ask them if they have implemented an API to access their products data?
Yes, it is possible, but not using JavaScript due to same-origin-policy. You must send that URL to the server, read that external page on the server side and return results back to the server.
On the server side (in whichever language you are using) download the web page, parse it (using xml/xpath if you can) and extract relevant information.
As already noted watch out, some websites forbid such access (called web-scraping), other might actively try to prevent that, e.g. by discovering fake clients.
What you're talking about is website scraping and yes, it's possible and there are loads of tools out there to help you with it. Some websites aren't happy with you doing it though.
You could do it in C# using the HttpWebRequest class to request data from a url and then parse it with something like XmlReader or the http://html-agility-pack.net/
In my app, I will provide my client a javascript plugin, which will collect some HTML data and send to my server. I wonder what's the best way to identify my client. Say someone copied the javascript and put into his website. A similar case is the live chat plugin.
Really your questions it is not very clear to me. I am monitoring it from the beginning, so as no one answers I can say the following:
1.- If your javascript plugin is to plug in websites, as a jquery plugin, then you don't be sure about nothing because the code can easily be modified to remove any security procedure.
2.- If your javascript plugin is to plug in browsers, as a FF addon. Well, indeed can be modified too, but in the most of cases you can track simply with cookies or a login procedure.
Said that I think that if the case is the first (plug in websites) you could identify the websites asking for a authentication token stored in the server's website (requested by AJAX) and add it to the HTML data that is send to your server.
Hopefully you can understand my Emglizch :) and do not say pure garbage.
In the new Yahoo mail inbox, when u click the message it is displayed in a tab automatically (I guess with out server interaction). Does that mean Yahoo loads all the data first and then use them with java script when requested or not...anyho i don't have any idea and I would like if some one explain to me how it works since am planning to do the same with my application. I am sure this will boost application performance and i am eager to know.
I guess Yahoo did something similar to what Hotmail describes here
Basically they decide depending on several aspect what+when to preload...
I have not seen it but what you're describing sounds like dynamic AJAX loading. Basically, only load information when it is requested by the user. This will reduce network load and initial loading times. Most JS libraries have some form of AJAX helper. You can read more on AJAX here and here.
I am pretty sure it does have some server interaction. It definitely is using some sort of AJAX to fetch data from the server and show it to you. There are tons of tutorials about using AJAX which you can refer. You can probably start with http://w3schools.com/ajax/default.asp