A few weeks ago I started learning Javascript and the Google Apps Script API, specifically in regard to spreadsheets. I have been trying to make a spreadsheet that fetches web pages and pulls stats about my friends for the game League of Legends. However, I have been running into a problem with the site I want to use, which is basically the only free LoL stats site that updates frequently. I'm not familiar at all with web development, but it seems when I try to access a page on lolking.net, for example http://www.lolking.net/summoner/na/60783 with Google's UrlFetchApp.fetch() it does not load the dynamic page. So instead of the final source, I get this which doesn't help me. Is there an easy way around this or would I simply have to use another website?
Thanks for thie info! Although it turns out I was mistaken. The UrlFetchApp was indeed returning the full source code, but I was using GAS's Logger to view the text. It seems the Logger has a length limit, so when I searched for the stats I wanted they weren't there simply because the source code got truncated. So, due to an oversight on my part, I never had a problem in the first place. For other people reading this question, in the end I have no idea how UrlFetchApp works with dynamic pages using clientside js (you'd probably want to talk to the poster below or post a new question).
You are getting fhe raw html page with clientside js included. That wont work from any system not just gas. You need to debug that page js and find where it does an ajax call to get the data you want.
Then do the same from your gas. Might not work if the call is authenticated etc.
Related
So, i am kinda of a new python programmer, if i can call myself like that, and i am trying to learn through the "picking new projects" procedure.
What i want to do now is :
enter a website that gives live score results as http://www.livescore.com for example.
somehow scrape all the teams that playing against each other and manipulate those data.
Then i want to build an app that takes those data, arrange them nicely in a table format (let's say) and then update it every time a team scores a goal (possibly through scrapping again ? i don't know..). So i want to project them as my own data.
As i am new to python, i don't even know if that's possible to be done.
If so, can you help me ? Point me to some directions maybe, point me specific chapters of python to read, specific modules etc etc ?
I really need any help i can get because i am really lost on the matter.
I don't know where to begin with.
Thanks in advance
For web scraping i would recommand using regular requests library of python + the BeautifulSoup library for parsing HTML. This way you can have a look at the content of the website.
The problem starts with dynamically added data, which is probably the case for you. the actual live data is probably coming from XHR requests the site is making to the server, so there lies the data you are really interested in.
In order to get the data you can try looking at those XHR requests and maybe try to mimic them.
Another platform from extracting data from sites is the Selenium project. Its more of an automated web browser, which gives you the access to all the data , even the dynamically loaded ones.
Hey so currently working on my first personal project so bear with the questions!
Currently trying to create a Javascript program that will parse info from google forms to produce slides displaying the info. So far from my research the best way I've found to facilitate this process is googles app script editor. However, I was wondering if I can run this code by requesting it from a different javascript (or maybe even java) program that I will write code on webstorm. If I cant do this what is the best way to utilize the google apps script editor?
Thanks!
Google Apps Script is just javascript with extra built-in APIs (like SpreadsheetApp, FormApp, etc.).
It also has a UrlFetchApp API.
So you can run code like this:
// The code below logs the HTML code of the Google home page.
var response = UrlFetchApp.fetch("http://www.google.com/");
Logger.log(response.getContentText());
As such, if you want to provide JavaScript from elsewhere, you could fetch it and then eval it on the Google Apps Script side. (but we all know how tricky eval can get)
One other option is to have your own server side written using Google App Engine (or any other framework) and use Google's OAuth and authorize your app to fetch data from the Forms form
Slides and Google Apps Script
You might like to take a look at the addon "Slides Merge" by Bruce McPherson. I've never used it but it sounds like it might work for you. Here's what it's looks like in the addon store:
Getting information from Google Forms is a snap with google apps script since your can link the form right up to a spreadsheet. The Google Apps Script documentation is really quite good these days. Here's the documentation link. Google Apps Script is loosely based on Javascript 1.6. If your already a programmer my guess is that you'll have few problems learning to use it. In my experience the most difficult thing was dealing with the arrays of arrays produced by the getValues() method of ranges in google apps script and I made a short video that might be of some help to you.
I also have a script that I wrote in Google Apps Script that produces a sheet show that is a slide show inside of a spreadsheet.
I've found that using the Script Editor is pretty easy. There's some documentation in the support section of the documentation. It can be a bit buggy at times but overall I think it's a pretty good tool.
I have an iOS app in which I use parse.com as backend service. Now, I hired someone to do a website interface using HTML and CSS. I want to share the same data between iOS app and website, I know parse.com offers me a few ways to do this, including creating a javaScriptapplication. The problem is, my programmer doesn't have any experience in JavaScript, nor do I.
My question is: Is it possible to use what I have (objective-c, xcode) as far as retrieving data from parse.com and showing on website? Even if I need to code something new, is it possible to use objective-c together with HTML and CSS?
Thanks.
Parse has several APIs, one of which is REST. Your web developer should use the REST API to get data from Parse
https://www.parse.com/docs/rest
If there is will there is way, but you'll be making something really specific to your use and will be non standard and will be immediately hard to maintain, I recommend that you hire another developer and do things properly using the technologies given to you by parse !. if the cost will be high now I can promise you it'll be much higher if you went the path you're going to now.
So my answer is:
Yes, everything is possible and no, don't do it ! :)
Edit: Added an example to a possible way to do it to actually answer OP's question.
Example case:
1-Create a simple Mac Application in Xcode that fetches data exactly like you do it on iOS, and store the needed data into a database of your choice on your server
2-You now have access to the data you needed from parse, but on a local mirror. you will need some tool to fetch that data though, I recommend a simple PHP script.
Note that this will require an OSX server to always be running to fetch that data, you'll also need of find a way to fetch data on demand when a user needs it Vs. polling at specified intervals, this will hardly scale and will be costly as I said.
I have an app i'm developing with JQ Mobile and PhoneGap. Within this app, i want to include a 'Notes Taking' page and it has to do the following.
Make new notes and save them locally.
Recall them whenever i need to,
Edit them and delete them
i could not find a good example to follow, i tried this:
http://tutorialzine.com/2012/09/simple-note-taking-app-ajax/
but he lost me at the index php and how to implement that, i also tried this example:
http://miamicoder.com/2011/building-a-jquery-mobile-application-part-1/
but its so complex i got lost in all the code.
I'm a beginner at programming and my JavaScript is very very poor and i have no idea where to start and what kind of code is involved in making this part of the app.
Please help. Thank you.
Your first site (http://tutorialzine.com/2012/09/simple-note-taking-app-ajax/) will not work for you as the examples are embedding PHP into the html and that is not acceptable with phonegap apps.
Your second site is making it way more complicated than it needs to be for your limited scope.
Not going to do this for you but I will give you some advice and places to look.
Draw out what you want the user interface to look like and create that html
Create a load and save function in javascript to... well... load and save your notes. Take a look at the below references:
This will show you how to interact with the local database and store and retrieve info. Always a good thing to read the documentation.
This is a great resource for help all sorts of stuff. In this case, with SQL queries.
Test, test, test. With the above, you will be able to figure it out if you put your mind to it. Come back here and post specific questions with code samples of what you have tried if you get stuck.
I'm trying to create a simple google gadget and am encountering trouble right out the door. I've read the getting started guide top to bottom and feel like i'm still missing something.
Starting with the Hello World gadget they have provided, how do I begin editing and testing? I'm using the Google Gadget Editor with the preview tab. This works for simple html, but any javascript (such as an "alert") I embed doesn't work.
I can publish the gadget and add it to iGoogle. When I do this, my embeded javascript works. However there seems to be a good bit of caching and latency going on behind the scenes. Right now this process of editing, uploading, publishing, debugging is quite painful. Am I missing something obvious here? What is the "correct" way to develop and test a gadget? Is there a program that can run gadget code on my machine without having to go online?
Ah, the joys of editing a Google Gadget.
From my experience the Google Gadget editor is VERY buggy. So that may be the Javascript issue you are having, try switching browsers.
Also, to disable caching:
http://www.google.com/ig/directory?type=gadgets&url=www.google.com/ig/modules/developer.xml
The answer appears to be "No" as far as I can tell. I have also suffered the painful latency you mention: one code/debug cycle can easily take 10 minutes of waiting for the gadget to update to my latest code.
I hope someone marks me down and provides a "Yes" answer.
Try appending a query string to your gadget URL and then add it to your gmail or iGoogle page. Use a different query parameter every time you make changes to your gadget. Like for example:
www.mycompany.com/gadgets/main.xml?q=1234
Google refreshes the cache every hour or so. So, adding a query string will force it to refresh the gadget immediately.