I am attempting to scrape details from two archived webpages, here and here.
As you can see, the content lives in a paginated table (10 records are shown out of 30+) that I believe is being controlled with javascript - if you hover over the right arrows at the bottom of the table, you can see they are advanced through a doPostBack call. Unfortunately, these arrows are non-functional in the archived pages themselves, as is the drop-down that allows you to change the number of visible rows in the table.
Does anyone know how to either (a) get the javascript working to advance the table correctly or (b) gather the information some other way from this webpage? I've attempted to look at the request/response objects as well, but I'm quite a novice with web scraping so there is a good chance I missed something.
Any direction (or being told its a fool's errand!) is helpful, thanks in advance to whoever takes a look!
Related
I'm trying to make a table that is two columns. The one of the left will have many rows and a scroll bar. The one on the right will be a big box that I'll have buttons in that will change when you click on a different block on the left. Is this even possible?
Offcourse this is possible, almost everything is possible. I made some assumptions; the box on the right is some sort of detailed view of the record on the left? Ans because the question is tagged as javascript it should be a ajax call or something.
It is not so much if this is possible in Rails, on the back-end it would be a simple model with a number of records. On the left you'll have a list something like; Item.all.select( :id, :name ) which could be available from /items.
On the right the route /items/{id} should be defined to retrieve one item Item.find_by_id( {id} ). If you want it to be responsive you can respond with json so that you can parse it to html using javascript. Backwards compatibility would be that if you'll go to /items/{id} with your browser the detailed version should already be loaded.
I suggest you'll take a look at http://guides.rubyonrails.org/getting_started.html. Ruby only does the back-end, it serves pages. It does nothing with styling or responsive design.
For the styling and scrolling on the left side of the page, I suggest you'll use a list or just div's, a table is not very easy to style I'm not sure what you're styling skills are. But it has nothing to do with Ruby or Rails. Perhaps you can use bootstrap or some other html/css framework.
For the responsiveness you can use JQuery.get. For the basic idea, I suggest looking into http://www.w3schools.com/jquery/jquery_ajax_intro.asp, it is far from perfect but it will give you the idea.
I am in a specific situation that I could not find an answer to by searching here. I am a .net developer just getting my feet wet with asp/html/css/javascript. I haven't done too much actual training with them, I am simply trying things out with a template called "Flat" to guide me.
I work for a medical billing company and was tasked with creating a few web pages for productivity for our billing departments. These web pages are displayed on 60" TV's throughout the building so that employees can look up and see how they are doing. The pages not only contain a good amount of data, but it also must be displayed so that someone can see it from their desk. Within reason of course.
I have thrown something together and it is functionally fine, but it is a little dull. I simply do not know all of the interesting tools out there to make a site more flashy, or rather the tools I have found don't work too well within the confines of the rules, in that all of the data needs to be onscreen at once and it must be highly visible.
Here is a screenshot of what I have worked out, as the site isn't public. Click on the screenshot to enlarge it.
This just barely fits the TV when the browser is full screen.
Anyone have any tips or ideas to make this look better? Maybe something subtle that makes all the difference in a page like this?
You could give a slight demarcation between each cell in the table. Similar to he vertical gutter between the columns. This is because horizontally viewed, one might tend to get confused due to the amount of data in each cell.
Provide a 4-5px space between each vertically adjoining cell
Hope this was useful
This is my first time asking questions here and I have just started learning jQuery mobile so if it is not quite relevant or if it is just really obvious I apologize in advance.
What I am trying to do is build a mobile site for an online catalog for lots of books.
The original website has a very sophisticated way of filtering and sorting the search results based on the user's input and preference so unless it's absolutely necessary, I would really like to keep the original functionality of the website.
I was thinking maybe I can create a "mask" that looks like a mobile site to cover the real content; and whatever the user requests, I will go "underneath" the mask, do the search and when the data is loaded in the original website, I will then present them in a more mobile-friendly way.
So here is my question:
Is it possible to somehow include/embed a mobile site in the original html?
If so, could you please give me a few hints (books, articles, tutorials, anything) as to how I can approach this?
If not, could you please tell me how I should make the mobile site without having to re-create all the nice functionality that original website already has?
Thank you so much! :)
You could do this but you shouldn't. Mobile websites are optimized for mobile users. Not only as layout but also in loading weight. The direction you are heading you end up with a regular side (probably big in load) and add an overlay (extra load) to rebuild the layout (CPU heavy). So it's not the right way. A better solution is to create a optimized layout for mobile users only.
I'm not sure if javascript is the best solution but here's what i want to achieve.
The web page is divided in 3 columns...
The left column should have links in picture format like Arts, History, Science etc
Clicking on any of these link should generate a series of links in the rightmost column say clicking on history should generate links for Civilization, World war etc Science should generate links like Physics chemistry represented by pictures etc and so on
Finally Clicking on these links say World war should display the content in the central column giving a brief history on World war....
Also each time a user clicks on an image link say Science it should get highlighted so the user can navigate easily from one topic to another.
I'm a novice to programming, not sure if javascript is the best approach. Also I'm very confused as to how to go about this problem where to begin and how. Any help in terms of code chunks or references or algorithms or advice on how to approach this problem would be most appreciated. If solving this problem means learning a new programming language i'm open to try.....
In case if i'm not clear in explaining the problem let me know
JavaScript would be appropriate for this. The appropriate approach will depend on how much content you have.
If it's reasonably short, you can achieve all of this by simply changing the display property on your elements (use JavaScript to set the CSS of certain page elements to show (display:block) or to hide (display:none).
If you have pages of text, you may want to look into Ajax. It is a bit more advanced that simply showing and hiding elements that are already there.
I would like to display to user a table which contains ~500,000 rows.
The table should be calculated based on a file (i.e. the table is known at the beginning, and does not change).
I guess that building the HTML text of this table is not a good idea in terms of memory performance.
So how can I build such table ?
I would like user to be able to scroll the table using a vertical scroll bar. Is it possible to build on the fly only the visible part of the table ? I'm afraid to see delays because of this.
Is it a better idea to use server side programming rather than Javascript ?
I'm not restricted to any server side programming language, so any advise would be appreciated !
Send the first 250-ish rows to the user, as he scrolls down, perhaps past row 200, fetch the next 250 rows, append to the table and so on.
This is a (ui) design pattern known as "Infinite scroll".
Displaying 500,000 rows all at once to a user is a bad idea. Consider other options:
allow user to download file as CSV
show paginated (still not very useful)
provide filtering mechanisms (by date etc.) to allow the user to only see the data they need
If the users really needs to see all that data at once, then viewing it in the browser is one of the worst ways to do that - they should be using a tool made specifically for viewing data, like Tableau.
Classic example of where to use ajax.
Use javascript in combination with a server side script.
This really is a case for server-side pagination, I would say, maybe in combination with Ajax. A HTML table with 500.000 rows is going to crash a lot of browsers.
You're not specifying which server side technology you work with. If you update your question, I'm sure people will be able to give you some pointers.
Try this:
Clusterize.js
Tiny plugin to display large data sets easily
https://clusterize.js.org/
Is it possible to build on the fly only the visible part of the table ?
If you build a "fake" scroll (e.g. jquery slider) you can retrieve parts of the table instead of the whole.