Develop Reference

loading a web page for a fake query string

I don't even know how to phrase the title of this question, but hopefully the following description will explain my issue.
I have a web application that is made up of a single, bare search page with a search field. The search is actually performed by the client browser and results are loaded via ajax. In other words, the server does nothing but serve up the bare search page at http://server/index.html
Once the query is performed, I use history.pushState() to change the URI in the browser address bar to something more sensible like http://server/index.html?q=searchterm&page=1&size=10. Pagination is performed by prev and next links that too are called via ajax along with the appropriately incremented or decremented page and size values. All is good.
But, I want my application to be a good web citizen, and be bookmark-able. In other words, if someone enters http://server/index.html?q=searchterm&page=1&size=10 directly in the browser address bar, I want to load the results correctly. Except, if I send that URI to the server, the serve will croak unless I implement some server-side processing. And, that is something I don't want to do as that will change the complexity of my application completely. Unless I can do that with plain, vanilla nginx (my web server). In other words, I don't want to implement any server side scripting other than what can be done with the web server itself, such as SSI.
So, how do I solve this problem?

hi the exact term for what you are trying to do is "Client side routing". It involves a combination of manipulating the browsers history using history.pushState() [which you are already doing] and server side config setting
.htaccess if you are using apache
config file if you are using nginx.
The server side settings will make your web server your base index.html for whatever request the browser makes(http://server/index.html?q=searchterm&page=1&size=10) once loaded in the client you have to get the query string in the window address bar and handle accordingly(make an ajax request).
This implementation has implications when search engines crawl your site using the URL but that is not within the scope of this question.
this SO question will give you a start

actually, I think this is a lot easier than I thought. When I send the browser to http://server/index.html?q=searchterm&page=1&size=10, it doesn't complain. It simply sends back http://server/index.html. Then it is just a matter for me to use js to extract the query string and do my ajax bit. This should work.

Differenciate Between User Requests and AJAX/Resource Requests

I'm attempting to create an app with Node.js (using http.createServer()) which will be a single page application with requests for data via XMLHttpRequest. To do this I need to be able to differentiate between a user navigating to my domain, and AJAX requests and requests generated by the browser for linked resources.
If the request is from the user I always want to return the index.html page which will handle requesting content but if the request is browser generated or AJAX and is for CSS, Javascript or other linked files I want to serve those files. Is there any way to detect this?
Looking at the request headers for the different file types I saw the referer header appeared when the request for content was generated by the page. I figured that was the solution I was looking for but that header is also set when a user clicks on a link to the page making it useless.
The only other thing which seems to change is the accept header which could sort of work but might not be a catch all solution. Any user requests always seem to have text/html as the preferred return type regardless of which url was entered. I could detect that but I'm pretty sure AJAX requests for html files would also have that accept header which would cause problems.
Is there anything I'm missing here (any headers or properties I can look for)?
Edit: I do not need the solution to protect files and I don't care about users bypassing it with their own requests. My intention is not to hide files or make them secure, but rather to keep any data that is requested within the scope of the app.
For example, if a user navigates to http://example.com/images/someimage.jpg they are instead shown the index.html file which can then show the image in a richer context and include all of the links and functionality to go with it.
TL/DR: I need to detect when someone is trying to access the app to then serve them the index page and have that send them the content they want. I also need to detect when the browser has requested resources (JS, CSS, HTML, images, etc) needed by the app to be able to actually return the resource not the index file.

In terms of HTTP protocol there are NO difference between a user-generated-query and a browser-generated-query.
Every query is just... a query.
You can make a query with a command line, with a browser, you can click a link, send some ascii text via telnet, request a proxy which will make the query for you, the server goal is never to identify how the query was requested by the user.
See for example a request made by a user on a reverse proxy cache, this query will never reach your server (response comes from the cache), the first query made to build this response could have been made by a real user or by a browser.
In terms of security trying to control that the user is never requesting data by-himself cannot be done by detecting that the query is a real human click (and search google for clickjacking if you want to be afraid). Every query that a browser can make can also be played by the user, every one, you have no way to prevent that.
Some browsers plugins are even doing pre-fetching, detecting links on the page and making the request before you do it yourself (if it's a GET query).
For ajax, some libraries like JQuery will add an X-Requested-With: XMLHttpRequest header, and this is used on most framework to detect ajax mode.
But it is more robust to depend on a location policy for that (like making your ajax queries with a /format/ajax, which could also be used on other ways (like /format/json, /format/html, or /format/csv).
Spending time on a location policy based routing is certainly more usefull.
But one thing can make a difference, POST queries are not indempotent, it means the browser cannot make a POST query without a real user interaction, because a POST query may alter the state of the session or the state of the server data (but js can make POST queries, this is just a default behavior of browsers). The browser will never automatically retrieve a POST query, so you could make a website where all users interactions are POST queries (via forms or via some js altering link clicks to send POST ajax queries instead). But I'm not that's your real goal.

Not technically an answer to the question but I found a simple solution which does what I want: prefix all app based requests with a subdomain eg. http://data.example.com/. It's then really simple to check the host header for that subdomain: if present send the resource else send the index page.

how to prevent querystring values from being visible

We have an application which is destined for https so the data being transmitted is protected (or should I say as protected as need be) I would like to know about protecting/hiding (or similar) queryString values. Its a jQuery/Javascript front end which communicates using a mixture of GET and POST with the database via classic ASP web services. The web services sends JSON back to the client.
I realize the front end code could be changed so everything is passed using POST however the application is finished and tested ready to deploy. There are some key values that are being passed in the querystring which should not have been. Is it possible to make it so the querystring values can not be inspected or sniffed. The URL and querystring together will provide a direct link to the raw JSON. We would like to prevent this. Perhaps there is some jQuery/AJAX feature which can be explored. Perhaps some server IIS level tactic? I guess the sniffing occur before the request gets the the server where the webservice sites therefore some server/IIS level tactic is not an option.
Any ideas/advice would be great, thank you.

You can use HTTP headers to send data to the server that is slightly less visible, but can still be detected using more advanced developer tools and loggers. For example, this answer descibes using jQuery/Javascript (as you've asked) to send data without using QueryString.
You can't really prevent the client from being able to trace these details though.
The solution I personally suggest to you is to look into session state. By scoping a valid data response to a certain session state, and returning null when the state is invalid or expired, you can limit access to the data. This could be after just 1 time its been retrieved. This strategy would involve a generation of a token or code that is passed out from your server at an earlier stage, and used when asking for the data in question.

Another alternative is to either use SSL or encrypt your data and drop it into a posted control such as a text input box. Microsoft adopted a similar process for their VIEWSTATE within ASP.NET.

Can POST request be an alternative to GET request in most of the scenarios?

I wet thru http://www.w3schools.com/tags/ref_httpmethods.asp and wondered why should i not always prefer
POST request over GET request. I can think of two scnarios where i have to use POST request instead of GET
Request. These are:-
1)Where i have requirement to bookmark the the URL
2)Where my requirement is to cache the web page (as POST request does
not cache the web page), so that next time same url is hit it can be
obtained from cache and optimize the performance.
I agree POST request is designed to create/update the resource where GET request is designed
to retrieve the resource.Though,techincally they can be used vice versa also.
So i was wondering
is it not always benefecial to use POST request over GET request(except the two requirements i mentioned
above) as Post is more secure? Is my understanding correct?

There are many reasons to use HTTP the way it was meant to be used. Here's a couple:
The value of the web is built on URLs. Every time you provide a page which is obtainable only via POST, you are denying the option to link to it, as well as to bookmark it. (Obviously a form button can still be made, but that's not as convenient.) Even if the page is some kind of “service”, there is still often value in linking — that you won't have thought of beforehand.
If the user reloads a page obtained via POST, most web browsers will warn that they are “resubmitting a form” and confirm the action. This is because in poorly designed applications this can result in things like placing a duplicate order or posting a duplicate message. Therefore, using GET for requests which do not have side effects eliminates this unnecessary warning. In fact, a useful practice for POSTs which have effects is to make the response to them be a redirect to a URL (which the browser will GET) for a page describing the results of the action (for example, if the POST posted a comment, it would then redirect to a link to the comment); this way the page can be reloaded (which could be implicit e.g. if the browser were restarted) without any ambiguity about whether it's re-executing the action.

Upload contents of current html page from a bookmarklet

What would be a good way to upload the html content of the current page viewed in the browser to another server from a bookmarklet?
Assuming this url is on a server that requires authentication, so I want to avoid fetching the page on the sever side, but rather would like to see if it's possible to get the contents and upload them directly from within the browser.
Thanks in advance for any suggestions
Elisha

Considering that you are most probably going to have a situation in which the page being viewed in the browser is on a different domain from the domain you want to send the data to, an AJAX request will definitely fail (due to Cross-Domain restrictions). So doing this server side would be your best bet.

Retrieve location.href with XHR into string
Create FORM with desired cross-site action
POST data to server
?????
PROFIT!