I am working on device applications, that we will deploy some minimum device specific code to the device, and this component will dynamically load the rest of resources (javascript, images, etc) from our backend server. That way we can upgrade the app more often without depending too much on the device upgrade. In a way, this is similar to some iphone developers who want to avoid the app approval cycle with faster upgrades.
The code in device should be able to detect new upgrades on the server code, but only to a point, when we need to upgrade the device code as well, so we can support phased roll-out (upgrade 10% of devices at a time, etc).
I am sure some folks have already done this. Could you share your thinking and best practices on this?
One way would be for the application to have a polling process that asks a server if there is a new version of a particular resource.
The server then has complete control over who gets the upgrades and who doesn't (part of the "phone-home" could include the current version), and that would allow you to enforce the "only X% of clients get upgraded" capabilities. You may also want to include the ability for things to get "downgraded" so that you can roll back if you see too many errors in the first 10%.
This is essentially what web browsers do when they have a resource in the cache but it may have been expired. They say "Hey, Server, I have this resource that I got from you at XYZ date. If it's changed, give me a new one, if not, let me know." and the server either sends back a "Hasn't Changed" response, or a "Here's the new one" response.
Related
Recently, I switched the server for my site, and I managed to lose the decrypted SSL key, and I cannot remember the password for the encrypted one.
It turned out that the server had set HSTS on, and now many visitors are unable to load the pages since I don't have a valid SSL cert, and their browsers refuse to connect via http due to the HSTS.
So, I need a way to disable that HSTS from their browsers. Asking them to clear browsing data is a no-go, but I was wondering if I could make a firefox/chrome compatible javascript to clear it. (The script would be on a different domain)
I've been digging around a bit, but haven't found much info on how I should approach the problem, if it is even possible. All other suggestions are welcome too.
HSTS is there to make a tradeoff: you take responsibility to from now and forever provide a secure SSL connection which the browser can count on, which will cause the browser to refuse anything but an SSL connection to your domain. It puts an additional burden on you, but increases security for your visitors.
The browser stores this preference in an internal database which cannot be cleared by any website. If it'd be possible for any site to simply revoke this preference via Javascript, the whole system would be pointless.
You'll have to manually clear the database and/or remove that specific entry. Every browser does it differently, see http://classically.me/blogs/how-clear-hsts-settings-major-browsers for an overview.
The real solution:
Install a valid cert
Get people to visit your site
Send a new header
These days, getting a valid cert is free, or costs less than a sandwich ($8 or so).
I have a web application that listens for Server Sent Events. While I was working and testing with multiple windows open, things were not working and I banged my head for several times looking in the wrong direction: eventually, I realized that the problem was concurrent connections.
However I was testing a very limited number and even if I am running the test on Apache (I know, I should use node).
I then, switched browser and noticed something really interesting: apparently Chrome limits Server Sent Events connections to 4-5, while Opera doesn't. Firefox, on the other hand, after 4-5 simultaneous connections, refuses to load any other page.
What is the reason behind this? Does the limit only apply to SSE connections from the same source, or would it be the same if I were to test open them from a different domain? Is there any chance that I am misusing SSE and this is actually blocking the browsers, or this is a known behaviour? Is there any way around it?
The way this works in all browsers are that each domain gets a limited amount of connections and the limits are global for your whole application. That means if you have one connection open for realtime communication you have one less for loading images, CSS and other pages. On top of that you don't get new connections for new tabs or windows, all of them needs to share the same amount of connections. This is very frustrating but there are good reasons for limiting the connections. A few years back, this limit was 2 in all browsers (based on the rules in (http://www.ietf.org/rfc/rfc2616.txt) HTTP1.1 spec) but now most browsers use 4-10 connections in general. Mobile browsers on the other hand still needs to limit the amount of connections for battery saving purposes.
These tricks are available:
Use more host names. By assigning ex. www1.example.com, www2.example.com you get new connections for each host name. This trick works in all browsers. Don't forget to change the cookie domain to include the whole domain (example.com, not www.example.com)
Use web sockets. Web sockets are not limited by these restrictions and more importantly they are not competing with the rest of your websites content.
Reuse the same connection when you open new tabs/windows. If you have gathered all realtime communication logic to an object call Hub you can recall that object on all opened windows like this:
window.hub = window.opener ? window.opener.hub || new Hub()
4. or use flash - not quite the best advice these days but it might still be an option if websockets aren't an option.
5. Remember to add a few seconds of time between each SSE request to let queued requests to be cleared before starting a new one. Also add a little more waiting time for each second the user is inactive, that way you can concentrate your server resources on those users that are active. Also add a random number of delay to avoid the Thundering Herd Problem
Another thing to remember when using a multithreaded and blocking language such as Java or C# you risk using resources in your long polling request that are needed for the rest of your application. For example in C# each request locks the Session object which means that the whole application is unresponsive during the time a SSE request is active.
NodeJs is great for these things for many reasons as you have already figured out and if you were using NodeJS you would have used socket.io or engine.io that takes care of all these problems for you by using websockets, flashsockets and XHR-polling and also because it is non blocking and single threaded which means it will consume very little resources on the server when it is waiting for things to send. A C# application consumes one thread per waiting request which takes at least 2MB of memory just for the thread.
One way to get around this issue is to shut down the connections on all the hidden tabs, and reconnect when the user visits a hidden tab.
I'm working with an application that uniquely identifies users which allowed me to implement this simple work-around:
When users connect to sse, store their identifier, along with a timestamp of when their tab loaded. If you are not currently identifying users in your app, consider using sessions & cookies.
When a new tab opens and connects to sse, in your server-side code, send a message to all other connections associated with that identifier (that do not have the current timestamp) telling the front-end to close down the EventSource. The front-end handler would look something like this:
myEventSourceObject.addEventListener('close', () => {
myEventSourceObject.close();
myEventSourceObject = null;
});
Use the javascript page visibility api to check to see if an old tab is visible again, and re-connect that tab to the sse if it is.
document.addEventListener('visibilitychange', () => {
if (!document.hidden && myEventSourceObject === null) {
// reconnect your eventsource here
}
});
If you set up your server code like step 2 describes, on re-connect, the server-side code will remove all the other connections to the sse. Hence, you can click between your tabs and the EventSource for each tab will only be connected when you are viewing the page.
Note that the page visibility api isn't available on some legacy browsers:
https://caniuse.com/#feat=pagevisibility
2022 Update
This problem has been fixed in HTTP/2.
According to mozilla docs:-
When not used over HTTP/2, SSE suffers from a limitation to the maximum number of open connections, which can be especially painful when opening multiple tabs, as the limit is per browser and is set to a very low number (6).
The issue has been marked as "Won't fix" in Chrome and Firefox.
This limit is per browser + domain, which means that you can open 6 SSE connections across all of the tabs to www.1.example and another 6 SSE connections to www.2.example (per Stackoverflow).
When using HTTP/2, the maximum number of simultaneous HTTP streams is negotiated between the server and the client (defaults to 100).
Spring Boot 2.1+ ships by default with Tomcat 9.0.x which supports HTTP/2 out of the box when using JDK 9 or later.
If you are using any other backend, please enable http/2 to fix this issue.
You are right about the number of simultaneous connections.
You can check this list for max values: http://www.browserscope.org/?category=network
And unfortunately, I never found any work around, except multiplexing and/or using different hostnames.
I wish to tailor a particular website to a low bandwidth version if the client is connected via WWAN (metered) connection or otherwise.
The connection type is important for the site to know if it should provide a rich experience or a bandwidth efficient experience. My desktop runs on a metered connection and in the future, more desktops will be connected via metered cellular networks (including Windows 8 tablet PCs), efficient web-apps should respect that a user might not require high detail assets on a pay-per-byte-connection.
How do I check if the client is connected via WiFi/LAN/WWAN using HTML5/JavaScript?
NOTE : I do not wish to explicitly check for browser headers (which is not really a solution for desktop browsers that can connect to the internet via multiple methods).
See navigator.connection (prefixed). This API exposes information about the bandwidth of the client.
You can't. The information on how a client is connected (i.e., which technologies it uses, if it is pay-per-byte or a flatrate) to a server is not public. You might be able to get a trace route (with all problems that are connected to tracing), if that helps you, but that information won't be accessible JavaScript.
What can do with JS is simple bandwith testing. Download a file of known size via XHR, and measure the time that needs.
The pay model of a connection is absolute private data. Neither will it be sent along to servers on the internet, nor is it available to a loaded website. If you need this information to offer the client a custom app, ask the user directly. He will tell you if he wants.
Yet, wait. Latest Chrome and Firefox can be set (user preference) to provide that data on the experimental navigator.connection object.
Also, for developing Metro apps, Windows offeres such information on the Windows.Networking.Connectivity API, see this tutorial.
Maybe you are approaching it in the wrong way. Have a look at the Gmail model. They have a rich client that the user can opt out of if the page is taking a long time to load. The standard client is much lighter and uses a more "traditional" web design.
Trying to automatically detect things that are not meant to be can take you down a very deep rabbit hole.
Old question, but in searching for my own purposes, I found this -- I have yet to try it and so therefore YMMV: https://github.com/ashanbh/detectClientSpeed.
The underlying notion of measuring time to download an asset to derive bandwidth information is subjective and not necessarily consistent, but short of native platform API for iOS/Android, this is not a bad option. A lighter alternative may be to measure ping times to something like google.com.
I need to load a couple thousand records of user data (user contacts in a contact-management system, to be precise) from a REST service and run a seach on them. Unfortunately, the REST service doesn't offer a search which meets my needs, so I'm reduced to just loading a bunch of data and searching through it myself. Loading the records is time-consuming, so I only want to do it once for each user.
Obviously this data needs to be cached. Unfortunately, server-side caching is not an option. My client runs apps on multiple servers, and there's no way to predict which server a given request will land on.
So, the next option is to cache this data on the browser side and run searches on it there. For a user with thousands of contacts, this could mean caching several megs of data. What problems might I run in to storing several megs of javascript data in browser memory?
Storing several megs of Javascript data should cause no problems. Memory leaks will. Think about how much RAM modern computers have - a few megabytes is a molecule in the drop in the proverbial bucket.
Be careful when doing anything client side if you intend your users to use mobile devices. While desktops won't have an issue, Mobile Safari will stop working at (I believe) 10Mb of JavaScript data. (See this article for more info on Mobile Safari). Other mobile browsers are likely to have similar memory restrictions. Figure out the minimal set of info that you can return to allow the user to perform the search, and then lazy load richer records from the REST API as you need them.
As an alternative, proxy the REST Service in question, and create your own search on a server that you then control. You could do this with pretty quickly and easily with Python + Django + XML Models. No doubt there are equally simple ways to do this with whatever your preferred dev language is. (In re-reading, I see that you can't do server-side caching which may make this point moot).
You can manage tens of thousands of records safely in the browser. I'm running search & sorting benchmarks with jOrder (http://github.com/danstocker/jorder) on such datasets with no problem.
I would look at a distributed server side cache. If you keep the data in the browser, as system grows you will have to increase the browser cache lifetime to keep traffic down.
I have to program websites, but I rather don't like the static HTML nature. I prefer more of a client-server architecture.
Now I've figured, that with XMLhttp, you can basically dynamically update your page and send/request for information/action to/from a server. So this would basically cover the client area.
But to complete a client-server architecture, it is necessary for the server to send/request information, too, without being queried.
Is there any way, for example for a chat server, to send back a received message to all clients (the clients use a web browser) without that the clients have to query in a fixed interval? I want to implement that one can see while you type something in.
There are several different ways to accomplish this. Some of them are already answered here, but I wanted to include a few more as well as my thoughts on them.
1. Polling
Frequent requests are made to the server to check for new info. This is the worst way to do this, but probably the easiest. If your site will have a low number of users, it might be worth doing it this way.
This can be accomplished by using setInterval(myFunction, n) in javascript to send XMLHttpRequests to the server every n milliseconds. Then, on the server, you respond to this with your new info, when you have it, or some message that implies no new info.
2. Long Polling
When the page is loaded, it makes a request to the server for new info. The server holds the connection open until there is something to send back. This method reduces the amount of network traffic used, but increases the resources used on the server. You can use this for a small number of users, but it doesn't scale very well.
The easiest way to do this is to have the page that handles the AJAX request simply wait for new information to be available, then respond. This can tie up a lot connections on your server. So, use with care.
3. COMET
COMET is basically long polling, but the server is setup properly for it. It knows that these connections aren't "real" and it uses less resources to handle them. This is a great solution for this problem, but it requires that the server is explicitly setup for this purpose. There are COMET servers and COMET addins for other popular servers, but it will require some setup and sometimes some money.
Implementing this on .NET isn't the easiest thing in the world. You can pay for solutions, try to find someone else's code that does something similar, or try to write it yourself. I've not found any decent free solutions for this. If someone else has, please comment.
4. RIA
Another solution would be to include Flash, Silverlight, or Java Applet on your page. People often do this by using a 1x1 object so that they can use Flash or Silverlight to talk to the server. If you don't mind adding the dependency, this is a decent solution. If you already know Silverlight or Flash, it could be relatively simple to implement.
You can find tutorials on the internet for each of these options.
5. Web Sockets
If you are on the cutting edge, you can look into Web Sockets. It's only available in the latest builds of modern browsers. It was part of HTML5, but it might be its own spec now. Regardless, it means that older browsers won't be able to handle it. But, if you don't mind limiting yourself to the latest of browsers, you can use this amazing feature.
I believe that Chromium is the only browser that currently supports it. However, there is work being done to implement this in Firefox and WebKit.
I'll spare you the controversy and simply say that this does exactly what you want it to. The Abstract of the spec says it all.
This specification defines an API that enables Web pages to use the Web Sockets protocol for two-way communication with a remote host.
Special Mention
If you are interested in the world of Node JS, you can't go wrong with Socket IO. It will implement the best of whichever technology is available to the browser.
Conclusion
The best option is Socket.IO on Node JS. However, for an ASP.Net solution, go for COMET or Web Sockets, if you can. Otherwise, using Flash/Silverlight isn't terrible. Finally, polling and long polling should be last resorts. You could always support one of these, then fall back to another if there isn't support for it in the client's browser.
Yes, you can use COMET.
The client has to tell the server when the client-user begins typing. You've got a couple options here.
Frequent requests from the server for the latest activity. This would be taking place for each user involved in the chat. The same request could be used to send user-specific activity to the server as well: "Jonathan is typing..."
Long-polling. This essentially requests information from the server, and the server keeps the connection opened until it has something to send back. So your requests are minimized, but your connections stay opened longer.
Depending on your traffic, type of data being transmitted, server-environment, and many other factors, one of these options may shine more than the other.
You can use Silverlight for push notifications. Look at PollingDuplexHttpBinding. Since you are using ASP.Net MVC, adding Silverlight will be easy.
Look at this page for more information.
Based upon the REST architecture the html system is based upon, the servers role is to simply act as a resource for the client to pull from. I am generalizing but there are tools to implement this type of action on the client side, rather than on the server side.
You are better off writing/using a library that can request updates from the server periodically. You can encapsulate these types of requests in a javascript object that can fire events. This way your client side script can act like it's getting notified from the server. Review some common stuff with COMET you can probably find some tools to help you client side code.
HTML 5 has some tentative attempts at this type of functionality, but if you want your app to work on older browsers, your better off using more stable methods, like AJAX requested updates.