This is the case.
I'm reverse-engineering a chat. The client runs in a browser, in Javascript.
Nothing is documented, but by checking the HTTP requests/responses (using Burp Suite), I've been able to figure out everything.
For simplicity, let's say that "everything" means loading the last messages and also sending a message.
However, the chat client detects "events". E.g., when a new message arrives. And I'm pretty sure the program is not polling any endpoint for this.
So, there should be an existing active connection receiving these events.
I am sure it's not WebSockets. So I need to look elsewhere, and checking the Javascript code (which is fragmented, obfuscated and minified) seems to be a HUGE task without knowing what to look for.
I've used Wireshark / mitmproxy for clues but could not get anything more useful that what I get with Burp.
What's the cheapest route to find out where is this program connecting to? I'm not much versed in Javascript so I don't know what are the ways of connecting to a given host and receiving events.
Any hints?
Thank you very much!
As weird as it may sound, the events were hidden behind a default filter.
They come as text/event-stream, which is strangely recognized as CSS by Burp.
Related
Sometimes it could be useful for a web application to be able to determine whether a seeming slowness in the application is due to network conditions or something else.
To make it easier to determine the cause in this kind of situation, I was thinking about creating something like a "check the connectivity" client-side script makes e.g. an AJAX call to the server and returns some useful metrics that make it easier to determine if the cause lies with the network.
But how much useful network troubleshooting could be done using a JavaScript that calls the server?
It sounds like you need a way to keep an eye on your site's performance from the end-user's perspective.
One way to do this is to have your client-side scripts include a way to log to a log aggregation site like SumoLogic. Here is a doc to reference about using client-side JavaScript to log to SumoLogic.
On your server side, you could implement a /ping API endpoint that would just immediately return true so you know how long it takes your user to at least reach your site. You can then log to SumoLogic how long that request took. You could do this with other requests as well to see which APIs are slower than others.
If you include geo-location when logging to SumoLogic, you can see how well your site performs around the world.
And if you want to get really fancy, then you should implement a custom header that your APIs understand which is a transaction token of some sort for all requests. When your server receives that header, it should use that token throughout the request's logs so you can see where things go wrong and what to do about them.
Another good site to check out for this sort of thing is New Relic - Browser Monitoring. This is much more performance-centric and you don't get the insights of your own logs, but it's an awesome app in its own right.
EDIT
As mentioned in the comments by #Bergi, you could also have your server respond with the headers immediately and measure performance that way.
I have a SignalR chat site that's meant for a school project (also uses C#). Theoretically, it is for trusted users, but as everyone will attest - never trust your users. This was proven to me as I sent out the link to a couple of my friends and they immediately tried to break it, ha ha.
I've sanitized all inputs properly now, but one thing that they were still able to do was to use the browser console tools to manually call the functions needed to send messages, etc..
Example: $.connection.chatHub.server.sendMessageToAll('FakeUser','FakeMsg',0);
I would like to prevent these types of actions. I recall a while back Facebook actually disabled the console window for "security" purposes. I even found several{1} resources{2}, which detail how this was done and attempts to further prevent console use once Chrome had fixed this.
However, none of these options work anymore and because browsers are constantly in flux, I'd rather not attempt to block at this level.
I was wondering if anyone on Stack knows of a better way to prevent these types of attacks? Is there a good way to check where the call is coming from? Does SignalR have a good method to prevent this? Ideas/Discussion would be surely welcome.
Trying to lock down the client like that might work reasonably well to prevent non-technical users from messing with your app, but it will do next to nothing against a knowledgeable and resourceful opponent. The circumstances under which such security measures make sense are rather limited, and certainly do not include any application that is accessible to everyone from the internet.
The only safe approach is well-known and very simple: the server does not trust the client for anything. It doesn't then matter what the client attempts to do as the server will refuse all actions it does not deem valid.
In your example, the server would assign a randomized opaque connection id to each session. The client would only be able to convince the server to do anything if they sent a valid id as part of their request; then, the server would not need to trust the client for a username because it would already know what connection each user has logged in from and could produce the username when given the id.
I've been attempting to do some research on this topic for a while, and even cite the following Stack Overflow threads :
Javascript Hijacking - When and How Much Should I Worry
JSON Security Best Practices
But my basic problem is this.
When I am building my web applications, I use tools like Fiddler, Chrome Developer Tools, Firebug, etc. I change things on the fly to test things. I can even seem to use fiddler to change the data that gets sent to the server.
What stops someone else from just opening up my webpage and doing this too? All of the jQuery validation in the world is useless if a user can just hit F12 and open up Chrome Developer tools, and change the data being sent over the wire, right?
I'm still relatively new in this field and this just has me very concerned as I see "Open" Protocols become more and more ubiquitous. I don't understand SSL yet (which is on my list of things to begin researching), so perhaps that is the answer and I just haven't dug deep enough. But the level of flexibility I have over manipulating my pages seems very extreme - which has me very concerned about what someone malicious could do.
Your concerns are indeed justified. This is why you should always validate everything on the server. Client-side validation should only be used for UX.
JavaScript's security is, in a nutshell, based around a trusted server. If you always trust what code the server sends you, it should be safe. It's impossible for a third party (like an ad supplier) to fetch data from the domain it's included on.
If the server also sends you user generated content, and in particular user generated code, then you have a potential security problem. This is what XSS attacks focus on (running a malicious script in a trusted environment).
Client side validation should focus on easy of use, make it easy to correct mistakes or guide the user so no mistakes are made. The server should always do validation, but validation of a more strict nature.
Validation should always happen Server Side, Client Side Validation is only valuable to make for a more convenient experience for the user. You can never trust a user to not manipulate the data on their end. (Javascript is ClientSide)
Next if you are wanting to secure your service so that only user1 can edit user1's profile you'll need to sign you JSON request with OAuth (or similar protocol).
yeah nothing can stop anybody from interfering the data that is being sent from the browser to your server and that's the reason you shouldn't trust it
always check the data from the user for authenticity and validity
also with it you can check and interfere with the data that big sites like google and microsoft send back and you might get an idea.
You have to assume that the client is malicious-- using SSL does not prevent this at all. All data validation and authorization checking needs to be done server side.
Javascript isn't going to be you only line of defense against hackers, in fact it shouldn't be used for security at all. Client side code can be used to verify form input so that users trying to use the page can have faster response times, and the page runs nice. Anyone who is trying to hack your page isn't going to care if your page works or not. No matter what, everything coming into your server should be verified and never assumed as safe.
I'm trying to automate the process of getting my current student records at my college. In a browser the process involves typing in my college's URL, then clicking on the login link which then brings me to a https:// URLed page were I type my password and user-name in. Then from there it is one or two more links and reading some text on the page. Now, my question is, how might I go about do doing this but in an automated way, so my records would be displayed on the command line. The https:// in the URL signifies, I think, that it uses SSL are there certain libraries that can handle this? Also the 'submit' button on the login page I'm pretty sure uses JavaScript, again, are there libraries to handle this?
I'm sure I missed something or other in my question's description, so please ask if you do not understand my question or need more information.
PS. I am not well versed in Internet protocols and I am also new to Python. In fact I started studying it for this project. But, I am fluent in C and I am pretty good with C++.
Thanks in advance.
Michael,
You don't have to mimic all the actions you do in the browser.
First. There is no problem with https/ssl as long as you don't have to verify them (it seems that you don't have to), urllib2.urlopen will handle them.
Second. When you click 'Submit' browser sends a request to the server with your username, password and probably some other data. The type of that request is probably POST. As a response server will probably send you a cookie with session id. So all you need to do is to investigate the exact format of request to the server (e.g. using FireBug), and get the cookie from the server's response.
Third. Just use that cookie to navigate the pages on the site. This might help.
P.S. As you see, there is too much 'probably' word in the answer - the exact authentication process may differ from described above and you'd have to investigate it by yourself.
Roman's answer is good advice: you generally don't need to act like a real user when your script can call HTTP methods directly.
However, if you are not comfortable with reverse engineering the HTTP operations that the site requires, then an alternative would be to use Selenium, a tool for simulating interaction with web pages. Selenium is usually used by web application developers to test their applications, but it can also be used as an automatable client for an existing website.
I'm trying to reverse engineer how facebook handles their notifications, where when you get a message you get instantly notified via the browser.
I've fiddled with it for a little bit and realized that there is always a pending GET request "listening" if you will to some sort of update from the server. This appears to be some sort of observer pattern. I was just wondering if this pattern was documented somewhere.
The technique is actually called Long Polling. This is one of the popular Comet techniques to get around the limitations of traditional polling.
You may want to check out the following Stack Overflow post for a very simple example:
Simple “Long Polling” example code?
UPDATE:
In addition to the above, I suggest that you check out the accepted answer to the following Stack Overflow post for a detailed description of the technique:
How does facebook, gmail send the real time notification?
The technique is called Comet, aka 'server push'
There are currently 2 main ways of implementing comet.
1) As Daniel mentioned, long-polling, where you can use ajax to leave a hanging request to the browser that doesn't send the response back until the server decides to (whether it be based on someone else's actions or another server event).
2) The second approach, used by Google, is streaming. This involvs using ajax to leave a hanging request, but the response is never sent back to you, ever. Instead, the server updates bits of data and you use javascript to monitor changes, and fire events based on new data being pushed in. What happens is you get one very long continuous stream of data flowing in on a document that never closes, taking new data as it comes in.
HTML5 has a specification for a simpler way to do this with Web-Sockets. In the future, this type of live web-app will become commonplace as Web-Sockets are easy to use, but it is not supported on all browsers yet.
If you want to build a Comet site for production, you'll need to use a non-blocking I/O async server like one of the following.
http://www.tornadoweb.org/ - python
http://nodejs.org/ - server side javascript
-- or google for comet servers.
You'll need to know how to program for comet type apps on the server-side, as the javascript for Comet is pretty trivial, just your normal ajax calls with a couple event handlers.