It seems like this is a solved problem in older frameworks (Django, Rails), but I can't for the life of me find a solution in Express.
A super common pattern in one-page webapps is to use template data to create html and then echo the same data as JSON to the client so that it can maintain state.
each comment as comments
div= comment
script.
var comments = !{JSON.stringify(comments)}
Obviously this isn't safe because a user could easily create a comment that closes the script tag and performs all kinds of nastiness. What's the proper way to deal with this then?
I've seen people claim you can get by with just
JSON.stringify(comments).replace(/<\//g, '<\/')
but that seems naive especially when working with large, forgetful teams.
Similarly, I wrote a function that html escapes recursively before stringifying, but replacing " with " in every string seems like overkill and bad for data binding.
EDIT
For reference, here's Django's solution https://docs.djangoproject.com/en/dev/ref/templates/builtins/#escapejs
If I understand you correctly, you're asking how one might sanitize user input to prevent content injection attacks, XSS, etc.
There are at least three existing Express middleware packages you can take a look at for this sort of thing. express-validator has some sanitization features. It in turn uses node-validator. The current version of node-validator does not do XSS sanitization, so see express-sanitizer below.
The much-beloved helmet middleware has some XSS protection stuff that might meet some or all of your needs. If you are writing an Express app and at all concerned about security, you should definitely checkout helmet if you don't already know about it.
There is an Express middleware modulecalled express-sanitizer. It appears to be recent and only have one contributor, so check the code to see if it meets your needs and seems mature. But it is trying to do XSS sanitization now that node-validator does not do that anymore. (See express-validator above.)
Related
Let's say we have some proprietary web components which were designed and developed for a specific company needs.
Can we safeguard our web components from being used by others? If yes, how?
Note: I am not talking about other developers modifying the component, I am only thinking about others using them straight away in the first place.
It is just JS. All you can do is delay hackers.
The most you can do is use tools like JSObfuscator and JSFuck to make your code as unreadable as possible. But ofcourse experienced hackers have tools also...
Then it dawned me; it is all about adding extra hurdles, and there is one more hurdle we can call to action.
I wrote a nerdy DEV.to blog post about using the URI to encode your Web Components
Basically boils down to NOT putting the Dictionary in the file itself, like Obfuscators do;
but placing the encoding dictionary in the URI (Domain Path even better!):
<script src="element.js?-customElements-define-HTMLElement-"></script>
And executing:
let D = document.currentScript.src.split`-`;
// D = ["element.js?","customElements","define","HTMLElement"];
window[D[1]][D[2]]("my-element",class extends window[D[3]]{ ... });
I am getting some html from my server that I want to put into my page. However I want it to be sanitized (just in case).
However I am not quite sure how to do this.
So far I've tried:
<div .innerHTML="${body}"></div>
Since that should parse it as HTML but I am not 100% sure that this is the best way.
I have also looked at online sanitizers but haven't been able to find any that match my project (Lit-element web component).
Is there a better way to parse HTML and if so how?
Take a look at the DOMParser interface API
document.getElementById('my-target').append(new DOMParser().parseFromString(data, 'text/html').body.children);
It's not clear whether you want to render the html as html or as text.
I seem to remember that lit-html does some things behind the scenes to produce secure templates but surprisingly I cannot find official content to back up that statement. Others have asked about this before.
In that GitHub issue we can see that Mike Samuel mentions that what you're doing is not secure.You can trust Mike Samuel: he's worked in the security field at Google and I had the privilege to listen to one of his talks on the topic two years ago.
Here's a quick example:
<p .innerHTML=${'<button onclick="alert(42)">click</button>'}></p>
This renders a button which produces an alert when you click on it. In this example the JavaScript code is harmless but it's not hard to imagine something way more dangerous.
However this simply renders the code as string. The content is somehow escaped and therefore totally harmless.
<p>${'<button onclick="alert(42)">click</button>'}></p>
In fact similar to React's dangerouslySetInnerHTML attribute you need to "opt out" from secure templating via lit-html unsafeHTML directive:
Renders the argument as HTML, rather than text.
Note, this is unsafe to use with any user-provided input that hasn't been
sanitized or escaped, as it may lead to cross-site-scripting vulnerabilities.
<p>${unsafeHTML('<button onclick="alert(42)">click</button>')}></p>
About DOMParser#parseFromString
In this introductory article about trusted-types we can see that this method is a known XSS sink.
Sure it won't execute <script> blocks but it won't sanitise the string for you. You are still at risk of XSS here:
<p .innerHTML="${(new DOMParser()).parseFromString('<button onclick="alert(42)">click</button>','text/html').body.innerHTML}"></p>
A question regarding ng-bind-html whilst upgrading an Angular app from 1.0.8 to 1.2.8:
I have locale strings stored in files named en_GB.json, fr_FR.json, etc. So far, I have allowed the use of HTML within the locale strings to allow the team writing the localized content to apply basic styling or adding inline anchor tags. This would result in the following example JSON:
{
"changesLater": "<strong>Don't forget</strong> that you can always make changes later."
"errorEmailExists": "That email address already exists, please sign in to continue."
}
When using these strings with ng-bind-html="myStr", I understand that I now need to use $sce.trustAsHtml(myStr). I could even write a filter as suggested in this StackOverflow answer which would result in using ng-bind-html="myStr | unsafe".
Questions:
By doing something like this, is my app now insecure? And if so, how might an attacker exploit this?
I can understand potential exploits if the source of the displayed HTML string was a user (ie. blog post-style comments that will be displayed to other users), but would my app really be at risk if I'm only displaying HTML from a JSON file hosted on the same domain?
Is there any other way I should be looking to achieve the marking-up of externally loaded content strings in an angular app?
You are not making your app any less secure. You were already inserting HTML in your page with the old method of ng-bind-html-unsafe. You are still doing the same thing, except now you have to explicitly trust the source of the HTML rather than just specifying that part of your template can output raw HTML. Requiring the use of $sce makes it harder to accidentally accept raw HTML from an untrusted source - in the old method where you only declared the trust in the template, bad input might make its way into your model in ways you didn't think of.
If the content comes from your domain, or a domain you control, then you're safe - at least as safe as you can be. If someone is somehow able to highjack the payload of a response from your own domain, then your security is already all manner of screwed. Note, however, you should definitely not ever call $sce.trustAsHtml on content that comes from a domain that isn't yours.
Apart from maintainability concerns, I don't see anything wrong with the way you're doing it. Having a ton of HTML live in a JSON file is maybe not ideal, but as long as the markup is reasonably semantic and not too dense, I think it's fine. If the markup becomes significantly more complex, I'd consider splitting it into separate angular template files or directives as needed, rather than trying to manage a bunch of markup wrapped in JSON strings.
I am doing an application with durandal, breeze, and knockout. I have started to implement some test. The first problem that I have had is to decide what I should test and what not. I know that I should test everything, but it is not always possible in a little company.
My second problem is how I can test my calls to the server. I have seen some information in the breeze page about testing. Also I have seen the DocCode example. But I would like to know more opinions about how I can do that.
My questions are:
What do I should test in the breeze calls?
I would like to test this, emulating the backend. Is it possible? Any example?
Any advice or comment would be great
Wow ... that's a big question!
There's is a little on this subject in the documentation. Not nearly enough to be sure.
I'm guessing that you are fairly new to JavaScript testing. If you've seen DocCode, you know that we use QUnit around here. Many prefer Jasmine, Mocha or something else; I can only speak to QUnit.
First step is learn QUnit. It's not hard. QUnit's own intro is good. I like Ben Alhman's slide deck.
Next I'd practice with small tests of your business logic that does NOT go over the wire. Could be any interesting logic in a ViewModel or perhaps some calculated property in a model (entity) object.
You can test a VM's interaction with a "DataContext" quite easily without going over the wire. Create a "FakeDataContext" and inject that into your tests instead of the real one. Alternatively you could "monkey patch" the real "DataContext" in strategic places that turn it into a fake.
When faking the DataContext, I find it useful to leverage Breeze's ability to confine queries to the local cache. The local cache serves as an in-memory surrogate for data that would otherwise have been retrieved from the server.
This can be as simple as setting the FetchStrategy of the manager's default QueryOptions ... perhaps like this
var queryOptions = new QueryOptions({
mergeStrategy: MergeStrategy.PreserveChanges,
fetchStrategy: FetchStrategy.FromCache
});
var entityManager = new EntityManager({
serviceName: "yourEndpoint",
queryOptions: queryOptions
});
Now your queries will all be directed to the cache (unless they have an explicit QueryStrategy of their own).
Now make it useful by populating the cache with test data. There are numerous examples of faked entities in DocCode. Here's an example of a faked Customer:
var testCustomer = manager.createEntity('Customer', {
// test values
CustomerID: testCustomerID,
CompanyName: testCustomerName,
...
}, breeze.EntityState.Unchanged); // as if fetched from the database
If I need the same test data repeatedly, I write a "Data Mother" that populates an EntityManager with test data for me.
I can do a lot of testing this way without ever hitting the server. The whole time I'm working with the Breeze entities in JavaScript ... much as I do in production code. I don't have to learn a mocking library or inject another tool.
Another approach - harder, lower level, but more powerful - is to replace the Breeze default AJAX adapter with a fake that returns test JSON values as if they had come from the server. You can use a tool like Fiddler to make fake JSON from actual payload snapshots. You can also use this trick for simulating server-side save behavior.
Update 3 May 2013
The DocCode Sample includes a new TestAjaxAdapter for simulating server responses as I just described. Check out the testAjaxAdapter.js and see how to use it in testAjaxAdapterTests.js. This particular version of DocCode is only on GitHub right now but it will be published officially in the release immediately after v.1.3.2.
... end of update; back to original post ...
Does faking the JSON stream within your fake AJAX adapter seem like too much of a PITA?
Break out your mad Breeze skills and write a custom JsonResultsAdapter to create those fakes. Have your fake AJAX adapter return an empty array for every query request. Then you can do anything you want in the extractData and visitNode methods of your JsonResultsAdapter.
I trust it is obvious that you can fake your server-side controller too. Of course your tests will still make trips "over the wire" to reach that controller.
Hope these clues are sufficient to get you headed in a satisfactory direction.
Update 30 April 2013
Breeze will need metadata to do its thing. Your metadata comes from the server. Calling the server for metadata would seem to defeat the purpose of running tests entirely disconnected.
So true. Now I'm not a stickler on this point. I don't mind hitting the server for metadata at the top of a test module ... exactly once ... and then running the rest of my tests without going to the server for metadata. But if you're a purist or you just don't want to do that, you can write the server-side metadata to a JavaScript file on the server and load that script statically on your test runner's HTML page just like any other script.
For an example of this technique, look at App_Data/WriteMetadataScriptFiles.cs which generates a JavaScript file for the Northwind model in the forthcoming (later this week) v.1.3.2 DocCode sample. The DocCode tests load JavaScript files dynamically with require.js. The metadataTests.js test file shows how to load the generated northwindMetadata.js with require. You don't have to be that clever if you're not using require.js.
Note to self: write some samples illustrating these techniques.
https://urbantastic-blog.tumblr.com/post/81336210/tech-tuesday-the-fiddly-bits/amp
Heath from Urbantastic writes about his HTML generation system:
All the HTML in Urbantastic is completely static. All dynamic data is sent via AJAX in JSON format and then combined with the HTML using Javascript. Put another way, the server software for Urbantastic produces and consumes JSON exclusively. HTML, CSS, Javascript, and images are all sent via a different service (a vanilla Nginx server).
I think this is an interesting model as it separates presentation from data physically. I am not an expert in architecture but it seems like there would be a jump in efficiency and stability.
However, the following concerns me:
[subjective] Clojure is extremely powerful; Javascript is not. Writing all the content generation on a language created for another goals will create some pain (imagine writing Javascript-type code in CSS). Unless he has a macro-system for generating Javascript, Heath is probably up to constant switching between JavaScript and Clojure. He'll also have a lot of JS code; probably a lot more than Clojure. That might not be good in terms of power, rapid development, succinctness and all the things we are looking at when switching to LISP-based langauges.
[performance] I am not sure on this but rendering everything on user's machine might lag.
[accessibility] If you have JS disabled you can't use site at all.
[accessibility#2] i suspect that a lot of dynamic data filling with JavaScript will create cross-browser issues.
Can anyone comment? I'd be interested in reading your opinions on this type of architecture.
References:
Link to discussion on HN.
Link to discussion on /r/programming.
"All the HTML in Urbantastic is completely static. All dynamic data is sent via AJAX in JSON format and then combined with the HTML using Javascript."
I think that's the standard model of an RIA. The emphasis word seems to be 'All' here. Cause in many websites a lot of the dynamic content is still not obtained through Ajax, only key features are.
I don't think the rendering issues would be a major bottleneck if you don't have a huge webpage with a lot of elements.
JS accessibility is indeed a problem. But then, users who want to experience AJAX must have JS enabled. Have you done a survey on how many of YOUR users don't have it enabled?
The advantage is, you can serve 99% (by weight) of the content through CDN (like Akamai) or even put it on external storage (eg. S3). Serving only the JSON it's almost impossible for a site to get slashdoted.
When AJAX began to hit it big, late 2005 I wrote a client-side template engine and basically turned my blogger template into a fully fledged AJAX experience.
The thing is, that template stuff, it was really easy to implement and it eliminated a lot of the grunt work.
Here's how it's was done.
<div id="blogger-post-template">
<h1><span id="blogger-post-header"/></h1>
<p><span id="blogger-post-body"/><p>
<div>
And then in JavaScript:
var response = // <- AJAX response
var container = document.getElementById("blogger-post-template");
if (!template) { // template context
template = container.cloneNode(true); // deep clone
}
// clear container
while(container.firstChild)
container.removeChild(template.firstChild);
container.appendChild(instantiate(template, response));
The instantiate function makes a deep clone of the template then searches the clone for identifiers to replace with data found in the response. The end result is a populated DOM tree which was originally defined in HTML. If I had more than one result I just looped through the above code.