Email processing with NodeJS - javascript

I'm writing a Node application which among other things needs to receive email and process it so that it can be rendered in a Web page (as happens in Web mail, list archives, etc.). I've got the receiving part covered with Haraka (https://github.com/baudehlo/Haraka). From there to storing the received emails in Mongo is easy, and so long as they're plain text I can display them.
The part I'm missing is handling the rather involved varieties of ways in which email content can come, including alternatives, HTML, attachments, inlined images, and much more. That's a rather steep amount of functionality to have to put together, and I can't seem to find a JS library that will do it for me. Given the number of libraries out there and the speed at which new ones are added, it might just be that I've missed it — so pointers welcome!
And failing that, if I were to port an existing library for this over from another language, which one(s) would you recommend I look at?
Thanks!

You should check out haraka-couchdb and the other work maxogden is doing.
https://github.com/maxogden/haraka-couchdb
https://github.com/maxogden/couchmail
Haraka-CouchDB deals with storing the attachments. At the very least it should tell you how to do it with Mongo too.
At some point I'm hoping to put a simpler layer over the email parser in Haraka, so that you can just see the email contents easily - but that requires a javascript HTML to Text converter. Or I guess you could do it the other way around and convert the Text to HTML.

Im probably a little late responding, But I had a similar requirement for persisting emails in mongo-db. I created this stub of code that gets the body of emails and inserts into mongo, Next iteration is attachments.
The plugin can be found at https://github.com/jamescowie/haraka-mongo

Check out Kyatchi -> https://github.com/hamin/kyatchi
hope that helps

Related

Javascript library to manage translation forms

Is anybody aware of any javascript tool (compatible with jQuery, tinymce or any other clientside library) able to manage the following requirements?
I need to show translation forms in which every field (either input or textarea) could contain some segment variables or code sections (mostly HTML).
For example:
"Hello {{firstname}}, this is your personal page."
or
"You improved your personal score of <strong>{{n}} points</strong>."
Of course I obtain these segments from a template parser and I need to show them to a set of translators that will perform localization towards many languages. I know that in many cases I can (and should!) avoid variables and code inside translation segments, but in many other cases I really can't.
The problem is: I would like to manage coherence about variables and code directly on the browser (I trust my translators but a bit more of UI/UX help is always a good thing!).
A nice approach could be providing the set of variables and code tags, ready to be inserted by means of a single click (in order to avoid mispelled variables or incorrect code syntax) and a bit of pre-submit validation to be sure everything was inserted.
I've seen this approach in other websites, such as Facebook or Freelancer.com (who have the power and the ability to reimplement the whole thing from scratch!).
Do you know about any almost-ready tool/library for this purpose?
Thank you all in advance for any suggestion.
If you are asking for a library to translate text - here is Google Translate API: https://developers.google.com/translate/?csw=1
If you are asking for a library which can take user input, perform validation, and insert into the DOM - then Jquery has everything you need.
If you are asking for something else, let me know and I'll edit my question.

How do I escape SQL query's single quotes for query generated in javascript/jquery in cartodb database?

I am using javascript/jquery to generate a sql query.
I have a sql query I'm generating and using inside a javascript/jquery script.
Something like this:
var storeName;
var query = "SELECT * FROM stores where storeName = '" + storeName + "';";
(storeName is generated through jquery when a user selects from html)
So when storeName is something like "Jackson Deli" the query runs just fine.
But then when storeName is "Jackson's Deli" it does not work and it seems to be because the apostrophe in Jackson's is treated like a closing quote. I know I can escape a quote by doubling it if I was hard-coding the query... so
SELECT * FROM stores where storeName = 'Jackson''s Deli';
should work. But I'm not hard-coding the query. Instead it's being generated by user input and may or may not have an apostrophe in the name. How would I go about escaping ' this character in this case? I would need it to work inside Javascript/jquery.
Would I need to write an if statement that looks for ' in storeName and replaces it with '' ??
Or is there another way to go about this?
EDIT:
Ouch! Normally, yes, I realize the perils of generating a query on the client side.
So here's some more context. I'm working with cartodb and following their documentation. Here's an example from their repo doing something similar to what I'm talking about (they have other examples too):
https://github.com/CartoDB/cartodb.js/blob/develop/examples/layer_selector.html
You can't run a query in cartodb that lets you modify data in any way -- you can only run queries that let you retrieve data. So I'm still thinking about what the best way to escape this quote character would be.
DO NOT GENERATE SQL ON THE CLIENT SIDE... EVER
That being said, if you are going to use a dynamic query, you are best off escaping the user input and binding it to a prepared statement on the server side.
If you post more details about which database (MySQL, Postgres, etc.) and what language you are using for server processing- you will get better answers.
Yes... I am fully aware this doesn't answer the question. Nobody should be creating code this way though.
Edit: Made the warning bigger for emphasis.
I see others have answered but I wanted to approach this question from a few angles.
The question you're asking is a good one. You recognize that the SQL doesn't work with single quotes. You realize that something needs to be escaped. These are a good starting point for a few considerations that will hopefully help you to architect software in a secure and maintainable way.
Never directly execute client code/content - Generating SQL or any kind of code/instructions (javascript, bytecode, compiled code) from a client is always a poor idea because it breaks a few critical concepts.
It's hard to maintain because you cannot control the input fully. Sure you could escape the SQL but that doesn't fix both strange case scenarios where you have other characters you didn't account for.
It isn't secure - Your relationship to variables, inputs, CGI params, file contents, database fields whose values came from the aforementioned list, or just about anything that came from a remote system, remote user cannot ever be trusted. Always check, sanitize and validate inputs. I can open the source to your page, see where you add a check for single quotes and change that and then execute the code to delete your records, have it email if certain stored procedures are available, run code on the SQL backend, drop databases (assuming the query runs under appropriate privileges.)
It blends/blurs the lines between client input/display and business logic. Research MVC, n-Tier development and other concepts for an introduction to the concepts of separating your business logic from display/inputs. This is critical not only for scalability and performance but also to reduce the change of issues such as this from causing critical security flaws.
Approach your software development from the bad-guys perspective - Instead of "How can I escape this string to make it work." try "How can I bypass the escape on this page to allow me to delete records, view things I should, etc.
Don't feel bad because the approach is wrong,learn from it. I see alot of comments about how you should never ever do this (and they're right) but many of us learned this lesson the hard way. We laugh at Little Bobby Tables because we've all written or had to support code that did this. The key is to understand the underpinning of why it's a bad idea and then use that in designing software. Welcome to the school of hard knocks. We're all graduates and thankfully you could learn from our comments rather than when somebody tinkers and corrupts, deletes or infiltrates your database and application.
To get you started on this journey may I suggest reading the following:
SQL Injections Explained
And as an added bonus XSS E.g. escaping OUTPUT that originated from an external system or person. for example a comment entry that contains Hi!!! <script>alert('Thanks to this site not escaping this output I get to run this code under your login. Thanks for the 4000 crates of free tshirts you just ordered for me');</script> how are you??? so that when you output it you get
Comments:Hi!!! <script>alert('Thanks to this site not escaping this output I get to run this code under your login. Thanks for the 4000 crates of free tshirts you just ordered for me');</script> how are you???
Which is "valid" HTML and the browser will execute it.
Final thoughts - Adopt the motto Trust but Verify and you'll be OK
FYI, CartoDB does not allow you to execute a query that changes something in the table, it's read-only.
Send data to your server first, then escape all chars that need to be escaped with addslashes() command (provided that you are using PHP).
addslashes() command on PHP
After you are done with eascaping characters, you can send your data to cartoDB using their API and your API key.
cartoDB does provide insert/update/delete tasks through its SQL API. See this link:
http://developers.cartodb.com/documentation/sql-api.html

Is it a bad idea to store javascript and css in a database?

Ok so I'm trying to store user custom css and javascript in mysql database to be used later the head section on the page, so is it a good idea to store css and javascript in a database ? and if not what is the safe way to do this ?
I'm using wordpress and using esc_js() on javascript code I noticed it adds a backslash before quotes, and adds \n instead of new lines, so is it enough for javascript ?
and thanks in advance.
Short answer: Yes, it's fine as long as it's people you trust.
Long answer:
Make sure you 100% trust your users. There's a lot of exploits that can be done through CSS/JS, so make sure you're only allowing people you completely trust to do this. Even if you tried to implement filtering of some kind, people would find ways around.
Depending on how easily you could figure out the file name, I might consider storing the data in a file. After all, the database is unnecessary overhead if you can readily determine the file name. If you can't determine the file name easily, I just go ahead and store it in the database. (Or if you're planning on storing any meta data or version information or anything, I would go ahead and go for a DB.)

XML File Parse in javascript how to. Large File maybe use SAX?

G'day All,
I am pulling my hair out, getting headaches and my eyes hurt. I have been hither and thither and I seem to get whither.
This will be my first experience with xml and would really want to get this working. It is a large file. Well large in my eyes +-5mb. I can not imagine that this file would be loaded into memory to process. Users will get a bit peeved with this.
Basically we are using a 3rd parties site to do our ecommerce. So we have no access to the database other than via the admin area.
What we want to do is make sure that there is no stuff ups when it comes to addresses. Therefore we got this xml file put together listing all postcodes with areas and states:
<?xml version="1.0"?>
<POSTCODES>
<PostCode id="2035">
<Area>2035 1</Area>
<Area>2035 2</Area>
<Area>2035 3</Area>
<State>NSW</State>
</Postcode>
<PostCode id="2038">
<Area>2038 1</Area>
<Area>2038 2</Area>
<Area>2038 3</Area>
<State>NSW</State>
</Postcode>
<PostCode id="2111">
<Area>2111 1</Area>
<Area>2111 2</Area>
<Area>2111 3</Area>
<State>NSW</State>
</Postcode>
</POSTCODES>
Someone suggested SAX but suddenly died when asked how? The web is not helping unless I am not looking properly. I see a lot of examples. Either they do not show how to read the file but rather do it from a textarea or the example is in java.
What do we want? User enters a post code of 2038. We want to go to the javascript with that data and have returned to us all the suburbs that full within that post code.
Anyone out there that can please tell me what to download and how to use it to get what i need?
Please, please, please. It is hard to see a grown man begging and crying but I am.
Sounds like you want a script on the server which will suggest suburbs based on the users postcode selection? You could use jQuery's ajax functionality to do this.
You might also be able to use jQueryUI's autocomplete control to parse XML and make suggestions: http://jqueryui.com/demos/autocomplete/#xml
It's also possible to do this entirely in javascript without any script on the server side, but it would be pretty slow at loading if the XML file is 5MB. You might be able to get a significant reduction in file size thought by gzipping it before transmission from the server.
If you need to parse this in Javascript, you can use jQuery.
http://www.switchonthecode.com/tutorials/xml-parsing-with-jquery

Is there any alternative to obfuscation to make it harder to get any string in javascript?

I use DropBox and I've had some trouble reaching to my files from other computers:
I not always want to login to anything when I'm in a public computer, but I like being able to reach my stuff from wherever I am.
So I've made a simple application that when put in the public folder, ran and given the right UID, creates (still in your public folder) an HTML of all the content in the folder (including subfolders) as a tree of links.
But I didn't risk loading it anywhere, since there are slightly private things in there (yes, I know that the folder's name is "PUBLIC").
So I've came up with the idea to make it a simple login page, given the right password, the rest of the page should load. brilliant!, but how?
If I did this by redirecting to other HTML on the same folder, I'd still put the html link in the web history and the "url's accessed" history of the administrator. So I should generate itin the same page.
I've done it:
alt text http://dl.dropbox.com/u/3045472/validate.png
And currently the page is a textbox and a button, and only if you type in the right password (defined in the generator) the rest of the page (with the link-tree) loads. The fault is that everything (password, URL's) is easily reachable through the source code.
Now, assuming I only want to avoid silly people to get it all too easily, not make a bulletproof all-content-holding NSA certified website, I though about some ways to make these information a bit harder to get.
As you may have already figured, I use a streamwritter to write an html file (head, loop through links, bottom), then it's extremely configurable, and I can come up with a pretty messy-but-working c# code, though my javascript knowledge is not that good.
Public links in DropBox look like this:
Summarizing: How do I hide the URL's ande the password to show them (MAINLY the password, of course) in my source-code so that no that it should require some effort on reading ?
P.S.: It's not that personal, if someone REALLY wants it, it could never be 100% protected, and if it was that important, I wouldnt put it in the public folder, also, if the dude really wants to get it that hard, he should deserve it.
P.S. 2.: "Use the ultra-3000'tron obfuscator!!11" is not a real answer, since my javascript is GENERATED by my c# program.
P.S. 3.: I don't want other solutions as "use a serverside application and host it somewhere to redirect and bla bla" or "compress the links in a .RAR file and put a password in it" since I'm doing this ALSO to learn, and I want the thrill of it =)
Update 1:
The one answer so far gives a perfect way (according to this question) to hide my password.
Now I want a good way to hide the URL's, maby a code snippet of the example URL I gave being composed, and if it's too tricky, maby how to generate it in C#, or anything ?
Update 2:
I thought about maybe making three "obfuscating methods" and choosing them randomly in the runtime. So anyone who figures out how to read one XML, could only read about one third of them, and maybe having a hard time finding the other rest of this third..
Update 3:
Just thought about REGEX, the URL could be neatly crowded by dummy not-url-allowed characters added randomly that would be removed by something like:
regex.replace(url, ^[^\w\d/:-\.%]+$,"")
So the nosy dude should have to be pretty advanced into programming somehow, eh? could anyone tell me if it would work or not ?
Well, as it seems you already know, this is a rather poor choice of security mechanism, but if you insist...
Don't store the actual string in the source. Store, for example, its MD5 hash. Then, when the user types in a password, compute its MD5 hash and compare it with the expected one.
Check out:
MD5 in JavaScript
MD5 in C#
To elaborate on miorel's idea, you can also encrypt the whole page, using password as a key. Basically, encode all content into one big string, ask for the password and decrypt that string. If the password is wrong, it will show loads of rubbish, that is it. Like
content = "encrypted string"
function decrypt(str, key) { your algorithm of choice here }
document.write(decrypt(content, prompt('Password?')))
The only thing you need is a decrypt implementation in javascript - but that's easy to google out, for example here or here.
This also renders the separate 'login' page useless.
Granted, this is akin to asking how you can strip in public without people seeing you, but given that, I'm assuming that the password you are trying to store is the one to DropBox. I suppose you could obfuscate the password and store it in a cookie. That would at least prevent someone from simply viewing the source to see the password, but obviously wouldn't stop someone running something like Fiddler and seeing it.
[snipped server side suggestion]
EDIT: To munge the Urls, why don't you simply build the urls on the fly and have the links call a javascript function to get the url? Your server-side code would populate an array in this function with obfuscated urls and the calling code would simply pass an index into the array. Thus, on viewing the source, there would be no instances of "http" anywhere other than static unsecure links.
ADDITION Ok. now that I have a better bead on the problem, it is easier to devise solution. There are libraries for doing encryption on the net in javascript (e.g. http://point-at-infinity.org/jsaes/) but the problem comes down to key management. Since its javascript, it is going to be public but there are hoops you can devise to make it harder to determine the key. In general, those tricks involve indirection. For example, store a lengthy stream of random characters (e.g. 40-50 or more) that is generated by your C# code and stored in the HTM file. In addition, the C# code would would store into your javascript function an array numeric values that represent pointers into the long stream of text that were used by the C# code to encrypt the passwords (or just the whole url).

Categories

Resources