This question's answers are a community effort. Edit existing answers to improve this post. It is not currently accepting new answers or interactions.
I was wondering how one would go about developing a website from scratch with Node.js.
I understand how I could possibly do it, but I am interested in the best design practice.
I need this theoretical website to:
Do a lot of AJAX
Be very straightforward
Be relatively small
Connect to... let's say a MySQL server
In PHP, building a pretty small website was very straightforward - I set up PHP on Apache and a MySQL server and then do something like:
includes/db/ which has connect.php for connecting to the db, a file with common db related functions and so on
includes/layout/ which had stuff like footer.php, header.php, and other layout related stuff
includes/users/ to handle user related actions
Then PHP just let you build pages and include these files together to form a website - I could go something like:
<?php
require_once('inclues/users/user_session.php');
require_once('inclues/db/connect.php');
require_once('inclues/design/header.php')
?>
// Other php or html or related content relating to the page
<?php
require_once('inclues/.../footer.php');
?>
I was wondering what might be similar in Node.js - I am looking for a way to accomplish this which is as simple, fast and straightforward as possible.
If the answer is not simple, I would love a book recommendation, I don't mind reading.
I love event based programming, I really love JavaScript's abilities and I'm really excited about Node.js. I want to learn how to develop this sort of stuff with it the right way from the start.
To start with the bad news: As Node.js is a pretty young technique, I think you'll find that the proces of creating a full fledged website and maintaining/operating it will be very different than what you're currently used to.
Josh3736 adds: Once you figure out how Node.js and its various packages (Connect, Express) work, I found that you can develop new sites very quickly.
The rough edges that currently exist in Node.js, combined with the fast pace of its development and all modules involved can complicate things though, and make things less simple, fast and straightforward than you'd like.
Having that out of the way, here's the good news:
The Node Package Manager, NPM has a lot of good tools and frameworks to expand Node.js's bare bones functionality, making it suitable to create a webserver.
Most notably would be the Express Framework which contains almost everything you need to run a webserver (including cookies, sessions and path routing). Additionally Express supports partials, which take care of your header and footer includes.
Express is built on top of Sencha's Connect. Cookies and sessions are actually powered by Connect. Express is what simplifies your routing and handles views/partials. So if you don't need all bells and whistles that come with Express you could just go for Connect instead.
If you like to use templates for these partials, the Jade Template Engine can speed things up for you. Though Josh3736 points out that Jade is slow and whitespace-significant. A more complete overview can be found here, which includes his favourite, doT. (I personally use Node.js for socket.io based applications only, so he's a better source than me when it comes to templating).
You can connect to MySQL from Node.js using the db-mysql module, but if you don't need that because you're accessing data connected to an already present system, I'd advise to use a more... 'modern' approach, which is to use a NoSQL database as most Node.js projects seem to do. MongoDB via Mongoose is the popular way to go.
Or if it's just storing objects you're interested in, just go for Redis instead (which you're probably going to need at some point anyway).
Once your website is complete, you'll have to deploy it and make sure it keeps running. There are many ways to do so, like using built-in cluster support or use the more feature-friendly forever npm module. See this SO question of mine for more information.
Conclusion:
What I'm trying to get at is this:
Asking what the best practice for building a website in Node.js is, is about the same as asking what the best way to build a website in PHP is: 100 developers will give you 100 different answers.
NPM is blessed with a variety of excellent frameworks that greatly simplify a lot of tasks involved, but it's all based on preference which one is the way to go really.
As I've said, Node.js is still a pretty young technique, so none of the frameworks or additional tools have emerged as 'defacto standard' yet; for most things you're trying to do there are probably various alternatives, and expect your code to break when using most of them during updates, because development of Node.js itself and most modules is fast paced. You'll have to keep up.
Putting it all together:
As I've said, my main production use for Node.js is to be able to use socket.io, so I don't have any good production examples present (And as I'm about to leave on a well-deserved vacation I don't have the time to put one together either). There are some good examples though:
Setup and deployment using Express and Jade
A very complete blog example using Express, Jade and MongoDB
Combining Restify (an extension of Express), Backbone.js and Mongoose
Again, the way to go (and subsequently the example to follow) depends greatly on your ultimate goals and the techniques chosen, but luckily there are plenty of resources available for all of the choices available. Most modules use well documented GitHub repositories and include examples in combination with the most popular modules (See the /examples/ dir that seems to be present in most repositories).
(thanks to Josh3736 for rectifying my errors.)
Related
I have an idea to create a browser-based game and I struggle to find the right tools to use. I'm a Python/Django developer right now, but I also would like to learn something new and try using node.js for backend. The thing is that there are a lot of frameworks around and I have no idea which one can be the most suitable for game development.
I have read a little bit about sails.js, hapi.js, total.js and others. Many of them sound nice and it is very hard to pick one.
A few of the requirements are:
Good performance
Easy use of web sockets
ORM with migration support (sails.js doesn't have migration support production ready)
Large community (a lot of packages for common scenarios etc., many questions already answered)
I hope somebody can help me to pick the right tool to use.
I'm currently working on a new google polymer web application and wondered if I should use firebase as the backend/db. I took a look at the project, made some test applications and really liked it! But to fully convince me, that firebase is the way to go I need the following questions answered:
I'm a little bit concerned about security: So, I know, that firebase uses read, write and validate to implement server side security. From the samples, I noticed that the validation basically is a one-line JS script, that represents a 'if'. As I'm planning to build a web e-commerce application I need to validate quite some inputs. Is there a possibility, to outsource the validation in a separate file, to make it more readable? Also I wondered, if there is a possibility, to test these server side validations, with for example unit tests?
I'm not 100% sure at the moment, that firebase can cover all of our use cases. Would it be possible/a good solution to use a "normal" backend for some critical functions and then persist the data from the backend in firebase?
I saw some nice polymer elements for firebase. Is firebase 100% supported in polymer/web components?
Is there an other way (like Java approach) to implement server business logic?
Is there a way, to define update scripts, so that new releases can easily be pushed to production?
Thanks & kind regards
Marc
So, I asked the firebase supprt and got the following answer:
Great to meet you.
I'm a little bit concerned about security: So, I know, that firebase uses read, write and validate to implement server side security. From the samples, I noticed that the validation basically is a one-line JS script, that represents a 'if'. As I'm planning to build a web e-commerce application I need to validate quite some inputs. Is there a possibility, to outsource the validation in a separate file, to make it more readable? Also I wondered, if there is a possibility, to test these server side validations, with for example unit tests?
You can implement extremely complex and effective rules using our security rules language. You can deploy security rules as part of your hosting deploy process, or via the REST API. It's not possible to break the contents into multiple files on the server, but you could certainly build your own process for merging multiple files into a single JSON result.
I'm not 100% sure at the moment, that firebase can cover all of our use cases. Would it be possible/a good solution to use a "normal" backend for some critical functions and then persist the data from the backend in firebase?
Generally speaking, synchronizing Firebase and a SQL back end is not very practical and they don't translate well. It's probably entirely redundant as well.
I saw some nice polymer elements for firebase. Is firebase 100% supported in polymer/web components?
I don't know what 100% supported means in this context. We offer a JavaScript SDK so they should play fine together.
Is there an other way (like Java approach) to implement server business logic?
We offer official SDKs in Java, Objective-C/Swift, Android, Node.js, JavaScript, and a REST API for use with other languages.
Is there a way, to define update scripts, so that new releases can easily be pushed to production?
I'm not sure what this means. Most likely the answer is no, since we don't provide a build process or any tools to release your software.
I hope that helps!
I responded:
Thank you for the information, it helped me very much! After reading your response on question number 5 one further question popped into my mind:
…
5. Is there a way, to define update scripts, so that new releases can easily be pushed to production?
I'm not sure what this means. Most likely the answer is no, since we don't provide a build process or any tools to release your software.
Is there like a best practice on how to handle the database schema? I only have one web application (without apps, etc.) in my case... I expect, that the database will change drastically over time and releases. Should I write JS logic, that checks the current database version and update it, if it's necessary? Maybe this would make a nice feature...
For example: I deployed Version 1.0 of my application and everything works fine. After 3 months of programming I notice, that the user data needs a further attribute: address, which is a 'not null' attribute. I now deploy Version 2.0 of my application and every new registered user has a address, but the old users (from Version 1.0) do not have this field or a value.
How should I handle this?
Support responded:
Hi Marc,
There’s no best practice here, but your ideas seem fairly sound. You probably don’t need to check in your JavaScript. You can probably store a version number in the user’s profiles, and when they upgrade to the latest software, you can upgrade that in their profile data.
Then your validation rules could use something like the following:
{
"user": {
".write": "newData.hasChild('address') || newData.child('appVersion') < 4",
"address": {
".validate": "newData.isString() && newData.val().length < 1000"
}
}
}
So if you are concerned about versioning, this could be used to deal with legacy releases.
Another popular approach I’ve seen from devs is to do intermediate upgrades by duplicating data. Thus, you release an intermediate version that writes to the old path and to the new path with the updated data structure (which keeps the app working for old users till they upgrade). Once a reasonable percent of clients are upgraded, then release a final version that no longer does a dual write to the old structure and newer structure.
Of course, flattening data, while it makes joining and fetching data bit more of a pain, will make upgrades much easier as the modular data structure adapts more easily to changes. And, naturally, a pragmatic design where you wrap the various records in a class (e.g. the UserProfile class with getter/setter methods) makes transitions simpler as you can easily hack in versioning at one place.
Hope this helps someone :)
I'm about to start writing an application and I want to include newer technologies to improve my knowledge and gain a skill set.
I'm using mysql to store user data (no choice), Angular.js to display/template content on the dom, and I wanted to use node.js to return results from the database in json format to be used by Angular.js.
My assumption was I could use node.js (along with the node-mysql module and maybe express)to accomplish my database queries that feed into my Angular.js templates.
Additionally, the application will live on a different server than the database itself and may reside in a mobile framework.
Am I way off base when it come to how to use node.js? Am I just using the wrong tool? Am I on the right track?
Any help, explanation, and points in the right direction would be great. Most of the info I've seen are copy/paste from the node.js site/wiki and don't really answer my question.
This is largely an opinion-based question and likely to be closed, but you're not way off base. What you want to do is absolutely (and commonly) done using Node.js.
As a complete aside, you might like to check out Platform-as-a-Service providers such as Heroku.com and Nodejitsu.com; both offer great and easy Node.js hosting and addons (such as NoSQL and RDBMS data stores) which have free tiers.
For that matter, MongoLab and MongoHQ both offer free tiers independent of Heroku, which you could use from your MySQL-only hosting provider if you like.
To preface; I'm an experienced web developer, though I have barely used nodejs, and have not yet had the opportunity to work with emberjs (though I've done a decent amount of work with backbone).
I'm about to start a project to build a web based writing application (literature). In theory, the user would potentially have the application open for long periods of time, perhaps leaving it on. I intend to develop an expanding functionality set over time, which I would like to be able to drop into different places with relative ease. For example, perhaps develop a character list which I could then include in many places. I'm keen on using emberjs for the experimentation alone, but it also sounds like a good client side framework for the job.
I've looked at django briefly (haven't used it much before), but given that I'd like to use emberjs, it seems like django isn't a great choice as I'd be opting not to use many of its most compelling features.
Would writing my server side code in nodejs + express be a reasonable choice? The web application itself will have common features like login, admin, different permissions for users/tasks (eventually) - but I also anticipate some more beefy stuff.
This is a solution that works well for my team and I so far. While lacking express, Charcoal is a good starter project to get up and running with Ember without a Rails asset pipeline and Express can easily be added as middleware. In addition to using Express to serve your JSON, it can handle the minimal HTML needed for such requirements that Ember can't fulfill by itself such as SEO since content negotiation capability is very simple in Express.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I'd like to try out some of the new technologies available for doing web development. I've been using php frameworks, rails and asp.net mvc for quite some time now and I saw that Node.Js provided a lot of good stuff to the table.
But I don't know how and where to begin. I setup mongodb, node.js and express.js and I played with it a bit but I'd like to know how to make the link between express.js and mongodb (I have mongoose but I don't know how to include it yet) and I'd like some brief and simple examples (unlike stuff that is in express.js docs) to see some actual usage cases.
I'd like to build some quick content management system and with a simple frontend, and I wanted to make a change and try that out. Any advice and help will be appreciated!
[edit: I already read http://howtonode.org/express-mongodb it helped but it doesn't explain much about how to do it]
Thanks!
Here's a get started blog post for Node.js + MongoDB. You can easily ignore the Cloudfoundry stuff. Mongoose is a layer over the basic driver, so you should probably start with the basic driver.
If you want some code / ideas to steal, take a look at railwayjs.
Start simple. Code a basic web site using vanilla node.js standard library HTTP module (require('http');). Learn the basics of the http request and response interfaces, especially the evented interfaces for streaming data. Learn the basic fs module and how to take plain .html files on disk and send them as HTTP responses. You can do this in a single .js file without needing any third party npm modules. Full stop until you have that done and it makes sense to you. If something isn't clear, experiment until it is (and ask more questions here). This is straightforward, but there are still lots of ways to get confused.
Once you have that, you'll see that it's actually already pretty high-level and straightforward to write a small web site. Next add express.js. Study the concept of connect middleware and how the middleware chain works, including the next callback, bodyParser, and static middlewares. Make sure you understand why middleware often needs to be applied in a specific order and how it breaks if you apply them in the wrong order. Understand express's app.error, dynamicHelpers, and most of the examples in the documentation, which are all very good and realistic but I agree because they are just tiny snippets they leave a lot of room for questions about the surrounding context and how things fit together entirely. Don't go nuts with complex layouts and templating stuff, just stick to the basics.
Then maybe start persisting some domain level data into simple files on the filesystem, perhaps just .json data. This will present new challenges and help you understand more complex things in an async world. In particular, if you start having to do a series of operations like recursively ensure a directory path exists and then write a new file in the final leaf directory, you will want to look at a flow control library like async or async.js or similar.
Only then should you start adding mongoose and mongodb to your stack and replace your fs.writeFile calls with mongoose model.save calls.
For me the biggest help I found when experimenting with all these awesome new technologies was looking at this sample application:
Demo App - https://github.com/madhums/nodejs-express-mongoose-demo
It shows some good usage of how to organise a node-express application, some common utilities for showing mongoose validation errors messages, stylus for css, jade views, etc. And the author appears to take requests if you think something should be added to the demo app to help others learn a new express/mongo/etc feature - I haven't asked personally but there is a check-list on the github page showing what features are completed and what's upcoming.
I would also recommend just getting your hands dirty :) Come up with a simple idea and try to build it. And when something doesn't feel right or you have a block then there's a pretty amazing and growing community out there.
Here are some tutorials not mentioned above:
Mongoskin (mongoose alternative) - http://www.hacksparrow.com/mongoskin-tutorial-with-examples.html
Exress JS tutorial: http://www.hacksparrow.com/express-js-tutorial.html
Also if you go to the github page for expressjs you will see many examples of specific express functionality - which goes into some great specifics:
Express js examples: https://github.com/visionmedia/express/tree/master/examples