We are building the messaging application in javascript using nodejs. As part of durability of the overall solution, we need to store text messages and media content outside ejabberd. To keep the text messages we used DynamoDB, DB as Service. But when we port-in the solution with different customers, customers are asking for different DBs for which they have licenses, like mysql, postgres, sql server, oracle, DB2, mongoDB etc...
Now my question is,
Does ORM/ODM is a good fit here for a loosely coupled persistency layer? If so, javascript supports ORM/ODM based solution
If ORM/ODM is not good fit, what are the alternatives
Just a thought, I looked at TypeORM, it does provide loosely coupled persistency layer to port-in with different back-end systems..., is this a feasible framework to take it forward.
Happy to hear experts inputs.
Regards
Venkata Madhu
TypeORM is a good candidate since it supports almost all the databases you mentioned (except DB2), I've worked with it and it's one of the best if you also want good typescript support.
Related
I'm developing a websocket real time browser game ( socketio nodejs) and wanted to start implementing my database and would like to know which type of database (nosql, sql .. ) would fit my situation best.
A small description of how i intend to use it :
I want to have game rooms with all their options stored ( which can be changed frequently)
a list of the connected users and their linked accounts( if a user is a guest he won't have a linked account, however if he does it would be linked with another table containing some extra data like lvl or something like that)
a list of user thema they like ( so for example when a game starts, it will take the themas in common for all users to use the game) all typical themas are stored in another big table that has a lot of data that won't be updated in a while.
So basically I have some tables that need a lot of quick and concurrent access which get updated and deleted frequently and some that don't need it and have a lot of permanent data.
I was thinking about using mongodb but honestly don't want to commit on something i don't have experience with ( i do have sql knowledge). I need suggestions thanks
TD;LR: I would personally go with PostgreSQL.
Recently, I had the same question as you, but with the only difference, I was not looking to build game rooms. After some research, I was convinced that PostgreSQL is awesome and suitable for every project. Well, tech giants use it too, including Facebook, Uber, Netflix... (https://stackshare.io/postgresql). It is scalable, easy to set one DB up, great community with lots of tutorials (both videos and articles) and it is also extensible, as it supports JSON, which is great!
You can use a great ORM for Node.js, such as TypeORM or Sequelize. Furthermore, I suggest you take a look at GraphQL, an API that has subscriptions (real-time operations). As I guess it will be a web application, I highly recommend you to go with React.js for the front-end functionalities, which interact great with GraphQL & Node.js. Last but not least, lots of developers and companies use both MongoDB and PostgreSQL for different functionalities and purposes for each. For user's tables use PostgreSQL, for multiple "big" JSON objects use MongoDB.
I hope that helps and lets you understand what you want in a more clear way. Good luck with your project!
I'm mainly a front-end devloper, so the basic server-side aspect of the project I'm working on is escaping me.
My main webpage has a list of products pulled from a JSON array with angularjs' ng-repeat, but I'd like to add an administrative page in which authenticated users could push a JSON object to the array. How can I do this with no experience in any server-side scripting languages?
I hate to be the bringer of bad news, but when dealing with user permissions, you need to be using some kind of back-end technology.
Theoretically, you could store authentication information in the JSON that denoted whether or not a user is an admin, and display the content/assign permissions accordingly. The problem with this technique is that, since the code is on the client side, hacking the admin page would be as easy as altering the JavaScript. This method would provide literally no security whatsoever.
If this is okay with you, then go ahead. But I would strongly advise learning some kind of server-side technology.
Given that you're a font-end developer, I'm assuming you already know JavaScript, so maybe doing this in Node would be a good option. Node does have a learning curve in and of itself, but knowing JavaScript will help significantly. You'll also need some kind of basic database to store the login information. Some kind of noSQL dialect such as MongoDB would probably work well here.
For doing advanced permissions, you'd want to use something like LDAP or Active Directory, but for what you're describing this would probably be overkill.
Remember, learning new things is fun! Good luck!
I'm currently working on a new google polymer web application and wondered if I should use firebase as the backend/db. I took a look at the project, made some test applications and really liked it! But to fully convince me, that firebase is the way to go I need the following questions answered:
I'm a little bit concerned about security: So, I know, that firebase uses read, write and validate to implement server side security. From the samples, I noticed that the validation basically is a one-line JS script, that represents a 'if'. As I'm planning to build a web e-commerce application I need to validate quite some inputs. Is there a possibility, to outsource the validation in a separate file, to make it more readable? Also I wondered, if there is a possibility, to test these server side validations, with for example unit tests?
I'm not 100% sure at the moment, that firebase can cover all of our use cases. Would it be possible/a good solution to use a "normal" backend for some critical functions and then persist the data from the backend in firebase?
I saw some nice polymer elements for firebase. Is firebase 100% supported in polymer/web components?
Is there an other way (like Java approach) to implement server business logic?
Is there a way, to define update scripts, so that new releases can easily be pushed to production?
Thanks & kind regards
Marc
So, I asked the firebase supprt and got the following answer:
Great to meet you.
I'm a little bit concerned about security: So, I know, that firebase uses read, write and validate to implement server side security. From the samples, I noticed that the validation basically is a one-line JS script, that represents a 'if'. As I'm planning to build a web e-commerce application I need to validate quite some inputs. Is there a possibility, to outsource the validation in a separate file, to make it more readable? Also I wondered, if there is a possibility, to test these server side validations, with for example unit tests?
You can implement extremely complex and effective rules using our security rules language. You can deploy security rules as part of your hosting deploy process, or via the REST API. It's not possible to break the contents into multiple files on the server, but you could certainly build your own process for merging multiple files into a single JSON result.
I'm not 100% sure at the moment, that firebase can cover all of our use cases. Would it be possible/a good solution to use a "normal" backend for some critical functions and then persist the data from the backend in firebase?
Generally speaking, synchronizing Firebase and a SQL back end is not very practical and they don't translate well. It's probably entirely redundant as well.
I saw some nice polymer elements for firebase. Is firebase 100% supported in polymer/web components?
I don't know what 100% supported means in this context. We offer a JavaScript SDK so they should play fine together.
Is there an other way (like Java approach) to implement server business logic?
We offer official SDKs in Java, Objective-C/Swift, Android, Node.js, JavaScript, and a REST API for use with other languages.
Is there a way, to define update scripts, so that new releases can easily be pushed to production?
I'm not sure what this means. Most likely the answer is no, since we don't provide a build process or any tools to release your software.
I hope that helps!
I responded:
Thank you for the information, it helped me very much! After reading your response on question number 5 one further question popped into my mind:
…
5. Is there a way, to define update scripts, so that new releases can easily be pushed to production?
I'm not sure what this means. Most likely the answer is no, since we don't provide a build process or any tools to release your software.
Is there like a best practice on how to handle the database schema? I only have one web application (without apps, etc.) in my case... I expect, that the database will change drastically over time and releases. Should I write JS logic, that checks the current database version and update it, if it's necessary? Maybe this would make a nice feature...
For example: I deployed Version 1.0 of my application and everything works fine. After 3 months of programming I notice, that the user data needs a further attribute: address, which is a 'not null' attribute. I now deploy Version 2.0 of my application and every new registered user has a address, but the old users (from Version 1.0) do not have this field or a value.
How should I handle this?
Support responded:
Hi Marc,
There’s no best practice here, but your ideas seem fairly sound. You probably don’t need to check in your JavaScript. You can probably store a version number in the user’s profiles, and when they upgrade to the latest software, you can upgrade that in their profile data.
Then your validation rules could use something like the following:
{
"user": {
".write": "newData.hasChild('address') || newData.child('appVersion') < 4",
"address": {
".validate": "newData.isString() && newData.val().length < 1000"
}
}
}
So if you are concerned about versioning, this could be used to deal with legacy releases.
Another popular approach I’ve seen from devs is to do intermediate upgrades by duplicating data. Thus, you release an intermediate version that writes to the old path and to the new path with the updated data structure (which keeps the app working for old users till they upgrade). Once a reasonable percent of clients are upgraded, then release a final version that no longer does a dual write to the old structure and newer structure.
Of course, flattening data, while it makes joining and fetching data bit more of a pain, will make upgrades much easier as the modular data structure adapts more easily to changes. And, naturally, a pragmatic design where you wrap the various records in a class (e.g. the UserProfile class with getter/setter methods) makes transitions simpler as you can easily hack in versioning at one place.
Hope this helps someone :)
I'm about to start writing an application and I want to include newer technologies to improve my knowledge and gain a skill set.
I'm using mysql to store user data (no choice), Angular.js to display/template content on the dom, and I wanted to use node.js to return results from the database in json format to be used by Angular.js.
My assumption was I could use node.js (along with the node-mysql module and maybe express)to accomplish my database queries that feed into my Angular.js templates.
Additionally, the application will live on a different server than the database itself and may reside in a mobile framework.
Am I way off base when it come to how to use node.js? Am I just using the wrong tool? Am I on the right track?
Any help, explanation, and points in the right direction would be great. Most of the info I've seen are copy/paste from the node.js site/wiki and don't really answer my question.
This is largely an opinion-based question and likely to be closed, but you're not way off base. What you want to do is absolutely (and commonly) done using Node.js.
As a complete aside, you might like to check out Platform-as-a-Service providers such as Heroku.com and Nodejitsu.com; both offer great and easy Node.js hosting and addons (such as NoSQL and RDBMS data stores) which have free tiers.
For that matter, MongoLab and MongoHQ both offer free tiers independent of Heroku, which you could use from your MySQL-only hosting provider if you like.
I tried to implement a simple login system with Mongodb and Node.js, but I found it to be too much of a hassle (everyone is building things on a hundred of frameworks and not explaining half of the code). I probably could do it, but it would require too much work and so on, as I only want to make a simple multiplayer game.
My alternative to databases are plaintext files. I do plan to hash the passwords, of course. The first problem that came to my mind was that the file could potentially get too big, and that the server would slow down completely trying to read it (in a hypothetical situations, I doubt my game will have that many users). The solution would be to create a users/ folder on the server, and split the big file into many smaller ones. Their names would correspond with the usernames, and there hashed passwords and emails would be held. This solves the speed issue, and I can't see how I still need databases!
Am I missing something? Are there other advantages to using databases? Also, I would be very pleased if someone answered with a login Mongodb example, that would solve all my problems. Just make it as simple as possible.
There are a ton of reasons to use a real database instead of some hacked, half-baked file implementation. Some big ones:
ACID (this is really 4 reasons)
Performance & stability (databases are some of the most mature pieces of software still in use)
Standardization
If you want "lightweight," I'd recommend that you start with SQLite.
Related:
Why use SQL database?
Why use MySQL over flatfiles?
Let's say you have an authentication information as: UserName(10), Password(32). These makes 42 characters. Let's say each character is 4 bytes (consisting all characters in the universe), each user consumes 42 * 4 = 168 bytes. Even if you have 100.000 users, it would only require 16 MBs of memory. It can fit the whole RAM, you only need to wirte changes to disk, and read them all in the boot of the program. Of course it would have a litle overhead, but not a big deal.
However, handling files may become an issue, you do not have to re invent the wheel. This is what databases are for. I agree, using Mongo, MySQL can be a overkill sometiems, however If you only plan to use database for authentication, you may go with more lightweight solution, you can use SQLite, which has only a file, a ~270KB client and a nice NodeJS library. Have a look at this nice post
If you are working with mongo I would strongly recommend working with the Mongoose library. It provides an excellent ORM for Mongo. If you do end up deciding to work with that ORM there is an established authentication library called mongoose-auth. I used the default password example for my own site.
If you are using NodeJS mongoDB would work well for what you want, there is also the option of redis,couch etc... MongoDB is as good a place as any to start and f you have been playing with it a little might as well stick with it.
Have a look to this repo which using a cloudfoundry or Mongolab (both free) mongoDb server.
https://github.com/daithiw44/Node_Authenticated_MongoDB
The demo has a button which when clicks hits the db to get a value. Substitute the hardcoded value for values in username and password textfields and one could authenticated a user and set up an expressjs session.
This was written with an older version of node but a tweak to the package.json should give you a good place to get started.
Hope that helps.