Im quite a newbie when it comes to DBs and search so please bear with me. Im trying to implement a search function in Meteor app.
Ive checked through atmosphere and chanced upon these 4 options.
Mattodem easy search
Search Source
Elastic search package on Atmosphere (heard this is dated)
Elastic big data package on Atmosphere
My needs are for a simple search, able to handle larger data sets that may have data nesting. For example a task, may have comments or links etc. (Full text and more than regex query will be good)
I read that for easy search, even with some form of elastic or Mongodb application, seems to be application only for a single Mongo collection?For example what if I wanted to search across
Dinosaurs = new Meteor.Collection('dinosaurs'); and
Mammals = new Meteor.Collection('mammals');?
Appreciate any advice on the pros and cons on the 4 options above? 1 seems relative easy to implement but I,m not sure what it means by using elastic engine here. If I were to implement elastic, how do I go about doing it in Meteor? And would it differ from implementing a Elasticsearch HTTP API?
Similarly for Search Source, it supports Elastic too here. Honestly I have no clue on the differences.
This github project demonstrates how to use Meteor with ElasticSearch (not Mongo) for search applications
https://github.com/hharnisc/meteor-elasticsearch-demo/tree/master/elasticsearch-demo/packages/elastic-collection
it basically links a field to a session variable which triggers search on elastic. results are stored in a mongo collection that is pushed to the client through the Meteor mechanics. These are much smaller than the dataset and therefore are quick to update.
As far as searching accross collections, this is a typical problem of database schema definition. If you need to do joins, a mongodb is not the best solution. and relational DB like any SQL DB would be better suited, However, you should be looking into 'denormalization' of your data to see if you can create a structure that will work well for your use case.
This article explains what this means pretty well:
http://blog.mongodb.org/post/87892923503/6-rules-of-thumb-for-mongodb-schema-design-part-2
Good luck.
Related
We are building the messaging application in javascript using nodejs. As part of durability of the overall solution, we need to store text messages and media content outside ejabberd. To keep the text messages we used DynamoDB, DB as Service. But when we port-in the solution with different customers, customers are asking for different DBs for which they have licenses, like mysql, postgres, sql server, oracle, DB2, mongoDB etc...
Now my question is,
Does ORM/ODM is a good fit here for a loosely coupled persistency layer? If so, javascript supports ORM/ODM based solution
If ORM/ODM is not good fit, what are the alternatives
Just a thought, I looked at TypeORM, it does provide loosely coupled persistency layer to port-in with different back-end systems..., is this a feasible framework to take it forward.
Happy to hear experts inputs.
Regards
Venkata Madhu
TypeORM is a good candidate since it supports almost all the databases you mentioned (except DB2), I've worked with it and it's one of the best if you also want good typescript support.
I'm developing a websocket real time browser game ( socketio nodejs) and wanted to start implementing my database and would like to know which type of database (nosql, sql .. ) would fit my situation best.
A small description of how i intend to use it :
I want to have game rooms with all their options stored ( which can be changed frequently)
a list of the connected users and their linked accounts( if a user is a guest he won't have a linked account, however if he does it would be linked with another table containing some extra data like lvl or something like that)
a list of user thema they like ( so for example when a game starts, it will take the themas in common for all users to use the game) all typical themas are stored in another big table that has a lot of data that won't be updated in a while.
So basically I have some tables that need a lot of quick and concurrent access which get updated and deleted frequently and some that don't need it and have a lot of permanent data.
I was thinking about using mongodb but honestly don't want to commit on something i don't have experience with ( i do have sql knowledge). I need suggestions thanks
TD;LR: I would personally go with PostgreSQL.
Recently, I had the same question as you, but with the only difference, I was not looking to build game rooms. After some research, I was convinced that PostgreSQL is awesome and suitable for every project. Well, tech giants use it too, including Facebook, Uber, Netflix... (https://stackshare.io/postgresql). It is scalable, easy to set one DB up, great community with lots of tutorials (both videos and articles) and it is also extensible, as it supports JSON, which is great!
You can use a great ORM for Node.js, such as TypeORM or Sequelize. Furthermore, I suggest you take a look at GraphQL, an API that has subscriptions (real-time operations). As I guess it will be a web application, I highly recommend you to go with React.js for the front-end functionalities, which interact great with GraphQL & Node.js. Last but not least, lots of developers and companies use both MongoDB and PostgreSQL for different functionalities and purposes for each. For user's tables use PostgreSQL, for multiple "big" JSON objects use MongoDB.
I hope that helps and lets you understand what you want in a more clear way. Good luck with your project!
I want to create a search engine, using javascript for the cllient side and Amazon ES for the backend. However, since i am just discovering the elasticsearch api, i am looking for a detailed explanation of how i can index my dynamodb entries in real-time and then building a customized search engine with some features like auto suggest, auto filling etc.
I can't find any tutorial on the web that explain the entire process.
Any help would be very appreciated.
Thanks.
This plugin can help with streaming the data from DynamoDB:
https://github.com/awslabs/logstash-input-dynamodb
The readme has some information, I'm not sure if that's the kind of detail you are after, but it's a good place to start.
I recently found out about Firebase and it looks like something i wanted for my project. I'm trying to make a real-time html dashboard for ordering, yet i'm very much beginner at javascript so i was hoping someone could help me out.
I'm trying to make real-time dashboard that is blank, and updates div container when specific table in database in updated.This is somewhat what i'm aiming for.
I'm thinking it would work something like:
If database != null then create/show that container. Done button would just delete selected table in database.
But i don't know how to implement that using javascript. Any help/tutorial would be great, i searched a lot and didn't quite find what i needed.
Thanks in advance!
They answer you're looking for is beyond the scope of a single question. There are a number of great resources out there for learning how to set up your first web app using Firebase.
Take a look at their quickstart guide:
Firebase Quickstart Guide
Firebase also offers a number of examples:
Firebase's Examples
In addition, there are also a number of frameworks such as Angular.js, which add another layer of technology to learn, but provide useful features such as routing and data binding. I'd recommend Scotch.io's guide for creating an angular.js/firebase web app:
Scotch.io's Angular and Firebase Tutorial
I am currently using meteor.js for a social networking application that I am building. Currently I am struggling to find the best way to design a newsfeed for my users. Users are able to follow each other and create posts that should theoretically go to their followers. I have come up with two possible concepts that would work theoretically.
I. When a users makes a post the post goes into the global database which every user is subscribed and listening to, and whenever a post is added the user checks to see if the parent if they are following the parent of the post, and if they are they display it onto their newsfeed.
This way seems horribly inefficient, but it is a way.
II. The other way would be for each user to have their own little "hub" or each user would just subscribe to and listen for change in the user's hub.
This way seems better but I would have to do a for-loop on startup and subscribe to each user manually. Which may cause a slight delay on startup.
I am thinking that option two will work best, but I am wondering if anyone has any suggestions on how to accomplish this task. Or if there is a standard way to go about this. Thanks.
Look at the reywood:publish-composite package. I'm doing something similar in my app and am using this package to essentially perform a join. In your case people are following people so when creating a publication that returns the set of users that a person is following you can at the same time return the union of all their posts.
Building scalable feed technology is quite a hard problem to solve. With Meteor it is even harder because MongoDB doesn't lend itself very well for the storage of relational data, since it is a non-relational database, and creating publications that publish 'joined' data isn't straightforward. A lot of articles have been written about the problem of building scalable feed technology, have you seen my article on using Meteor in combination with Stream to build a solution that fits your needs?
Another route would be to use an open-source framework to create the backend for your feed technology. Have a look at Stream-Framework, the repositories README also has a great list of resources on building scalable feed technology.