I am currently using meteor.js for a social networking application that I am building. Currently I am struggling to find the best way to design a newsfeed for my users. Users are able to follow each other and create posts that should theoretically go to their followers. I have come up with two possible concepts that would work theoretically.
I. When a users makes a post the post goes into the global database which every user is subscribed and listening to, and whenever a post is added the user checks to see if the parent if they are following the parent of the post, and if they are they display it onto their newsfeed.
This way seems horribly inefficient, but it is a way.
II. The other way would be for each user to have their own little "hub" or each user would just subscribe to and listen for change in the user's hub.
This way seems better but I would have to do a for-loop on startup and subscribe to each user manually. Which may cause a slight delay on startup.
I am thinking that option two will work best, but I am wondering if anyone has any suggestions on how to accomplish this task. Or if there is a standard way to go about this. Thanks.
Look at the reywood:publish-composite package. I'm doing something similar in my app and am using this package to essentially perform a join. In your case people are following people so when creating a publication that returns the set of users that a person is following you can at the same time return the union of all their posts.
Building scalable feed technology is quite a hard problem to solve. With Meteor it is even harder because MongoDB doesn't lend itself very well for the storage of relational data, since it is a non-relational database, and creating publications that publish 'joined' data isn't straightforward. A lot of articles have been written about the problem of building scalable feed technology, have you seen my article on using Meteor in combination with Stream to build a solution that fits your needs?
Another route would be to use an open-source framework to create the backend for your feed technology. Have a look at Stream-Framework, the repositories README also has a great list of resources on building scalable feed technology.
Related
I'm developing a websocket real time browser game ( socketio nodejs) and wanted to start implementing my database and would like to know which type of database (nosql, sql .. ) would fit my situation best.
A small description of how i intend to use it :
I want to have game rooms with all their options stored ( which can be changed frequently)
a list of the connected users and their linked accounts( if a user is a guest he won't have a linked account, however if he does it would be linked with another table containing some extra data like lvl or something like that)
a list of user thema they like ( so for example when a game starts, it will take the themas in common for all users to use the game) all typical themas are stored in another big table that has a lot of data that won't be updated in a while.
So basically I have some tables that need a lot of quick and concurrent access which get updated and deleted frequently and some that don't need it and have a lot of permanent data.
I was thinking about using mongodb but honestly don't want to commit on something i don't have experience with ( i do have sql knowledge). I need suggestions thanks
TD;LR: I would personally go with PostgreSQL.
Recently, I had the same question as you, but with the only difference, I was not looking to build game rooms. After some research, I was convinced that PostgreSQL is awesome and suitable for every project. Well, tech giants use it too, including Facebook, Uber, Netflix... (https://stackshare.io/postgresql). It is scalable, easy to set one DB up, great community with lots of tutorials (both videos and articles) and it is also extensible, as it supports JSON, which is great!
You can use a great ORM for Node.js, such as TypeORM or Sequelize. Furthermore, I suggest you take a look at GraphQL, an API that has subscriptions (real-time operations). As I guess it will be a web application, I highly recommend you to go with React.js for the front-end functionalities, which interact great with GraphQL & Node.js. Last but not least, lots of developers and companies use both MongoDB and PostgreSQL for different functionalities and purposes for each. For user's tables use PostgreSQL, for multiple "big" JSON objects use MongoDB.
I hope that helps and lets you understand what you want in a more clear way. Good luck with your project!
I just start thinking about communication between two angular (or others framework) applications - what is the best way to do that.
For now, in my application, I used iframe to include the second app and then thanks to postMessage (docs) I can handle actions between apps.
But this solution is getting really hard to maintain because I didn't expect at the begging that I will have to handle soo much type of events.
Let's give you an example:
On Facebook when you go to someone profile you can click "Send message" and it will open a new window on chat application. In other way you can also click on some button in their chat and it will send some action to "main" application.
I just read few articles and start thinking about using WebSockets, using firebase etc but I want to make some more research and ask more expired developers (yes, you! :) ) what they thinking.
So, my question is:
What is the best approach to include two (or more) angular apps?
What is the best approach to handle communication between these apps?
It sounds like you have a really fun problem to solve. There are a number of different ways to go about this, and I'm not sure how your website architecture is setup and what constraint's you have, but I can at least provide some suggestions.
Possible Solutions
Look into separating separate functionality into lazy loaded Angular modules, and treat them as separate websites altogether.
I've done this for a few applications and it gives you some of the flexibility you're looking for. For this to work with your application communication constraints, there are a number of different ways to approach it.
If you can get away with a single application, then you're going to want to look into state management solutions like NgXS, NgRX, Akita, or roll your own statemanagement. I personally roll my own for these things because I rarely need to overhead of the bigger frameworkss, but to each their own.
If you can't get away with a single application, then you're going to want to separate your state into something like FireBase, or a custom websocket app. This would give you better persistence of state, and also allow you to connect completely independent web applications to one another.
Another promising solution to things like this is the tool SingleSpa.
Single spa is a meta web framework that can help navigate and orchestrate multiple SPA's written in different libraries/frameworks into a single application. This would allow you to re-use the existing website that you're using the <iframe> for, but have it bundled within the same parent application. I'm not too familiar with this framework in practice, but I have some coworkers and peers who use it, and it seems promising.
Take a step back and identify the core problem you're trying to solve
This one isn't trying to troll or anything, just helping to take a step back and look over your application from a 10,000 foot overview. Look over how the data is being passed in the solution, are you solving the problem directly? Or is this solution a hack to make due with the constraints that you had at the beginning. Don't be afraid to rebuild parts of the application, or service. Theres a reason people talk about the 10 to 1 rule of writing code.
I understand time and complexity constraints, you have deadlines and other things to deal with. Just make sure you're solving the right problem and thinking about the different edge cases you can leverage to your advantage.
I recognize after writing this out that its not a direct answer per say, but I hope it helped. Message me if you have any questions on this stuff.
I want to build a recipe maker webapp for practicing.
This webapp will contain 2 web pages: index and recipes.
On ,,index" the clients can make some recipes.
On the ,,recipes" page the user can see a table of recipes made by her/him and by other users (the last 10 or so).
The recipes will be stored in a database.
This sounds easy until now, but if a user creates a recipe and clicks on the ,,Add" button, I want to update the recipes table in the recipe page without refreshing, so it will contain the new recipe (like real-time notification).
Can I use node.js, express.js and socket.io to make this kind of behavior? If so, what functions, features should I look up? Recently I started learning these technologies but I can't see how can I pass data between pages and how to invoke functions if the data has arrived (maybe routing?).
I really appreciate any help you can provide.
Yes, definitely you can use these technologies or others. Its all about, with what you are comfortable with.
Secondly, Look at their getting started. Socket.io has few github repo to get started. It is pretty easy. I did it in around 10mins with 0% experience in socket.
Socket.io has node.js documentation too. HTML and JS are just complementing resources for look and feel.
Ref: https://github.com/shah-smit/socket.io-getting-started
Lastly, you want to take a look at JQuery for animation.
Me and my startup app company is working on a turn based multiplayer iPhone application. Let it be said that neither one of us have any database, or server, knowledge whatsoever. Though, we are willing to learn.
The flow of the game will be similar to such games as: WordFeud, WordsWithFriends, Rumble etc.
Let me start of by where a lot of searching on the web has gotten us:
We have decided to use CouchDB as a tool for storing information about users, game sessions and other stuff. CouchDB is an open source noSQL database system. The reason is that we have been taught, that it should support a lot off concurring users. Besides, that it scales - we are hoping to go big, of course.
Our CouchDB, is hosted on IrisCouch. IrisCouch is an "in cloud" hosting service designed for running CouchDb.
So, we've got a CouchDB server up and running, and we know the basics on how to query data from the server.
Our biggest confusion right now, is how we should set up the system to work according to best practices. Right now we are at the point where we are able to receive and submit data to the server.
Our game is supposed to have Facebook integration, so that the users can register via our app or through Facebook. After that they can play with random matched opponents, or play with friends. After a match is started, one player will get a set of question to be asked, after he has answered, the other player should be notified, through push notification, that it's their turn. After a few rounds the game is finished.
At this point, we think this might be the best solution for the flow of the application:
A user connects to another user -> a game session is opened as a
document in a database called "games".
The newly created document contains both player names, question,
answers etc
A field named "whos_turn" decides which of the two players turn it is.
After the game has ended, the session is erased.
Again, and as you may see, we are in the dark as of how to really do it, but this is the general idea.
So, my questions goes as following:
Is it best to query the data directly from the iPhone application, or through a web service?
What is the best way to set up the database, to best manage the flow of the application?
Any information, that could lead us closer will be gladly appreciated :)
In advance, thank you!
Olav Gundersen
EDIT#1 : Our Objective-C programmer managed to connect two iPhone devices using CouchDb. The iPhone application consists of a table view, that has a concurrent connection with the database, so that when someone POST to the database, it shows up on the tableview of all the other connected phones. Behold: a severely ineffective chat system.
If is a multiplayer you would need to have the app to communicate to the remote iriscouch.db but I am concerned by the point where you state that neither you or your friends have any database experience. You are willing to learn so I think the best place to start is:
http://guide.couchdb.org/editions/1/en/index.html
There are several issue you might find with scalability if you plan to erase documents continuously. DB Size can be considerable on couchdb and you will need to compact &cleanup the db regularly. But I don't think is a major issue for now as this is at a start up level.
The question "best way to setup the database and best manage the flow of the application" should be addressed by your team. If you do not have someone with any database experience you should try to find someone willing to help you. It should be someone with extensive experience in databases. You might find some fairly reasonable professionals at http://www.odesk.com
In total honestly I don't think you will be successful if you don't have such a figure - either as a freelancer or contributor - to help you having a solid database logic in the game that will ensure a great user experience.
For example: have you considered the latency-delay issue by using a db based in the USA (Iriscouch) vs. where your users are located?
For this reason you might want to do as much as possible client side (embedded database like sqlite or touchDB that is essentially couchDB for iPhone)
For an iPhone application you might want to try TouchDB that is made exactly for that
https://github.com/couchbaselabs/TouchDB-iOS (caveat: being that you need connectivity to check turns etc this might not be the ideal solution but it could work to store some information locally).
To lay this out you would need someone with experience with couchdb to set up a proper, usable application. There is nothing wrong in being enthusiast about your idea but to make it a success you need a technical mind in the database side. Of course you might be well capable to learn this yourself. After reading the CouchDB book you should be in position to create a basic flow to fit your needs.
Of course other more experienced users might come with a more comprehensive answer or a sample layout but I don't think would be the best approach. Even if someone posts a full layout of the doc structure and how to query it how are you going to service the app if something goes awry e.g. sessions don't get deleted, conflicts etc. ? hence my sincere advice to get some ad-hoc expertise for your case.
This might also result in analyzing suitable alternatives. I don't think you should buy into the idea that CouchDB can scale and hence is the best/only option for you (of course this might well be the case and if you feel that is a good option..go for it). For example twitter, google adwords and many other online apps are using mysql to store their data so for sure CouchDB is not the only database that can scale!
I think this demo app could be a good example to follow: iOS Couchbase Demo
Preamble
So, this question has already been answered, but as it was my first question for this project, I'm going to continue to reference it in other questions I ask for this project.
For anyone who came from another question, here is the basic idea: Create a web app that can make it much easier to create other web applications or websites. To do this, you would basically create a modular site with "widgets" and then combine them into the final display pages. Each widget would likely have its own set of functions combined in a Class if you use Prototype or .prototype.fn otherwise.
Currently
I am working on getting the basics down: editing CSS, creating user JavaScript functions and dynamically finding their names/inputs, and other critical technical aspects of the project. Soon I will create a rough timeline of the features I wish to create. Soon after I do this, I intent to create a Blog of sorts to keep everyone informed of the project's status.
Original Question
Hello all, I am currently trying to formalize an idea I have for a personal project (which may turn into a professional one later on). The concept is a reflective web application. In other words, a web application that can build other web applications and is actively used to build and improve itself. Think of it as sort of a webapp IDE for creating webapps.
So before I start explaining it further, my question to all of you is this: What do you think would be some of the hardest challenges along the way and where would be the best place to start?
Now let me try to explain some of the aspects of this concept briefly here. I want this application to be as close to a WYSIWYG as possible, in that you have a display area which shows all or part of the website as it would appear. You should be free to browse it to get to the areas you want to work on and use a JavaScript debugger/console to ask "what would happen if...?" questions.
I intend for the webapps to be built up via components. In other words, the result would be a very modular webapp so that you can tweak things on a small or large scale with a fair amount of ease (generally it should be better than hand coding everything in <insert editor of choice>).
Once the website/webapp is done, this webapp should be able to produce all the code necessary to install and run the created website/webapp (so CSS, JavaScript, PHP, and PHP installer for the database).
Here are the few major challenges I've come up with so far:
Changing CSS on the fly
Implementing reflection in JavaScript
Accurate and brief DOM tree viewer
Allowing users to choose JavaScript libraries (i.e. Prototype, jQuery, Dojo, extJS, etc.)
Any other comments and suggestions are also welcome.
Edit 1: I really like the idea of AppJet and I will check it out in detail when I get the time this weekend. However, my only concern is that this is supposed to create code that can go onto others webservers, so while AppJet might be a great way for me to develop this app more rapidly, I still think I will have to generate PHP code for my users to put on their servers.
Also, when I feel this is ready for beta testers, I will certainly release it for free for everyone on this site. But I was thinking that out of beta I should follow a scheme similar to that of git: Free for open source apps, costs money for private/proprietary apps.
Conceptually, you would be building widgets, a widget factory, and a factory making factory.
So, you would have to find all the different types of interactions that could be possible in making a widget, between widgets, within a factory, and between multiple widget making factories to get an idea.
Something to keep on top of how far would be too far to abstract?
**I think you would need to be able to abstract a few layers completely for the application space itself. Then you'd have to build some management tool for it all. **
- Presentation, Workflow and the Data tier.
Presentation: You are either receiving feedback, or putting in input. Usually as a result of clicking, or entering something. A simple example is making dynamic web forms in a database. What would you have to store in a database about where it comes/goes from? This would probably make up the presentation layer. This would probably be the best exercise to start with to get a feel for what you may need to go with.
Workflow: it would be wise to build a simple workflow engine. I built one modeled on Windows Workflow that I had up and running in 2 days. It could set the initial event that should be run, etc. From a designer perspective, I would imagine a visio type program to link these events. The events in the workflow would then drive the presentation tier.
Data: You would have to store the data about the application as much as the data in the application. So, form, event, data structures could possibly be done by storing xml docs depending on whether you need to work with any of the data in the forms or not. The data of the application could also be stored in empty xml templates that you fill in, or in actual tables. At that point you'd have to create a table creation routine that would maintain a table for an app to the spec. Google has something like this with their google DB online.
Hope that helps. Share what you end up coming up with.
Why use PHP?
Appjet does something really similar using 100% Javascript on the client and server side with rhino.
This makes it easier for programmers to use your service, and easier for you to deploy. In fact even their data storage technique uses Javascript (simple native objects), which is a really powerful idea.