In general I want to know the possible benefits of Graphite. For now I have a web app that receives data directly from JavaScript Ajax call and plots the data using high chart.
It first run 20 different queries for each graph using Python from my SQL database.
And sends each result data to HighChart library using GET Ajax call.
And HighChart adds plot to each graph in realtime.
There is no need to save data because I need only realtime plotting within certain time range. Data outside time range just plushes.
But when I see the 20 Ajax calls in one page I feel like I am doing this in an inefficient way although it gets the job done.
So I looked at the Graphite but it is hard for me to decide which is better. Since I will pull up all data from present SQL table I don't need another storage. But everybody says graphite performs fast but I would still need to instantiate 20 different graphite graphs. Please give me some guidance.
What would you do if you have to visualize 20 different realtime graphs in one page concurrently each of which receives its own query data?
Maybe better is call one ajax which gets all data and then prepare parser which will return data for each chart.
Related
I have front end in ReactJS and backend api in Spring Boot. I have to show a drop down list with records from Api.
The scenario is i get a list of Users Ids from one end point, then for each record i have to call another api end point to get address details associated to that user ( it may have large number of records), the drop down is name of address.
My problem is if i loop through all records and get the address details, it will time out and take forever.
Can anyone suggest me the correct way to do it, Java or JavaScript.
I have read a little about observable, observer etc. but i did not get anything. Is there any concept of updating an object continuously.
Thanks
The shorter the distance the faster it can go. If you do this client side, the server has to send data on the network and then the client can process it. The best way would be to write a SQL query that does the right joins to get the data quickly. Send the processed data to client.
One good way to handle data is by implementing some kind of a pagination. You don't need to display all of the data in one go and it will be expensive to do so. For instance, if you have 1000 items on a list which you want to fetch, you could do it by 10s or 20s depending on your preference. This way, you could minimize the number of queries. Thus, making it much faster.
Here is an example of doing that on React JS. I'm only tapping into a Fake Online REST API
Hope this helps.
I am working on a web app where I have made a call to an API and stored the data using MongoDB. This data gets updated daily so I will need to be able to update the data daily by clicking a button in admin site. What is the best way to approach this?
I am new to using databases so I do not know the best approach. The reason I am wanting to store data in database is so I can store it using Redux or Context API so when someone goes to a page the data will be available faster instead of having to make a new API call (and wasting an API call) every time someone visits a page.
My database contains about 630 documents at a time.
Issue:
I need to update the 630 documents in my database to match the 630 documents coming from API that changes daily so I need to figure out what to query MongoDB to accomplish this.
You can use node-schedule.
It's very much like cron-job. But runs on the node application. Make sure the scheduler runs every 24 hours interval and put this database oprtation there.
Note that Node Schedule is designed for in-process scheduling, i.e.
scheduled jobs will only fire as long as your script is running, and
the schedule will disappear when execution completes. If you need to
schedule jobs that will persist even when your script isn't running,
consider using actual cron.
I ended up finding a way to solve my issue.
Previously I have updated a document one at a time so I used something along the lines of db.collection.update(<query>,<update>) to update one document. Issue I was facing was I needed to update all the documents in collection at one time. So by using db.collection.remove({}) I was able to remove all the documents in collection then used db.collection.create(myData) to add new updated data.
Please look at my project architecture below.
I intend to use Arduino as Web server and pull data into the SQL database of my PC. I have installed XAMPP, so I have the apache server and MySQL running in my PC. I have verified that data from Arduino is updating a table in the SQL DB.
Question:
I'm not a web-developer and I have been exploring different ways of pulling data from the SQL DB and showing it on web-page.
My requirements:
Pull live data from the SQL DB (as it gets updated) and show it on web page
Plot graphs of recorded parameters from the db as and when required. I understand that Flot is a tool which can do that.
Make a drawing as shown in the image above and populate that with dynamic values (numbers) or change colors of objects (such as green/red)
Methods I have come across so far
Use PHP to pull the data and show on webpage. Use HTML/Javascript/jQuery to create such drawing and populate with image and values
Use Python to pull data and use HTML/Javascript/jQuery (I know only HTML among these)
Use Visual Basic to pull data directly from Arduino and display it in a form. For this option, I'm not sure if plotting data is possible or not.
This of-course is a broad question and as I said I'm not a web developer. But I'm in no hurry. I have couple of months of time to invest and learn the different languages involved.
I wish the answers could guide me to a better solution than I know or chose one among the others or just point out plus-and-minus of each of the options
You can use EventSource at JavaScript to stream data from PHP or Python the the HTML document.
The project I'm working on involves financial data, my issue specifically pertains to chart visualizations with d3.js. So my back-end is in PHP using Laravel and my front-end will request data through an AJAX call to my back-end and it will send data to the client in the response.
I then use d3.js to visualize the data in various graphs. Because this is async and uses async GET requests, I don't have any specific URL path to navigate to in order to 'save' a state. Navigating away from this page will cause a reset for anything a user might have entered.
I am wondering if there is a way to easily integrate some sort of method for saving a chart's state. I am thinking maybe cookies could be something to look into? The user can add various technical indicators such as moving averages and there is technically no limit to how many indicators they would like to show. As a result, a user can concoct some complex charts--all of which would be reset anytime a user wants to refresh the page or in the off-chance they accidentally hit the back button.
I already have user registration on the website so I could also consider just saving chart states on my end, as well, and fetching them when a user navigates to the charts page. I have access to both a MongoDB database and a MySQL database.
I've never done anything like this before since this is my first time I'm working with a deployed app and the first time I've really needed to consider something like this.
Multiple charts also are depicted on this page, if that makes any difference.
Just starting to use Highcharts. If I include data in an array within the javascript the data is available for anyone to download when they view the source. This would be the same when data is called, say, from a csv file. Is there a way of protecting the data against copying/download?
No, since HighCharts is a client-side JavaScript library, data available to it is also potentially available to the end user. There really is no way to "secure" it once the data reaches the user's browser, although you can use HTTPS, server-side authentication, etc to at least guarantee in principle that only the intended user receives the data.
If you need to visualize your data while keeping the actual raw data secure, the obvious solution is to render the data on the server and just (in the end) serve up an image or other static content to the user. But then you lose the nice, interactive charts.
You might be able to use Flash or Silverlight to retrieve the data, to make part of the process harder to reverse engineer. This is not securing anything, just making it a bit harder for a determined user.
On the other hand, a user can see the data anyway in the final chart. If they really want to download the data they could painstakingly identify each data point and create their own CSV file, right? You need to figure out what is good enough for your particular use case, and strike the appropriate balance.
Being that HighCharts is a client-side JS system, I don't believe there is a way to get data to it securely. If you just attempt an AJAX call to get data at runtime, a user can see that call and the response. As you said you cannot just populate a variable in the source, as it is visible there.
Try the render charts on server feature:
http://www.highcharts.com/docs/export-module/render-charts-serverside