I'm building a dashboard receiving tons of data from MQTT in JSON format (data frequency ± 4Hz). I'm trying to visualize this data and initially picked Highcharts (http://www.highcharts.com/) but quite quickly realized that browser can't simply handle that amount of information.
I'm quite new to visualizing such amounts of data so would appreciate any advice on how to handle that amounts of data (currently arduino's post data to given topics and I receive them on the server, store in mongo and send to the browser to be updated). There's a chance my entire approach is completely wrong so please guide me to a path!
Your problem is likely that the SVG that Highcharts is creating is too large for the browser to handle in a timely manner. If you have too many data points, you'll need to do 1 of 2 things.
Switch to a charting library that uses an HTML5 Canvas instead of an SVG
Pare down your data using a best fit or some other curve interpolation.
If you switch to a Canvas based chart, you'll lose some of the cool features that SVGs have, but every data point will make it to the page. I've used jqChart for this in the past. It's not free though.
If you pare down your data, not every data point will make it to the page, but the nice features that SVGs allow will still be there. You'll still probably be able to use highcharts.
I think you need to have a look at crossfilter (filtering lib), dc.js (helper library for d3 and crossfilter) they can handle large amount of data well
After using Highcharts for more than 3 years I can recommend it even for the questioned use case. To follow recommendations to use other charting libraries might be not a good choice if you already picked Highcharts (after comparison with others) and invested quite some time in it. If you like it, stick with it.
To address this edge-case with thousands of data points the team announced the boost.js module.
It originated from this issue on Github. The discussion reveals lots of background information about the problem domain. And it demonstrates how well the Highcharts team is doing customer-/community-driven development.
Looks like you can do it with Highcharts - if you still want. :-)
Last but not least I can recommend the Highcharts Cookbook from Packt
I use grafana to visualise my time series data. I don't believe it supports mongodb at the moment. I use influxdb as the data back end.
Related
I am building a dashboard using JS, that will show users plots of their sales data. Can you suggest a js library, which:
Can plot many points (100k for instance or even more)
Interactive
Supports SSR without losing interactivity (if it’s possible. I just assume that the only way to load this number of data without hurting UX is SSR)
Optional: works with React
Many thanks for any help!
There's probably many options but one could be Victory Chart. The library is built on top of D3.js and is heavily customizable. For your large dataset, I can think of one solution I used that is documented here.
One reason I'd suggest Victory is because they have a nice support community here.
I read many blog posts about that topic, here's one that compares a few "big" chart libraries with React Components.
For time series, I'd suggest having a look at this library.
If nothing works with large datasets, I think that your best option would be D3.js (with or without a React implementation).
I am working on a web application which is dealing with bigdata visualization. I am using highcharts for visualization of data. I have tried all of the possible methods proposed by highcharts.
Now there is one work around in my mind is to request more memory from system for my web page. But I don't know how is it possible by using javascript in browsers.
please help me to find any best solution.
What I am proposing here is not a full answer but a set of questions, you should ask yourself, before going with this or that solution.
What is the size of the data?
Are we talking about hundreds, thousands or millions data point?
Can you aggregate the data points?
If you struggle to efficiently aggregate on the client-side, do it server side (and cache/memoize the results where possible). Do you have access to the server side code?
Do you need all the data at first load?
If no, find out what you need at first, then load the rest on demand.
Is the animation interactive?
If so, consider to remove non essential interactions. If the visualization is static just create the visualization server-side where possible.
What are the target browsers?
Study your user base browser and focus on the features they provide. If you don't have to support older browser you can focus on some speedups that the Web API provides you.
Do you have multiple visualization in the page?
Consider to split heavy dataviz into multiple views.
This list of questions is not exhaustive, but it is a starting point. If anybody has better questions to add just comment here and I'll update.
I want to visualize a Neo4j database in a graphical application. (I'm doing this primarily as a learning exercise).
I want it to behave almost identically to the visualization in Neo4j's WebAdmin: When the graphic appears, I want to see the nodes moving around as they settle into their final locations, and I want the user to be able to click and drag node icons around the screen and have them stick where they are dropped, with the rest of the graph adjusting to account for the change.
I have two questions.
What are the technical terms for the three requirements crudely described in the paragraph above?
I'm a Java programmer with no experience in JavaScript, but I'm thinking that, with this project, it's time to get some - sigma.js in particular seems very powerful and relatively simple. However, ultimately this needs to be done within a Java Swing application, and while I know there are various containers I can use to embed JS/HTML content into Swing, still it seems that a pure Java solution would be ideal, and I'm curious about any frameworks that might make this possible. I'm looking into Gephi now, but it doesn't seem anywhere near as powerful or as simple to use as sigma.js. Am I wrong in this assumption?
To summarize the second question: Which would be the better investment of my time: Learning sigma.js and the various JS frameworks (which will require me to get comfortable with JS in general), or learning to use Gephi in Java? (assuming I have a general and long-term interest in graph visualization)
Java visualization
Gephi does provide a toolkit for Java and Python which doesn't contains the UI modules, so unless you want to use Gephi itself to visualise your network you cannot embed its UI in your Java/Swing application as long as I know.
The toolkit contains all the algorithms and plugin infrastructure so if you want to use some of their analysis tool embedded in your app you can use that, otherwise I don't think it's your case.
There are other Java options if you want for graph visualisation here: Java graph library for dynamic visualisation
About Gephi and sigmaJS
IMHO Gephi is far more powerful than sigma.js: it has full SNA tools, multiple layouts, a data editor, exporters, plugin-systems...
One important factor is also the size of the dataset you want to visualise: while sigmaJS can handle up to 500 nodes in the view, Gephi can layout easily 50000 nodes with GPU support as well.
Gephi is a scientific tool made for people that don't want to code to visualise a network and it does it's job. SigmaJS is one of many JS solutions to visualise graphs on a web page.
You can always export to sigmaJS from Gephi with a plugin.
What's better?
At the end of the story I guess the tool you are going to use will depends on:
do you have to visualise a big dataset? -> Java solutions will win for big ones
do you need to distribute it easily? -> Web app / SigmaJS wins for distributed
do you want to have a full integration with your app? -> Java wins
The best one I've found is Keylines, but its not cheap.
Gephi is OK, but the last time I looked it did not support Neo4j 2.0
If your a programmer Neo4j is open source, so you can fork the repository on Git and use that (you will have to check the licencing model before you commercialise any product though)
I'm looking for a JavaScript charting API that would permit querying the server with bounds on the horizontal and vertical axes to get new data points. Before you ask, the data set is mind-blowingly massive: there are ~300 million points.
HighCharts is a pretty good looking charting library. It does support zoom, and AJAX lookups. I have a feeling you'll have some legwork to do to hook zoom events to do data updates, but can't say for sure as I haven't used HighCharts a whole lot.
You may want to check out this presentation:
http://www.ted.com/talks/view/lang/en//id/129
Seadragon deals specifically with pixels similar to what you are asking about.
So I've been working on a project that requires offering the possibility to view the results of a simulation in 2D. Now the solution that was first considered was RGraph. However it seems that now the fact that RGraph's free license requires a link to their page is not in agreement with the client and we need to find some free alternatives.
Now I'm starting to look at JIT and Google Chart Tools but I'm not sure yet either if their license is better that the one of RGraph, or how difficult it would be to make the actual changes.
So basically I would like to know if you have any other javascript libraries that fit the needs (being capable of plotting a 2d array for example and having a free license) or if you have any reason why I should dismiss one of the two libraries I proposed to look at, that would also save me some time.
Thanks,
Bogdan
try flot
http://code.google.com/p/flot/
Ext JS 4 offers a variety of chart types.