I am working on a web application which is dealing with bigdata visualization. I am using highcharts for visualization of data. I have tried all of the possible methods proposed by highcharts.
Now there is one work around in my mind is to request more memory from system for my web page. But I don't know how is it possible by using javascript in browsers.
please help me to find any best solution.
What I am proposing here is not a full answer but a set of questions, you should ask yourself, before going with this or that solution.
What is the size of the data?
Are we talking about hundreds, thousands or millions data point?
Can you aggregate the data points?
If you struggle to efficiently aggregate on the client-side, do it server side (and cache/memoize the results where possible). Do you have access to the server side code?
Do you need all the data at first load?
If no, find out what you need at first, then load the rest on demand.
Is the animation interactive?
If so, consider to remove non essential interactions. If the visualization is static just create the visualization server-side where possible.
What are the target browsers?
Study your user base browser and focus on the features they provide. If you don't have to support older browser you can focus on some speedups that the Web API provides you.
Do you have multiple visualization in the page?
Consider to split heavy dataviz into multiple views.
This list of questions is not exhaustive, but it is a starting point. If anybody has better questions to add just comment here and I'll update.
Related
I'm building a dashboard receiving tons of data from MQTT in JSON format (data frequency ± 4Hz). I'm trying to visualize this data and initially picked Highcharts (http://www.highcharts.com/) but quite quickly realized that browser can't simply handle that amount of information.
I'm quite new to visualizing such amounts of data so would appreciate any advice on how to handle that amounts of data (currently arduino's post data to given topics and I receive them on the server, store in mongo and send to the browser to be updated). There's a chance my entire approach is completely wrong so please guide me to a path!
Your problem is likely that the SVG that Highcharts is creating is too large for the browser to handle in a timely manner. If you have too many data points, you'll need to do 1 of 2 things.
Switch to a charting library that uses an HTML5 Canvas instead of an SVG
Pare down your data using a best fit or some other curve interpolation.
If you switch to a Canvas based chart, you'll lose some of the cool features that SVGs have, but every data point will make it to the page. I've used jqChart for this in the past. It's not free though.
If you pare down your data, not every data point will make it to the page, but the nice features that SVGs allow will still be there. You'll still probably be able to use highcharts.
I think you need to have a look at crossfilter (filtering lib), dc.js (helper library for d3 and crossfilter) they can handle large amount of data well
After using Highcharts for more than 3 years I can recommend it even for the questioned use case. To follow recommendations to use other charting libraries might be not a good choice if you already picked Highcharts (after comparison with others) and invested quite some time in it. If you like it, stick with it.
To address this edge-case with thousands of data points the team announced the boost.js module.
It originated from this issue on Github. The discussion reveals lots of background information about the problem domain. And it demonstrates how well the Highcharts team is doing customer-/community-driven development.
Looks like you can do it with Highcharts - if you still want. :-)
Last but not least I can recommend the Highcharts Cookbook from Packt
I use grafana to visualise my time series data. I don't believe it supports mongodb at the moment. I use influxdb as the data back end.
I would like to write a 3D application using Golang, my favorite programming language. However, I would prefer not to use native OpenGL directly as the graphical frontend as this would entail a larger learning curve than I'm willing to tolerate. Additionally, I don't really want to use a Golang game engine like go:ngine.
After doing some research I found out about the amazing WebGL Javascript library three.js and I was so intrigued that I completed an introductory Javascript course in a few hours. I'm not really a web developer at all, so I'm wondering how practical it would be for me to write my application primarily in Go but with a three.js frontend.
Some specific questions:
Is it worthwhile / practical to use Javascript/three.js as a frontend to a Golang server like this?
If it is worthwhile, where can I look to learn how to integrate the two cleanly?
If it isn't really worthwhile, which alternatives do I have?
Thanks, any response appreciated.
Edit:
Do you plan to make operations on geometry in Go, possibly in realtime and communicate with your Javascript layer using Websockets?
Yes, this is what I'm thinking now. I'm intending my (Golang) program to generate streams of vertices and triangles based on a few parameters from the user. Each "structure" is generated all at once after the users supply their parameters (which I see as slider bars + input boxes on a Javascript frontend).
Here's an example of something the Go program might generate, plotted in GNUPlot:
(source: alexozer.com)
If this is the case you underestimate Javascript a bit. You should write most of your application in Javascript and use Go as a classical backend service layer like user accounts, persisting state, etc.
I'd be fine with doing that, except for these reasons:
The core generating process fundamentally depends on Goroutines and channels
I know close to nothing about web development, including frontend and backend
So I suppose I'm asking about the implementation details of one of these three possibilities:
Writing most of the program, including the generator, in JS, and a minimal backend in Go
Writing most of the program in Go and using JS just as a graphical frontend
Something else?
I recommend leaving out the whole Websocket aspect because it's only a possible optimization (and maybe not even that).
If you need to depend on Go's concurrency model then go for it, write your component in Go, then a Webserver in Go that takes parameters coming from an HTML request, uses them to compute the result and sends back the data in a JSON format.
On the frontend you will then only focus on sending this request when the user changed a parameter and for displaying of the JSON data you can use ThreeJS right away.
You'll still have to learn a bit of web development and Javascript though. But hey, they say the web is the future ;-)
I think it's the way to go for your case because your application operates in a quite strict request-result way.
The answer to your question depends a lot on where exactly you plan to draw the line between your Go component and the Javascript component. From the way you describe the problem I get the feeling that you want to write most of your application in Go and use JS only as a display layer? I wouldn't recommend doing that.
To rephrase my question: Do you plan to make operations on geometry in Go, possibly in realtime and communicate with your Javascript layer using Websockets? If this is the case you underestimate Javascript a bit. You should write most of your application in Javascript and use Go as a classical backend service layer like user accounts, persisting state, etc.
I might be able to give a more specific answer if I know what exactly you plan to do.
I'm currently developing a server side application presenting graphs, stats, reports and so on.
The application is using Highchart library (chart generating library) from Highsoft (Awesome tool!) based in javascript.
If I were to create charts in front end, using highchart is all cool. But problem.. or consideration comes when I were to create charts in back end (server side) with generating report.
Because the nature of Javscript, where it is design for front end (web), if I were to use them in backend, I have to use phantomjs/nodejs.
But if I were to do the same job in serverside languages, such as PHP, I'm certain that it can process/create the charts much faster. (with different charting lib)
Also it is difficult for me to fully rely on stability of using phantomjs/nodejs. (I have seen and experience few issue with running nodejs)
So the question I would like to ask people is that what is the "Performance Difference between processing in back end vs phantomjs/nodejs" under assumption that they do same calculation/computation task in terms of metrics. (eg approximately php is 3 times faster then javascript through phantomjs to do same task.)
As I'm a newbie developer and I need much advice or suggestion before I decide which language to choose before implementing them.
If anyone already have done it, please share the knowledge. It would be greatly appreciated!
Thank you for reading.
-Danny C
The nature of reports is that they need to present correct data and interactivity part is less important, or even impossible.
If you generate report as dedicated URL where user navigates to, use same chart you are using in frontend
If you are generating PDF/Email reports go for stable and (if needed) extendable server side solution, pChart for PHP or some other stable and well documented solution
And remember, people are quite used to idea that chart on reports look different that charts in App/Front end, so don't worry your self too much about that, find something stable, build well documented and maintainable system.
+ extra points if you use some kind of Api/Data provider layer so you can use it for both reports and Frontend, and switch implementation at any point.
I am currently developing a webapp aimed for mobile devices which basically consists of a couple of long and complex lists, lots of data, collapsible cascading elements.
I'm getting the data to be displayed as XML, now as I see it I have two options:
build the list on the server and send HTML to the client
send XML to the client, build the list with Javascript/jQuery
Not sure which is more efficient, less data to transfer is good, less load on (especially older) phones is also good.
Any other pros/cons I'm not seeing? Suggestions?
The less data you send the better.
I often prefer to make a one-page application. All of the data needed is loaded via AJAX or Web Sockets (check out Socket.IO for a nice wrapper, with compatibility for mobile browsers that don't directly support Web Sockets). That way, you can have a much smoother user experience, and save bandwidth while doing it.
The con with this method is SEO. Search engines won't be running your JavaScript, and therefore won't index any data you are displaying. For "application" like sites, this is typically not a problem. If your site is more content based, then it might be an issue for you. There are ways around this, and progressive enhancement goes a long way to helping the problem. You will need to decide if this is right for you and your situation.
The presentation of Nicholas C. Zakas on velocity 2013 can help you.
Enough with the JavaScript Already. The peroformace or js rendering is very bad, especially for mobile webapp. We have changed the render of data list form js to php in some appliction. You should try to compare the performace or these two ways, and choose the better one. I recommend you do the render of big list in php, just for performce.
I am looking to create a web based GPS skyplot.
Currently just in the planning stages. Was wondering if anyone knew any javascript libraries or such in which this would be best to start with or have a look at, of if you have done something similar before. I know Flot is decent for plotting but unsure if it would be easy in this scenario.
TLDR: Looking for javascript libraries or other alternatives to create a web based GPS Skyplot.
Further info.
I would like to be able to maintain it all generally in javascript / HTML but am open to other alternatives. I will have a server passing me the GPS details.
Skyplot Examples: For those who don't know.
http://www.gpsoftnav.com/skyplot.html
http://www.novariant.com/resources/technologies/skyplot.cfm#
I came accross:Geographic Information Systems (Stack Exchange) yesterday, would make a good site for these kind of questions.
So if anyone is interested in this please also go make a commit :). Otherwise I am still looking for any feedback on the question.
Is it the graphical issues you are concerned with, or are you intending to also calculate which satellites are visible, and their azimuths and elevations at the given location and time?
If it's just the graphical part you are concerned with, I would use jQuery and Keith Wood's SVG plugin - see http://keith-wood.name/svg.html . This gives you a Javascript handle to Scalable Vector Graphics, for drawing circles, lines and dots on a canvas in the web page. It's simple to use, and the page I've linked to has examples that should help.