Looking for advice on how to best structure d3 v4 plugin for the visualization of hierarchical data-sets, currently working on.
Want to visualize a network consisting of several subnets. The subnet nodes and edges are displayed in a tree structure and their positions are arranged by an algorithm (see left side of the example). Each subnet is then placed by another algorithm in a way to prevent overlapping (see right side of the example). The following link shows an abstract example.
Example
The goal is to develop a d3 plugin that visualizes all subnets and displays them in a complete connected network. An early prototype in JavaScript already exists and need to transfer the two parts (the algorithm for positioning the nodes / links of the subnets and the placement of the subnets in a certain areas) into a d3 plugin.
One idea to visualize the subnets is to modify the d3 tree layout module by replacing the algorithm to position the nodes and links. Not sure whether it's better to develop two d3 plugins for the two layers, which together build the visualization or to do this in one single d3 plugin.
Does anyone have experience developing d3 plugins on several layers and can help? How could a possible architecture for such a d3 plugin look like?
I'm designing a GUI and want to find a library on the JVM or for javascript that I can use with Clojure or Clojurescript.
The problem:
I need to display 'zoomable' directed acyclic graphs (DAGs). A picture is worth 1000 words, so here is what the interface ought to look like, and how the screen changes under 'zooming':
Specifically, every node can have its own internal graph, with the restriction that each inner graph has the same number of inputs and outputs as the node itself.
My experience with graphics is limited in Javascript and nonexistent in Java, but...
My ideas:
Naively draw the whole graph with all 'inner graphs' already inside the nodes, and fade the inner graphs out for text when the zoom level is sufficiently far away. Use click events to change the zoom to and from this level. I don't think this will behave well when there are several nested inner graphs.
Draw the unzoomed graph and zoom into a node when clicked so that its background fills the screen, then overlay the inner graph. For nested inner graphs do the same trick, maintaining a 'background node' and 'inner graph'.
Is option 2 sensible? If so is there a library (ie graphviz) I can modify to achieve this behaviour? If there isn't a library, what framework (ie JavaFX) should I use?
If you don't care about struggling with Java interop syntax, I can highly recommend FXDiagram:
Selected features:
Modern graphics [based on JavaFX]
Suggestive metaphors
Optimized for usability
Multi-touch gestures
Smooth transitions/animations
Animated auto-layout (based on KIELER and Graphviz)
Animated undo/redo
JSON persistence
SVG export
Xtext/Eclipse integration
Graphical context menu
…and much more
It's open source (and active), written in Xtend and easily integrates models expressed in Xtext-based DSLs.
we're currently building a web application (AngularJS) which needs to draw directed acyclic graphs (up to 1000 nodes, up to 10000 edges) based on dynamically created data.
Since over a year I am looking for a library/tool which is calculating the required layout and drawing a graph which can be styled, is zoom and panable, interactive (e.g. highlight children on mouse over).
Graphviz is the tool which produces the best results but it's not really ready to be used on webservers (especially as I cannot guarantee the OS and don't want to).
I tried dagre and it's d3 rendering - and I like it very much, but it has two major drawbacks: 1) it doesn't really support ranking and clustering - which makes the output rather chaotic and 2) with the graphs getting bigger its performance is getting unacceptable.
Next thing which really looked convincing was cytoscape.js as the output looks very nice and it's rather fast in drawing even larger graphs (and allowing some performance tweaking). But its standard layouting (e.g. cose or breadthfirst) doesn't produce the output we're requireing.
From my current point of view a have two chances:
1) Create layout with dagre.js and drawing the result with cytoscape.js (layout: 'preset', using the calculated x and y for nodes from dagre layout). But 'compounds'/clusters are not supported that way.
2) Using [viz.js](https://github.com/mdaines/viz.js) (an emscripted Javascript version of Graphviz, again not really performing well in drawing the graphs) to calculate the result as output format plain and again using this result to draw it with cytoscape.js.
Now my question(s):
1) Do you have another idea how to achieve it?
2) Could you give me any hint on how to ideally link the chain if my ideas are correct?
(AngularJS -> REST Backend -> JSON to Frontend -> Restructure JSON for dagre or viz -> Calculate layout -> input result to cytoscape -> render in browser?!?) and is there a chance to do the layout calculation on my node.js frontend server and not the client itself (again due to performance)?
Any hint and idea is heaviley appreciated.
It sounds like you're dissatisfied with Dagre as a compound layout. That makes sense, because the author of Dagre did not implement compound support for it. Dagre is separate from Cytoscape.js but can be used by it. You'll note the author of Dagre has stated he will no longer update the library with new features from himself. Thus, your options are:
(1) Use another Cytoscape.js layout actually designed for compound nodes, like CoSE or Cola.
(2) Write your own layout for Cytoscape.js that meets your needs: http://js.cytoscape.org/#extensions/layouts
(3) Adapt Dagre to recognise compound nodes and make a pull request to the author: https://github.com/cpettitt/dagre Then the Cytoscape.js layout for Dagre can be updated to use the new Dagre APIs that you add.
Currently, compound graph layout is an open research area -- so you'll be hard pressed to find many layouts that support them at all in any library. Cytoscape.js has some compound supporting layouts, and you're free to fork them if you want different behaviour. Also note that the 2.5 branch has CoSE2 with an improved algorithm.
Cytoscape.js can use a dagre layout. You can look here in the cytoscape docs for more information, but you can just add your nodes and edges to the graph and then run the dagre layout. That way you don't have to position all of the nodes manually based on the position of nodes from the separate dagre.js.
I have a 11MB JSON graph file with about 45K edges and 73K nodes without x, y locations, and I want to display this graph using the BFS layout. I am using promise/deferred to load the file. I haven't been able get Cytoscape to display this graph on chrome. So:
Are there some special techniques for displaying large graphs?
What is the largest graph anyone has displayed using cytoscape.js?
If cytoscape.js won't work are there other JS frameworks that will work for large graphs?
You are limited by the performance of the browsers themselves. Cytoscape.js uses several techniques to optimise rendering performance, but you'll still hit the ceiling of the browser's performance.
I don't think you'll find any browser tech today (July 2015) that supports rendering such large datasets.
We display a graph from a 5.4 MB JSON file with predefined coordinates on different browsers with great performance. Is there a specific reason for not precalculating the coordinates (e.g. in Cytoscape desktop)?
To increase rendering performance:
use haystack edges
provide "min-zoomed-font-size" for nodes and edges
hide non-selected edge labels
use batches for series of operations
I am aiming to build a site that will contain a lot of user generated data, hopefully.
I'm in my first year of self learning programming: Python, Django, MySQL, HTML and Javascript.
I can chart dummy data on a table just fine, but I'm now looking at turning that data into nice colorful looking graphs.
I am in my first day of investigation into finding out how to do this. But before I continue, I would like to ask a few questions.
There seems to be many JavaScript frameworks for building charts, such as Google charts and jquery charts, and some object orientated programs for building charts, such as Cairo Plot and matplotlib.
The Javascript frameworks seem initially like a nice easy way to do it. However, whereas with tables, where you can enter variable data tags in the body of an HTML page, and have Javascript make it look pretty, the data of a graph goes in the scripting area, where the variable data tags don't quite seem to work the same way. I'm using Django, so a variable tag looks like:
{{ uniquenum }}
Q1. Should this work and am I just doing it wrong, or am I right in thinking variable tags can't go in the scripting area?
Q2. Can you have Javascript frameworks produce graphs from data outside the <script> area?
Q3. I've read that Javascript frameworks are getting more powerful, but because I'll be potentially using large amounts of dynamic data, should I be concentrating on using OO style graph programs like Cairo Plot and matplotlib, which to me don't seem to have the same levels of support?
Just looking for a nudge in the right direction.
How are the plots (typically) placed on the web page?
Here's the usual API schema for javascript-based data visualization libraries:
i. pre-allocate a div as the chart container in your markup (or template); typically using an id selector using an id selector, like so:
<div id="chart1"> </div>
Often these libraries also require that this container be pre-sized--styled with height and width properties e.g.,
<div id="chart1" style="height:300px;width:500px; "></div>
HTML5 libraries are particular about the container--i.e., they require the chart to be placed inside the canvas tag, e.g.,
ii. call the plot constructor from your included javascript file and pass in (i) a data source; (ii) aesthetic options (e.g., axis labels), and (iii) the location in your markup of the container that will hold the plot; this location is usually expected to be in the form of an id selector. So in jqplot for instance inside the jQuery ready event,
plot1 = $.jqplot('chart1', [dataSet1, dataSet2], chartOptions)
javascript-based data visualization libraries i recommend (based on having used each for multiple projects).
I. conventional plotting formats: bar, line, point, pie
Among javascript plotting libraries, i recommend the jQuery-based options because you need less code to create yoru plots and because it's easier to use jQuery's AJAX methods to load your data, for instance, jqplot, flot, and HighCharts (the three libraries that i recommend below) all include in their basic distribution, complete (html, css, js) example plots that demonstate loading data via AJAX.
HighCharts (open source but requires paid license for commercial use, but the most polished and longest feature list; active and fairly high signal-noise ratio forums on the HighCharts Site)
flot (perhaps the most widely used)
jqplot (large selection of plot types, highly modular, e.g,. most functinality beyond the basics is added one plugin at a time)
II. graphs, trees, network diagrams, etc.
d3 (the successor to protovis; stunning graphic quality, rich interactive elements, animation; not strictly jQuery-based, but the author clearly borrowed the basic syntax patterns from jQuery; excellent tutorials on d3 by an accomplished data visualization specialist, Jan Willem Tulp Unlike the others mentioned here, this is a low-level library; indeed there are (at least) several plotting libraries based on d3, e.g., rickshaw by shutterstock, and cube by Square. If you want conventional x-y line/bar plots then for instance, you can build your plots in e.g., HighCharts much faster. D3 becomes more interesting as use cases become more specific--in particular animation and unorthodox visualization (sunburst diagrams, chord diagrams, parallel line plots, geographic maps, etc.)
RafaelJS, renders in SVG, along with d3 and processing.js, you can make just about anything (e.g., two-player games in the browser) with this library; gRafael is a separate library for creating the usual plot types (bar, line, pie)
III. time-series plots
dygraphs (a javascript library dedidated soley to time-series plotting, and its feature set reflects this mission, e.g., capacity
to process and render plots with high volumes of data (>10,000
points), wide range of opdtions for tick labels of time axis with
many formatting options
HighStock (a time-series library from the HighChart guys)
IV. real-time/streaming data
Smoothie Charts (sparse feature set, only intended to do one thing well which is smoothly render streaming data; HighCharts, jqplot, and flot will also do this, but depending on the character of your data (variance, rate) these three general-purpose libraries might not show the data as "spiky", which is precisely what Smoothie was designed to eliminate)
If you're going to deal with very large datasets (>10000 elements), you will probably run into performance problems, no matter what Javascript library you end up picking.
Having said that, there's a growing number of Javascript toolkits which dynamycally load the data set with an HTTP request as a JSON, XML, etc... flot is pretty fast and open source. Highcharts is very feature rich and free for non-commercial projects. And if you need more esoteric visualizations, you must take a look at d3.js.
I have found another javascript charting library to be useful for streaming data = https://github.com/INRIA/VisualSedimentation. The code is baed on d3.js, but has some good extensibility.