vis.js graph not stabilizing even after hours - javascript

I have a network of around 1000 nodes. I have set stabilize:true and zoomExtentOnStabilize: true. The nodes are being added from JSON using vis.network.gephiParser.parseGephi() function. When I tried to plot this graph it never stabilizes even after hours of letting it idle. But then smaller number of nodes stabilize in reasonable time. What am I missing here. Is there any way to stabilize big graphs. I even tried setting the number of iterations to stabilize to 1000 and even higher. Thanks in advance for the help.
P.S.:The coordinates of the nodes are not available from JSON. The graph is redrawn based on the user input.
EDIT 1:
The JSON data being plotted is available at http://pastebin.com/raw.php?i=Mzy4ncxw. I couldn't make a reproducible example at jsbin because of CORS error.
The JavaScript code is:
message = JSON.parse(json_data); // json_data is sent from R server.
var nodes = new vis.DataSet();
var edges = new vis.DataSet();
var container = document.getElementById("div_graph");
var data = {
nodes: nodes,
edges: edges
};
var options = {
tooltip: {
delay: 50,
fontColor: "black",
fontSize: 14,
fontFace: "verdana",
color: {
border: "#666",
background: "#FFFFC6"
}
},
clustering: {
enabled: clusteringOn,
clusterEdgeThreshold: 50
},
hideEdgesOnDrag: true,
stabilize: true,
zoomExtentOnStabilize: true,
navigation: true,
keyboard: true,
edges: {
inheritColor: "to"
}
};
var network = new vis.Network(container, data, options);
nodes.clear();
edges.clear();
var parsed = vis.network.gephiParser.parseGephi(message);
nodes.add(parsed.nodes);
edges.add(parsed.edges);
network.redraw();

I'm the developer of the network module of visjs. we have used it to stabilize much larger sets than 1000 nodes. I can't really say what's going wrong here based on the information you supply. I'd like to invite you to make an issue on our github page. We try to collect all questions there. Can you share the code you use or your data (labels scrambled for anonymity ofcourse).
If I were to guess, a 1000 node system would stabilize with about 3000 iterations. If you are using dynamic smooth curves this increases greatly as support nodes are added to position the curves. I have used 15000 iterations for a 3000 node and 25000 edge system and even then it is not finished but I stop the simulation at that point regardless.
When you say redrawn on user input, is the data reloaded or redrawn in the sense that you see the dragging or zooming (similar to the redraw function)?
~ Alex
EDIT:
Based on your data I encoutered a few problems.
First, it seems you do not allow the nodes to move but also do not supply their positions, leading to an infinite recursion in the quadtree building process. I'll make the gephiParser more robust for this in the future.
See here for settings of the gephi parser: http://visjs.org/docs/network.html#Gephi_import
Secondly, You use dynamic smooth curves and a lot of interconnected nodes. Each smooth curve has an invisible support node that helps the positioning. This makes your system unstable (look at it with stabilize of to see the behaviour). In the v4 version you can set your own timestep to rectify this, but alternatively you can change your physics settings. Try the configurePhysics option and see if that helps. You can still use static smooth curves for aesthetic purposes.
To wrap up, I could get your system to stabilize with static smooth curves in about 3000 iterations, taking about a minute. I disabled clustering in your options. I'd recommend you wait for the 4.0 release to use clustering as it will be much much more powerful.
EDIT 2:
Here is a JSBin showing a working stabilization with your code and data (although modified)
http://jsbin.com/tiwijixoha/5/edit?html,output
So if you ment that it does not stabilize in the sense that it does not hide itself and only shows when it is ready instead of never reaching a stabilized state, then the problem is that stabilization is only done with a setData(), not with a dataset update.
In this jsbin I have also changed your edges and altered the physics to make it stable. You can play around with it a bit more if you're unhappy with it.

Related

Optimizing smooth tween between svg paths in JavaScript / React Native

I'm currently porting an application to React Native that captures user input as a stroke and animates it to the correct position to match an svg (pictures below). In the web, I use a combination of multiple smoothing libraries & pixijs to achieve perfectly smooth transitions with no artifacts.
With React Native & reanimated I'm limited to functions I can write by hand to handle the interpolation between two paths. Currently what I'm doing is:
Convert the target svg to a fixed number N of points
Smooth the captured input and convert it to a series of N points
Loop over each coordinate and interpolate the value between those two points (linear interpolation)
Run the resulting points array through a Catmull-Rom function
Render the resulting SVG curve
1 & 2 I can cache prior to the animation, but steps 3 4 & 5 need to happen on each render.
Unfortunately, using this method, I'm limited to a value of around 300 N as the maximum amount of points before dropping some frames. I'm also still seeing some artifacts at the end of an animation that I don't know how to fix.
This is sufficient, but given that in the web I can animate tens of thousands of points without dropping frames, I feel like I am missing a key performance optimization here. For example, is there a way to combine steps 3 & 4? Are there more performant algorithms than Catmull-Rom?
Is there a better way to achieve a smooth transition between two vector paths using just pure JavaScript (or dropping into Swift if that is possible)?
Is there something more I can do to remove the artifacts pictured in the last photo? I'm not sure what these are called technically so it's hard for me to research - the catmull-rom spline removed most of them but I still see a few at the tail ends of the animation.
Animation end/start state:
Animation middle state:
Animation start/end state (with artifact):
You might want to have a look at flubber.js
Also why not ditch the catmull-rom for simple linear sections (probably detailed enough with 1000+ points)
If neither helps, or you want to get as fast as possible, you might want to leverage the GPUs power for embarrassingly parallel workflows like the interpolation between to N-sized arrays.
edit:
also consider using the skia renderer which already leverages the gpu and supports stuff perfectly fitting your use-case
import {Canvas, Path, Skia, interpolatePath} from "#shopify/react-native-skia";
//obv. you need to modify this to use your arrays
const path1 = new Path();
path1.moveTo(0, 0);
path1.lineTo(100, 0);
const path2 = new Path();
path2.moveTo(0, 0);
path2.lineTo(0, 100);
//you have to do this dynamically (maybe using skia animations)
let animationProgress = 0.5;
//magic already implemented for you
let path = interpolatePath(animationProgress, [0, 1], [path1, path2]);
const PathDemo = () => {
return (
<Canvas style={{ flex: 1 }}>
<Path
path={path}
color="lightblue"
/>
</Canvas>
);
};

How do I create a leaflet map with thousands of marks that doesn't crash my browser?

I'm using the leaflet package in R to generate a map with a large number of circles on it. The goal is a map I can publish to my website. The problem I'm having is that as I increase the number of circles, the resulting map loads very slowly, I get "unresponsive script" warnings and ultimately it completely freezes up my browser.
I know this sort of thing is possible, because I've found a leaflet map that works the way I want mine to work:
http://cartologic.com/geoapps/map_viewer/5/ny-crimes-2014-dot-density-map
I notice on the above map that the circles don't appear "clickable" like the circles on my map, and that they seem to load in square chunks. I have a hunch that these things are related to my problem. Unfortunately, I'm too much of a novice at leaflet/javascript stuff to figure this out on my own.
Here is a toy example illustrating my problem:
library("leaflet")
library("htmlwidgets")
dots <- data.frame(x=c(runif(10000, -93.701281, -93.533053)),
y=c(runif(10000, 41.515962, 41.644369)))
m <- leaflet(dots) %>%
addTiles('http://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}.png') %>%
setView(-93.617167, 41.580166, zoom = 12) %>%
addCircles(~x, ~y, weight = 1, radius = 5,
color = "#FFA500", stroke = TRUE, fillOpacity = 0.1)
m
saveWidget(widget = m, file="example.html", selfcontained = TRUE)
mapview can help you here. It builds upon the leaflet library for smaller data sets, but uses special javascript functionality for larger data.
your example with 1 Mio. points:
library(mapview)
library(sp)
dots <- data.frame(x=c(runif(1000000, -93.701281, -93.533053)),
y=c(runif(1000000, 41.515962, 41.644369)))
coordinates(dots) <- ~ x + y
proj4string(dots) <- "+init=epsg:4326"
mapview(dots)
It may still take a while to render, but once rendered it should be quite responsive. Note that mapview is designed to work with spatial* objects, that is why we need the calls to set the coordinate slot and the projection.
For more information have a look here:
http://environmentalinformatics-marburg.github.io/web-presentations/20150723_mapView.html
Hope that helps.
If you want to add a large number of vector objects to a map, it is rare that it can be done easily.
Notice that the raster data is broken into tiles so that all the information does not have to be shown at one time. For your vector data (in this case, circles) you have to do the same thing.
Basically what I like to do is to break the large data set into smaller (vector) tiles, with the same boundaries as the raster tiles you are showing. Duplicate the data if you want it to appear at several zoom level. As you are showing circle, imagine that you partition the circles' center points on the tile boundary.
I have an application similar to this where I basically partition my vector data on tile boundaries and store the information in geojson files. When I get an event that the raster tile has been loaded I can then load the equivalent vector file as a geojson layer (same thing when the raster tile is unloaded). In this way, you can limit the amount of vector data that has to be displayed at any one time.
If you have a lot of points, they are not really going to be visible at low zoom levels anyway, so it might be better just to show them at an appropriate zoom level (perhaps with a different representation at low zooms - like a heat map). This will keep the amount of data being shown at any one time lower.
Since this question has a few upvotes, I'll generally describe both of the solutions I found. Maybe if I have time later I'll get all the files together on GitHub.
First, I found TileMill. Simply load a data file of coordinates into TileMill, style the way you want them to appear, and output tiles (png). Host those tiles on the web somewhere and load them with leaflet. This process was a bit too manual for my liking because TileMill kept crashing when I loaded in csv files that were too large for it to render on my machine.
I found the best solution was use Processing, adapting Robert Manduca's code here: https://github.com/rmanduca/jobmaps. I don't use Python so I rewrote those parts in R and modified the Processing code according to my specifications.
Mapdeck (released on CRAN Aug 2018) uses WebGL (through Deck.gl) and is designed to handle millions of points (depending on your system's hardware of course)
library(mapdeck)
set_token("MAPBOX_TOKEN")
n <- 1e6
dots <- data.frame(x=c(runif(n, -93.701281, -93.533053)),
y=c(runif(n, 41.515962, 41.644369)))
dots$letter <- sample(letters, size = n, replace = T)
mapdeck(
style = mapdeck_style('dark')
) %>%
add_scatterplot(
data = dots
, lon = "x"
, lat = "y"
, fill_colour = "letter"
, radius = 5
, fill_opacity = 50
, layer_id = "dots"
)

Multi-tempo/meter js DAW

Has anyone implemented a javascript audio DAW with multiple tempo and meter change capabilities like most of the desktop daws (pro tools, sonar, and the like)? As far as I can tell, claw, openDAW, and web audio editor don't do this. Drawing a grid meter, converting between samples and MBT time, and rendering waveforms is easy when the tempo and meter do not change during the project, but when they do it gets quite a bit more complicated. I'm looking for any information on how to accomplish something like this. I'm aware that the source for Audacity is available, but I'd love to not have to dig through an enormous pile of code in a language I'm not an expert in to figure this out.
web-based DAW solutions exists.web-based DAW's are seen as SaaS(Software as a Service) applications.
They are lightweight and contain basic fundamental DAW features.
For designing rich client applications(RCA) you should take a look at GWT and Vaadin.
I recommend GWT because it is mature and has reusable components and its also AJAX driven.
Also here at musicradar site they have listed nine different browser based audio workstations.you can also refer to popcorn maker which is entirely javascript code.You can get some inspiration from there to get started.
You're missing the last step, which will make it easier.
All measures are relative to fractions of minutes, based on the time-signature and tempo.
The math gets a little more complex, now that you can't just plot 4/4 or 6/8 across the board and be done with it, but what you're looking at is running an actual time-line (whether drawn onscreen or not), and then figuring out where each measure starts and ends, based on either the running sum of a track's current length (in minutes/seconds), or based on the left-most take's x-coordinate (starting point) + duration...
or based on the running total of each measure's length in seconds, up to the current beat you care about.
var measure = { beats : 4, denomination : 4, tempo : 80 };
Given those three data-points, you should be able to say:
var measure_length = SECONDS_PER_MINUTE / measure.tempo * measure.beats;
Of course, that's currently in seconds. To get it in ms, you'd just use MS_PER_MINUTE, or whichever other ratio of minutes you'd want to measure by.
current_position + measure_length === start_of_next_measure;
You've now separated out each dimension required to allow you to calculate each measure on the fly.
Positioning each measure on the track, to match up with where it belongs on the timeline is as simple as keeping a running tally of where X is (the left edge of the measure) in ms (really in screen-space and project-coordinates, but ms can work fine for now).
var current_position = 0,
current_tempo = 120,
current_beats = 4,
current_denomination = 4,
measures = [ ];
measures.forEach(function (measure) {
if (measure.tempo !== current_tempo) {
/* draw tempo-change, set current_tempo */
/* draw time-signature */
}
if (measure.beats !== current_beats ||
measure.denomination !== current_denomination) {
/* set changes, draw time-signature */
}
draw_measure(measure, current_position);
current_position = MS_PER_MINUTE / measure.beats * measure.tempo;
});
Drawing samples just requires figuring out where you're starting from, and then sticking to some resolution (MS/MS*4/Seconds).
The added benefit of separating out the calculation of the time is that you can change the resolution of your rendering on the fly, by changing which time-scale you're comparing against (ms/sec/min/etc), so long as you re-render the whole thing, after scaling.
The rabbit hole goes deeper (for instance, actual audio tracks don't really care about measures/beats, though quantization-processes do), so to write a non-destructive, non-linear DAW, you can just set start-time and duration properties on views into your audio-buffer (or views into view-buffers of your audio buffer).
Those views would be the non-destructive windows that you can resize and drag around your track.
Then there's just the logic of figuring out snaps -- what your screen-space is, versus project-space, and when you click on a track's clip, which measure, et cetera, you're in, to do audio-snapping on resize/move.
Of course, to do a 1:1 recreation of ProTools in JS in the browser would not fly (gigs of RAM for one browser tab won't do, media capture API is still insufficient for multi-tracking, disk-writes are much, much more difficult in browser than in C++, in your OS of choice, et cetera), but this should at least give you enough to run with.
Let me know if I'm missing something.

Using ClusterProvider in Nokia/Here Maps Javascript API, invalid cluster numbers

I'm trying to add the cluster to my application and so far, everything works.
However, the number of items in clusters seems to be invalid depending on zoom levels.
For example, I add 3 marker in a range of about 30 feets. If I'm zoomed in all the way, I see all 3 markers. If I zoom out just a few steps, I can see 2 markers plus a cluster indicating 3 items.
I attached a picture, top part of the pictures shows the problem. If I zoom in a bit, it shows the bottom part of the picture. If I zoom out more, it shows a cluster of 3.
Thanks
Try fiddling around with the ClusterProvider.Options. Obviously all clustering algorithms are an approximation of the actual data set, and maybe the particular distribution of points you have just doesn't look good at a high zoom using the defaults.
Here are three suggestions to try:
Lower the eps value to get a finer grid.
Set max and min or minPts to avoid clustering at lower levels.
Set the strategy to STRATEGY_GRID_BASED rather than use the density default.
e.g. something like this:
function clusterDataPoints(data){
clusterProvider = new nokia.maps.clustering.ClusterProvider(map, {
eps: 5,
minPts: 5,
min: 18,
strategy: nokia.maps.clustering.ClusterProvider.STRATEGY_GRID_BASED,
dataPoints: data
});
clusterProvider.cluster();
}
And keep altering the parameters until it "looks right"

Detecting irregular Shape

Leading up from this question Detecting mouse coordinates with precision, I have learnt quite a bit in the past few days. Here are what I picked as best learning resources on this topic:
http://gamedev.tutsplus.com/tutorials/implementation/quick-tip-use-quadtrees-to-detect-likely-collisions-in-2d-space/
http://www.gamedev.net/page/resources/_/technical/graphics-programming-and-theory/quadtrees-r1303
http://jsfiddle.net/2dchA/2/
The code in (3) works in JSFiddle but breaks at this section in my testing environment (VS2012):
var myTree = new Quadtree({
x: 0,
y: 0,
width: 400,
height: 300
});
with the message Quadtree is undefined in IE. FF & Chrome just gloss over it and display an empty page. I couldn't sort it out. Question 1: Can someone help out with that?
My main question:
I have a region (parcels of land like a map) with about 1500 parcels drawn in html5, not jpg or png images. It is a lot of lines of code to complete that but the rendering is great, so I am keeping it that way. I intend to have a mouseover event tell me which parcel I am standing on when the mouse stops. As you will see in the previous question referred my previous attempts were not impressive. Based on the learning I have been doing, and thanks to Ken J's answer/comments, I would like to go with this new approach of slicing up my canvas into say 15 quads of 100 objects each. However, I would like some guidance before I take another wild dive the wrong way.
Question 2: Should I slice it up at creation or should the slicing happen when the mouse is over a region, ie, trail the mouse? The latter sounds better to me but I think I can do with some advice and, if possible, some start out code. The quadtree concept is completely new to me. Thanks.
Can't help with question 1.
You should definitely build the tree as early as possible, given that the objective is to get the page to respond as quick as possible once the user clicks somewhere.
Keep the tree for as long as the user interacts with the 2d area. Updating a quad tree shouldn't be too hard, so even if the area changes contents, you should be able to reuse the existing tree (just update it).
Given the fact that your draw area is well know i see no advantage in a QuadTree over a spacial hash function. This function will give you an integer out of an (x,y) point.
var blocWidth = 20;
var blocHeight = 20;
var blocsPerLine = ( 0 | ( worldWidth / blocWidth) ) + 1 ;
function hashPoint(x,y) {
return ( 0 | (x/blocWidth)) + blocsPerLine*(0|(y/blocHeight));
}
once you built that, hash all your parcels within an array :
parcelHash = [];
function addHash(i,p) {
if (!parcelHash[i]) { parcelHash[i]=[ p ]; return; }
if (parcelHash[i].indexOf(p) != -1 ) return;
parcelHash[i].push(p);
}
function hashParcel (p) {
var thisHash = hashPoint(p.x,p.y); // upper left
addHash( thisHash, p);
thisHash = hashPoint(p.x+width, p.y); // upper right
addHash(thisHash, p);
thisHash = hashPoint(p.x, p.y+p.height); // lower left
addHash(thisHash, p);
thisHash = hashPoint(p.x+width, p.y+p.height); // lower right
addHash(thisHash, p);
};
for (var i=0; i<allParcels.length; i++) { hashParcel(allParcels[i]) };
now if you have a mouse position, you can retrieve all the parcels in the
same block with :
function getParcels(x,y) {
var thisHash = hashPoint(x,y);
return parcelHash[thisHash];
}
I'll just give you few tips in addition to what others have said.
... have a mouseover event tell me which parcel I am standing on ...
From your other messages I conclude that parcels will have irregular shapes. Quadtrees in general work with rectangles, so you'd have to calculate the bounding rectangle around the shape of the parcel and insert that rectangle in the quadtree. Then are when you want to determine whether mouse is over a parcel, you'll query the quadtree which will give you a set of parcels that might be under the mouse, but you'll have to then do a more precise check on your own to see if it indeed is.
... when the mouse stops.
From your other questions I saw that you try to detect when the mouse has "stopped". Maybe you should look at it this way: mouse cursor is never moving, it's teleporting around the screen from previous point to next. It's always stopped, never moving. This might seem a bit philosophical, but it'll keep your code simpler. You should definitely be able to achieve what you intended without any setTimeout checks.
... slicing up my canvas into say 15 quads of 100 objects each.
... Should I slice it up at creation or should the slicing happen when the mouse is over a region
You won't (and can't) do slicing, quadtree implementation does that automatically (that's its purpose) when you insert or remove items from it (note that moving the item is actually removing then re-inserting it).
I didn't look into the implementation of quadtree that you're using, but here are two MX-CIF quadtree implementations in case that one doesn't work out for you:
https://github.com/pdehn/jsQuad
https://github.com/bjornharrtell/jsts/tree/master/src/jsts/index/quadtree
The problem in question 1 probably happens because jsfiddle (http) page is trying access quadtree.js which is on https

Categories

Resources