Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I'm about to create a GUI for my Raspi-Project. There is Nodejs running on the Raspi3 which runs a NodeJs-Server and a then it gets requested with Chromium in kiosk-mode.
One page of this GUI needs to visualize the states of 48x potentiometers, 12x buttons, 8x faders. The NodeJs-Server sends data (which was modified by user) via websocket to the client, which redraws the whole canvas. Works fine so far for a few elements:
works fine but with a slight delay if you look closer
Now the problem is, that whith the growing number of elements that need to be drawn, the performance drops down to unacceptable dalay-times.
works, but with a way too big dalay, as more elements are drawn
and those are not even the half of stuff that needs to be drawn.
I am confused now, because I read about how fast canvas is, before I decided to go that way, and if I deactivate all canvas-drawings and simply console.log() the data that comes in via websocket, it is fast like in realtime.
so what am I doning wrong? maybe there it would be better not to draw the whole canvas on every value-change but animate the canvas? maybe someone has experience on this?
Here is the code.. when you look into assets/js/menu.class.js, this is the file which generates the canvas. the function createControllerGUI(options) is called every via websocket, every time a value changes.
Canvas is quick, but still cpu intensive. Also speed changes with the platform a bit.
Your function does all the drawing operation each change. Those operations have strokes, fills, center aligned text and something more ( i did not look all in details ).
There are some ways in which you can optimize the drawing operations.
partial redrawing
maybe the most effective.
Keep track of where a widget is, keep track of what data changed from message to message and draw only the differences.
Use clearRect on the area occupied by the widget and redraw it. Do not touch the other pixels.
Unless an octopus is using the hardware, you will have 2 or 3 widget changing per frame at maximum.
stroke all at once.
Instead of stroking on a per widget basis, you can trace all the paths you need at once, using a moveTo to the new position when changing widget, and using a single stroke operation at the end of the loop.
caching
If you have some rotatory controls for example, you can draw them once on a small separate canvas, and use that canvas as a source image to be drawn at a different angle if you need to represent a rotated control.
DrawImage is often optimized with hardware operations while the single fill and stroke may not.
There are probably other ways, and you can look at high level libraries that can do this for you, exposing a widget logic instead of the low level drawing operations.
Related
I'm rendering a canvas with a lot of elements. Because the high amount of elements and the complex shapes of the elements the elements are not displayed instantly, but are displayed with a delay (sometimes more than 10 seconds). During this time the whole application freezes and no loading indication is shown. Is it possible to somehow show a loading indication whenever the canvas is not completely rendered yet?
In really depends on the app itself, drawings, and general architecture of your app.
That task can be very complex in some situations. There are some options:
1. Use a webworker
That is probably the best approach. You need to run a webworker and make all drawings into an offscreen canvas. It will be simple if you can use 2d native API for all drawings. If you are using Konva, take a look here: https://konvajs.org/docs/sandbox/Web_Worker.html#page-title
I don't know any ways to run a react-konva application inside a webworker
2. Use incremental drawing
You need to check where the slowness comes from. From rendering? From creating too many objects? If the js thread is spending too much time to just initialize the nodes, you can create them in steps. Like create 100 nodes -> draw canvas -> wait a bit -> create 100 nodes more -> etc. That way the drawing will slowly appear on the screen and the UI probably will not be frozen.
3. Optimizing drawings
There are many tips here: https://konvajs.org/docs/performance/All_Performance_Tips.html#page-title
I am using bokeh as a Server Application. When I make a selection in a plot I do some actions in python and I update some sources (CDS). This changes are reflected in the plot. Is there a way to check when the glyphs are completely rendered (after the update)? I want to call a JavaScript function when all is completely loaded? With that function I want to call other python method to update the CDS again.
If I do not wait for this profiles to be rendered probably the application breaks, and that´s what I want to avoid. Actually I did some tests in the past and I had to create a huge CDS instead of several smaller CDS to make it work properly.
My Use Case. Why do I want to make this?
I have many tabs in my layout, they can be 10 for example. And each tab has some plots (3-6 plots). If I update the entire ColumnDataSource at the same time, it will take a while. Then I want to make it more fluent, so I would like to update only the data of the current visible tab, it will render faster and the user would receive an immediate response. I can disable the rest of the tabs temporarily to prevent malfunctions. At this moment I would need to call the JS or python method in order to update the content of the rest of the tabs.
Here a drawing of what I want to achieve in order to speed up the process:
About the data
Basically I have two DataFrames, one to build the cloud of points (around 5000 row and 130 columns) and I extract from the selected points another DataFrame to know which lines I should draw (360 columns and 5 to 15 rows), making some filters and selections. The algorythm I have used is in the answer of a question I have written time ago. With this amount of data the algorythm takes 6 or 7 seconds to finish.
Any other idea of how to improve the performance or how to split or the computing?
To improve the rendering speed you could try the webgl JavaScript API. This Bokeh documentation page Speeding up with WebGL explains how to do it. webgl supports circles, lines and most of the markers. Application:
p = Plot(output_backend="webgl") # for the glyph API
p = figure(output_backend="webgl") # for the plotting API
Please be aware that users report issues with webgl like plot stuttering, etc... but it may work in your case depend on which type of glyphs your plot contains.
Also make sure your data passed to the plot doesn't include NaN's as it is known to slow down Bokeh performance.
To my knowledge there is no attribute that indicates that rendering is completed or is still ongoing but you may think about some other alternatives to speed things up like combination of Bokeh with Datashader (pre-rendering large datasets into a fixed-size raster image) or Dask (speed up data reading from multiple sources like multiple csv files)
For example you could have one standard Bokeh plot where you make a
selection and let the other plots being generated as Datashader images
and embed them in Bokeh plots.
This example shows how to combine Bokeh + Datashader which significantly improves performance especially when over-plotting takes place. Please note that each time a single point is added to the plot entire canvas area will be re-drawn in the browser. This is how browsers work. Datashader can provide a single image so updating the plot is much quicker while you can still use Toolbar tools like zoom, pan etc....
Also the Python code implementation details counts. Using e.g. gridplot to link many plots can slow down performance so it is better to add them one by one to the document root, etc...
Time ago I made a trick to check if my design would work if I could trigger some function if the plots were rendered:
First I updated the current tab. This worked very well and fast.
Then I set a timeout to update the data of the rest of the tabs. But, in the meantime this second algorythm was being executed I could not work with the plots of the current tab because they were frozen.
So, the approach of triggering a function when everything is rendered is not a good idea, because even with such a callback the app would not work as I was expecting.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am creating a piano roll like interface, one like you might find in a DAW such as ableton, that looks something like this http://www.abletonlife.com/wp-content/uploads/2010/04/midi-track-big.jpg . The grid represents a canvas to draw in notes to be played, the red squares being the notes to play. You double click on an empty space to create a new note, and you can drag the edges to change the length of the note.
I am new to web dev so I am having a bit of trouble seeing what the right architecture for this might be. With my limited knowledge, the following are the architectures I can think of.
1) Rows of horizontal flex-boxes.
display: box;
box-orient: horizontal;
box-flex: 1;
Something like this, http://jsfiddle.net/ZgzNw/.
Pros:
When resizing the browser window, the browser will automatically
handle resizing of the notes and therefore the grid. Resizing of divs/notes also handled easily for zooming in and out and changing quantization values.
All notes in all positions already exist, when double clicking to "create" a new note, all you have to do is change the css for that note (to be red) etc.
Cons:
Since there is a div for every space in the grid, even empty spaces where there is no note to be played, there will be a lot of divs. Can the browser handle thousands of divs? As an extreme example if there is a 32nd note quantization, a song of 200BPM would have 50 measures per minute, take a 10 minutes song, that would be 500 measures. Going back to the jsfiddle example above and setting measures=500 and quant=32, I get the following error in the Chrome Developer tools console after a few seconds "Uncaught RangeError: Maximum call stack size exceeded". This is when creating the divs in that bit of javascript, If I lower the number to around 300 it is able to create the divs, but things become laggy.
2) Create the grid using divs of width=1px for the vertical lines of the grid. Create new note divs on the fly, position them manually (with position: float?) based on the position of the mouse click.
Pros:
Only have a divs for actual note that are on, so don't have the con of method 1) being an issue
Cons:
Have to manually compute everything, Where to place newly created note, zooming in/out means repositioning vertical markers for grids, and calculating new sizes for note divs. This was mostly handled automatically in the method 1.
I'm sure there are a lot more architectures and pros/cons to the two methods I describe, but I've never created any web applications and the extent of my web-dev experience is the tutorials I've done over the last 2 weeks to teach myself.
My question I guess is what is the best architecture for creating this piano roll interface I am describing? Specifically the UI representation, not the backing model.
I would create a simple model (probably just a multi-dimensional array) to contain the representation of the score, where each array item represented a note at a point in time. From the model, you can then draw/redraw accordingly. You could also perform operations like time shifts, quantizing, thinning, etc. by modifying the arrays.
Can the browser handle thousands of divs?
A few thousand, yes, maybe more--maybe even a lot more--but results will greatly vary by browser, by computer, even by the way they are positioned (floats tend to be slower than absolute positioning, for example, because the browser has more to calculate).
Instead, I would research using a canvas and draw/redraw based on your underlying model. You can detect events on the whole canvas, and depending on coordinates, easily map the event to the note(s) to which it corresponds.
KineticJS has some cool examples using a canvas
I would use a background image to represent tracks and measures, and one div for each note. It is not necessary to keep all notes as div’s in the browser, it would work better only to have the div’s for the visible up to 10 measures.
Imho, this should be done fixed-width, and there should not be any resizing of the piano roll at all. So you can use calculation of pixels. I don’t think this will work without making use of Javascript heavily.
I have a project I'm planning which based on kind of an 'interactive world' style experience where the browsers viewport moves around to show many different graphic environments, it must all be fluid and no page-to-page breaks. The project is in js/html5/css3
The problem this poses is that the entire 'world' will be perhaps 8-15,000 px squared (it also rotates, and has various png alpha overlays on top of it)
I was going to run some tests but there are so many ways to approach this and I'm looking for the most fluid one. My knowledge of the internal workings of browser render engines isn't great so I thought I'd ask around.
I cant use the 'tiling' approach which google map uses as it's not fluid enough (too blocky) also when rotating around it's going to create headaches do the math-transforms to work out which tiles to load at what angles so here are the 2 choices I have boiled it down to:
(1) The "Huge" image approach
The benefit of this is that once it's loaded everything is easy, the downside is that it's going to be huge and I cannot show an incremental preloader as the image queue will essentially be 2 images (overlay and huge img)
(2) Image segments
The benefit is that I can show a preloader with an image queue at 10% increments (10x images)
Question:
is the 2nd approach going to have a more painful overhead on the browser's rendering engine due to there being 9 separate sets of calculations being done or do browser engines simply see them as one painted area once it's initially rendered and then update it as a whole? Or each time the dom is changed (rotated etc), the browser has to run the same transform/repaint process 9 times?
Thanks very much.
LOTS of tests later: result: Use a big image, seems to be less for the browser to deal with.
I'm trying to build something in HTML5/Canvas to allow tracing over an image and alert if deviating from a predefined path.
I've figured out how to load an external image into the canvas, and allow mousedown/mousemovement events over it to draw over the image, but what I'm having trouble getting my head around is comparing the two.
Images are all simple black on white outlines, so from what I can tell a getPixel style event can tell if there is black underneath where has been drawn upon or underneath where the mouse is on.
I could do it with just the mouse position, but that would require defining the paths of every image outline (and there are a fair number, hence ideally wanting to do it by analyzing the underlying image)..
I've been told that its possible with Flash, but would like to avoid that if possible so that compatability with non-flash platforms (namely the ipad) can be maintained as they are the primary target for the page to run.
Any insight or assistance would be appreciated!
I think you already touched upon the most straight-forward approach to solving this.
Given a black and white image on a canvas, you can attach a mousemove event handler to the element to track where the cursor is. If the user is holding left-mouse down, you want to determine whether or not they are currently tracing the pre-defined path. To make things less annoying for the user, I would approach this part of the problem by sampling a small window of pixels. Something around 9x9 pixels would probably be a good size. Note that you want your window size to be odd in both dimensions so that you have a symmetric sampling in both directions.
Using the location of the cursor, call getImageData() on the canvas. Your function call would look something like this: getImageData(center_x - Math.floor(window_size / 2), center_y - Math.floor(window_size / 2), window_size, window_size) so that you get a sample window of pixels with the center right over the cursor. From there, you could do a simple check to see if any non-white pixels are within the window, or you could be more strict and require a certain number of non-white pixels to declare the user on the path.
The key to making this work well, I think, is making sure the user doesn't receive negative feedback when they deviate the tiniest bit from the path (unless that's what you want). At that point you run the risk of making the user annoyed and frustrated.
Ultimately it comes down to one of two approaches. Either you load the actual vector path for the application to compare the user's cursor to (ie. do point-in-path checks), or you sample pixel data from the image. If you don't require the perfect accuracy of point-in-path checking, I think pixel sampling should work fine.
Edit: I just re-read your question and realized that, based on your reference to getPixel(), you might be using WebGL for this. The approach for WebGL would be the same, except you would of course be using different functions. I don't think you need to require WebGL, however, as a 2D context should give you enough flexibility (unless the app is more involved than it seems).