I'm playing some notes at regular intervals. Each one is delayed by a random number of milliseconds, creating a jarring irregular effect. How do I fix it?
Note: I'm OK with some latency, just as long as it's consistent.
Answers of the type "implement your own small SoundManager2 replacement, optimized for timing-sensitive playback" are OK, if you know how to do that :) but I'm trying to avoid rewriting my whole app in Flash for now.
For an example of app with zero audible latency see the flash-based ToneMatrix.
Testcase
(see it here live or get it in an zip):
<head>
<title></title>
<script type="text/javascript"
src="http://www.schillmania.com/projects/soundmanager2/script/soundmanager2.js">
</script>
<script type="text/javascript">
soundManager.url = '.'
soundManager.flashVersion = 9
soundManager.useHighPerformance = true
soundManager.useFastPolling = true
soundManager.autoLoad = true
function recur(func, delay) {
window.setTimeout(function() { recur(func, delay); func(); }, delay)
}
soundManager.onload = function() {
var sound = soundManager.createSound("test", "test.mp3")
recur(function() { sound.play() }, 300)
}
</script>
</head>
<body>
</body>
</html>
I know this isn't the answer you want to hear, but there is no way to stop this, regardless of whether you wrote your own flash library to play sound or not.
For everyone who said "it works fine for me!" try resizing or moving your browser window as the poster's demo plays out. You'll hear more than just a subtle amount of delay. This is most noticeable in Firefox and IE, but even Chrome will experience it.
What's worse, if you click and hold the mouse down on the close box for the browser window, the sound completely stops until you release your mouse (you can release it outside of the close box and not actually close the window, FYI).
What is going on here?
It turns out that when you start resizing or moving around the browser window, the browser tries to multi-task the act of changing the window properties with the act of keeping up with the javascript going on in the window. It short-changes the window when it needs to.
When you hold down the mouse over the close box in the browser window, time stops completely. This is what is happening in smaller increments when you are re-sizing or moving the window: time is standing still in the javascript world in small, sporadic chunks (or large chunks, depending on how slow your machine is).
Now, you might say "sure, resizing the browser or holding down the close button makes the browser pause, but normally this wouldn't happen". Unfortunately you would be wrong.
It happens all the time, actually. I've run tests and it turns out that even by leaving the browser window completely still, not touching the mouse, and not touching the keyboard, backgrounds processes on the computer can still cause "hiccups", which means that for brief periods (perhaps as small as a few milliseconds) time is "standing still" in the browser, at completely random intervals outside of your control.
What do I mean by "standing still"? Let's say you have a setInterval() call (this applies to setTimeout also) running every 33 milliseconds (about 30 frames per second). Now, you would expect that after every 33 "real world" milliseconds your function would get called. And most of the time, this is true.
But when "hiccups" start happening, your setInterval call might happen in 43 milliseconds. What happened during the 10 ms? Nothing. Time stood still. Nothing on the browser was being updated. If you had sound playing, it will continue playing, but no NEW sound calls would start playing, because no javascript is being executed at all. If you had 5 setInterval() functions running, they would have all been paused for 10ms at some point.
The only way to tell that "time stood still" is to poll real-world time in your setInterval function callbacks. You'll be able to see that the browser tries to keep up most of the time, but that when you start resizing the window or doing something stressfull, the intervals will be longer than usual, but that all of your code will remain synched up (I'm making games using this technique, so you will see that all your game updates happen in synch, but just get slightly stuttered).
Usually, I should point out, these stutters are completely unnoticeable, and unless you write a function to log real-world time during setInterval times (as I have done in my own testing) you wouldn't even know about it. But it becomes a problem if you try to create some type of repetitive sound (like the beeping in the background of Asteriods) using repetitive play() calls.
My suggestion? If you have a sound that you know will loop, give it a long duration, maybe 10 seconds, and you'll be less likely to notice the hiccups (now, the graphics on the screen could still hiccup, but you're screwed there).
If you are writing a game and having the main character fire off a machine gun, don't do 10 rapid-succession calls to playSound('singleShot'), do one call to playSound('machineGunFire10Rounds'), or something along those lines.
You'll have to do some trickery to get around it, but in most cases you'll be alright.
It seems that Flash applets run in a process that is somehow less affected this whole "time freezing" thing going on in the regular browser/javascript environment, but I can still get it to happen, even on your link to the ToneMatrix example, by resizing or moving the browser window.
But Flash still seems much better than javascript. When I leave the browser alone I'd be willing to bet that Flash is not freezing for any amount of time and that intervals are always running on time.
tl;dr:
you're screwed in what you're hoping to achieve
try to deal with it using some workarounds
re-write your project in pure flash (no javascript)
wait for browsers to get better (Firefox 4 is getting a new javascript engine called JaegerMonkey which will be interesting to watch)
how do I know all this? I've done a lot of testing & logging with javascript, setInterval, and soundManager/html5 audio calls
In my comment to your question I mentioned that I don't hear the irregularity when I play your sample. That means I'm either "rhythm deaf", or that there may be something in your setup that interferes with good realtime performance. You don't mention any details of your environment, but you may have other processes running on your computer that are sucking up CPU cycles, or an older version of Flash that may not do a good job of handling sound latencies. I myself am using a recent version of Flash (10.something), whereas your parameters call for Flash 9. But maybe I should assume that if you're smart enough to be using SoundManager2 and StackOverflow that you would have eliminated these problems.
So here are some troubleshooting possibilities and comments that come to mind:
1) the SoundManager site has a number of demos, including JS-DOM "painting" + Sound, V2. Are you hearing irregular latencies and delays there? If not, maybe you can compare what they're doing there against what you're doing. If you are, then maybe look at your machine environment. When I run that demo, it is very responsive. (EDIT: Looking at it more closely, however, you can watch how the size of the brush stamps varies during a stroke. Since it varies with the time interval between mouse events (assuming you are keeping a constant mouse speed), you can visually see any irregularities in the pattern of mouse events. I can see occasional variation in stamp sizes, which does indicate that mouse events are not coming in at regular times. Which brings us to Javascript events.)
2) Javascript setTimeout() and setInterval() are not very reliable when it comes to timing. Mostly they will come back in some ballpark of the interval you have requested, but there can be large variations, usually delays, that make them unreliable. I've found that the same is true when using ActionScript inside Flash. You might want to print out the times that your sound.play() call is being made to see whether the irregularities are due to the irregularities in setTimeout/setInterval(). If that's the case, you could try shortening the interval, and then polling the system time to get much closer to the 300ms interval that you want. You can poll system time using new Date().getTime(), and this seems to have ms accuracy. Polling is of course a hideous hack that sucks up cycles when they could be used for something else, and I don't recommend it in general, but you may try it to see whether it helps. EDIT: Here's a writeup by John Resig on the handling of input and timer events in js.
3) When flash plays sounds, there is usually a latency involved, just so that the player can "build up a head of steam" and make sure there's enough stuff in the buffer to be played before the next buffer request is filled. There's a trade off between this latency and the reliability of uninterrupted playback. This might be a limitation you can't do anything about, short of "implement[ing] your own small SoundManager2 replacement", which I know you don't want to do.
Hope this helps. I recently wrote an AS3 sound experiment which exposed me to some of the basics, and will be watching this space to see what other suggestions people come up with.
You are using the javascript interval, which can not be guaranteed to fire at an exact time. I am sure that the internal Flash timing is far more reliable.
But this might help, fire recur AFTER you have triggered playing the sound.
window.setTimeout(function() { func(); recur(func, delay); }, delay);
As explained in another answer, there's no way you can avoid this. But...
I've done some experiments to mitigate this issues, and in the end I resorted to using:
lowLag: responsive html5 audio, which uses SoundManager2 for some cases where it's the fastest option available.
GSAP JS – Professional-Grade JavaScript Animation, in order to do the animation of properties and syncing of the audio (you probably don't care about this :P)
Take a peek at the source on the prototype of the demo, and (if possible) give lowLag a shot. It worked nicely for me.
Good luck!
Related
I have been playing around with requestAnimationframe for chrome, and wondered how it actually behaves.
When i load my canvas and draw, I get a steady 60FPS. If i scroll around using offset like a click and drag around a map, the FPS will drop (as expected)...once i stop dragging around the map, the FPS creeps back up to its steady 60fps, again as expected.
Here how ever is where I'm wondered if this is delibrate for requestAnimationframe. If i drag the map around until the FPS drop, drops below 30 for an extended period of time, once i stop dragging, it climbs back up, but this time it hits 30FPS and will not go higher. It appears as if the browser decided 30FPS is perhaps the best option.
Is this delibrately done by the browser, i been trying to find out if this is the case. Because it will go to 60fps if i dont drop below 30fps for too long.
Yes, it's something that the browsers are capable of doing.
"How it's supposed to work" isn't really something that anybody can answer, here.
The reason for that is simply that under the hood is 100% browser-specific.
But it's very safe to say that yes, the browser is capable of deciding when you should be locked into a 30Hz refresh, rather than a 60Hz refresh.
An illustration of why this is the case:
requestAnimationFrame() is also tied into the Page Visibility API if the vendors want (very true for Chrome).
Basically, if the page isn't visible, they can slow the requestAnimationFrame() updates down to a few times per second or pause them altogether.
Given that knowledge, it's entirely plausible to believe that one of two things is happening:
they're intentionally capping you at 30fps, because they feel your experience will be more stable there, based on averaged performance data
they're intentionally throttling you, but there's some bug in the system (or some less than lenient math) which is preventing you from going back up to 60, after the coast has cleared, .and if they are using averaged performance data, then that might be part of the issue.
Either way, it is at very least mostly-intentional, with the only unanswered question being why it sticks to 30fps.
Did you leave it alone for 20 or 30 minutes after the fact, to see if it went back up at any time, afterwards?
You can run Timeframe analysis from Chrome DevTools to look for maverick JS that is slowing down your animation times.
https://developers.google.com/chrome-developer-tools/docs/timeline
RAF will find the best place to paint your changes not the closest one. So, if the JS in the RAF callback is taking two frames worth of time(around 16ms per frame on your 60hz hardware), then you FPS will drop to 30.
From Paul Irish via Boris
Actually, “It’s currently capped at 1000/(16 + N) fps, where N is the number of ms it takes your callback to execute. If your callback takes 1000ms to execute, then it’s capped at under 1fps. If your callback takes 1ms to execute, you get about 60fps.” (thx, Boris)
http://www.paulirish.com/2011/requestanimationframe-for-smart-animating/
This may be a stupid/previously answered question, but it is something that has been stumping me and my friends for a little while, and I have been unable to find a good answer.
Right now, i make all my JS Canvas games run in ticks. For example:
function tick(){
//calculate character position
//clear canvas
//draw sprites to canvas
if(gameOn == true)
t = setTimeout(tick(), timeout)
}
This works fine for CPU-cheep games on high-end systems, but when i try to draw a little more every tick, it starts to run in slow motion. So my question is, how can i keep the x,y position and hit-detection calculations going at full speed while allowing a variable framerate?
Side Note: I have tried to use the requestAnimationFrame API, but to be honest it was a little confusing (not all that many good tutorials on it) and, while it might speed up your processing, it doesn't entirely fix the problem.
Thanks guys -- any help is appreciated.
RequestAnimationFrame makes a big difference. It's probably the solution to your problem. There are two more things you could do: set up a second tick system which handles the model side of it, e.g. hit detection. A good example of this is how PhysiJS does it. It goes one step further, and uses a feature of some new browsers called a web worker. It allows you to utilise a second CPU core. John Resig has a good tutorial. But be warned, it's complicated, is not very well supported (and hence buggy, it tends to crash a lot).
Really, request animation frame is very simple, it's just a couple of lines which once you've set up you can forget about it. It shouldn't change any of your existing code. It is a bit of a challenge to understand what the code does but you can pretty much cut-and-replace your setTimeout code for the examples out there. If you ask me, setTimeout is just as complicated! They do virtually the same thing, except setTimeout has a delay time, whereas requestAnimationFrame doesn't - it just calls your function when it's ready, rather than after a set period of time.
You're not actually using the ticks. What's hapenning is that you are repeatedly calling tick() over and over and over again. You need to remove the () and just leave setTimeout(tick,timeout); Personally I like to use arguments.callee to explicitly state that a function calls itself (and thereby removing the dependency of knowing the function name).
With that being said, what I like to do when implementing a variable frame rate is to simplify the underlying engine as much as possible. For instance, to make a ball bounce against a wall, I check if the line from the ball's previous position to the next one hits the wall and, if so, when.
That being said you need to be careful because some browsers halt all JavaScript execution when a contaxt menu (or any other menu) is opened, so you could end up with a gap of several seconds or even minutes between two "frames". Personally I think frame-based timing is the way to go in most cases.
As Kolink mentioned. The setTimeout looks like a bug. Assuming it's only a typo and not actually a bug I'd say that it is unlikely that it's the animation itself (that is to say, DOM updates) that's really slowing down your code.
How much is a little more? I've animated hundreds of elements on screen at once with good results on IE7 in VMWare on a 1.2GHz Atom netbook (slowest browser I have on the slowest machine I have, the VMWare is because I use Linux).
In my experience, hit detection if not done properly causes the most slowdown when the number of elements you're animating increases. That's because a naive implementation is essentially exponential (it will try to do n^n compares). The way around this is to filter out the comparisons to avoid unnecessary comparisons.
One of the most common ways of doing this in game engines (regardless of language) is to segment your world map into a larger set of grids. Then you only do hit detection of items in the same grid (and adjacent grids if you want to be more accurate). This greatly reduces the number of comparisons you need to make especially if you have lots of characters.
I'm creating a game for the iPad using Phonegap, which means I'm using JavaScript/ CSS/ HTML for iPad's Safari. Basically, I'm moving around lots of img elements (sometimes changing their src) on 1024x768 resolution, just local files without any net connection. On the desktop Safari things work smoothly, but on the iPad, my setInterval feels delayed and flickering. Do you have any speed optimization tips for me that I could try? Thanks!
PS: I'm aware that switching to the iOS's native Objective-C would likely be much, much faster, but I'd really love to try it with standard JS/HTML/CSS.
You've run into one of the most common browser scripting issue with animated webpages.
The reason why your application slows down is because the browser is a single-threaded environment. As soon as you forget that you'll get into trouble.
setInterval makes you believe that your actions will happen in parallel like in a multi-threaded environment. But what really happens is that setInterval pushes the action to the UI stack to be handled at a later time. But if too many things are getting in to this stack at one time some actions will lag. The setInterval will keep pushing new actions, but the old ones will be still there, and the whole rendering becomes a sluggish mess.
As to when it happens, it depends on the hardware/software capabilities. iPad has much lower horse power than a desktop PC, that is pretty obvious.
Things you can do in order to avoid lag.
Trade smoothness for speed: raise your delay between intervals, so as to avoid cumulative actions in the UI stack.
setTimeout: this alternative is very similar to setInterval, except that it doesn't ensures a given interval between repetition, rather focuses on how long the browser should wait until repeating the action. So in order to make it more like setInterval you may need to keep track of the elapsed time between actions and calculate the measure of the change that has to be taken care of.
Group animations: you can have one interval for some related animations (you manage a mini-queue for them) and so you decrease the actual setInterval calls, and gain more power over controlling race conditions.
Also make sure to read this piece of article:
Making an iPad HTML5 App & making it really fast (Thomas Fuchs the creator of script.aculo.us)
Use CSS3 animations that use GPU acceleration... This will have a huge effect on any animations.
When writing a JavaScript animation, you of course make a loop using setInterval (or repeated setTimeout). But what is the best delay to use in the setInterval/setTimeout call(s)?
In the jQuery API page for the .animate() function, the user "fbogner" says:
Just if anyone is interested: Animations are "rendered" using a setInterval with a time out of 13ms. This is quite fast! Chrome's fastest possible interval is about 10ms. All other browsers "sample" at about 20-30ms.
Any idea how jQuery determined to use this specific number?
Started bounty. I'm hoping someone with knowledge of the source code behind Chromium or Firefox can provide some hard facts that might back up the decision of a specific framerate. Or perhaps a list of animations (or frameworks) and their delays. I believe this makes for an interesting opportunity to do a bit of research.
Interesting - I just took the time to look at Google's Pac-Man source to see what they did. They set up an array of possible FPSes (90, 45, 30), start at the first one, and then each frame they check the "slowness" of the frame (amount the frame exceeded its allotted time). If the slowness exceeds 50ms 20 times, the framerate is notched down to the next in the list (90 -> 45, 45 -> 30). It appears that the framerate is never raised back up, presumably because the game is so short-lived that it wouldn't be worth the trouble to code that.
Oh, and the setInterval delay is of course set to 1000 / framerate. They do, in fact, use setInterval and not repeated setTimeouts.
I think this dynamic framerate feature is pretty neat!
I would venture to say that a substantial fraction of web users are using monitors that refresh at 60Hz, which translates to one frame every 16.66ms. So to make the monitor the bottleneck, you need to produce frames faster than 16.66ms.
There are two reasons you would pick a value like 13ms. First, the browser needs a little bit of time to repaint the screen (in my experience, never less than 1ms). Which puts you at, say, updating every 15ms, which happens to be a very interesting number - the standard timer resolution on Windows is 15ms (see John Resig's blog post). I suspect that an well-written 15ms animation looks very close to the same on a wide variety of browsers/operating systems.
FWIW, fbogner is plain wrong about non-Chrome browsers firing setInterval every 20-30ms. I wrote a test to measure the speed of setInterval firing, and got these numbers:
Chrome - 4ms
Firefox 3.5 - 15ms
IE6 - 15ms
IE8 - 15ms
The pseudo-code for this is this one:
FPS_WANTED = 25
(just a number, it can be changed while executing, or it can be constant)
TIME_OF_DRAWING = 1000/FPS_WANTED
(this is in milliseconds, I believe it is accurate enough)
( should be updated when FPS_WANTED changes)
UntilTheUserLeavesTheDrawingApplication()
{
time1 = getTime();
doAnimation();
time2 = getTime();
animationTime = time2-time1;
if (animationTime > TIME_OF_DRAWING)
{
[the FPS_WANTED cannot be reached]
You can:
1. Decrease the number of FPS to see if a lower framerate can be achieved
2. Do nothing because you want to get all you can from the CPU
}
else
{
[the FPS can be reached - you can decide to]
1. wait(TIME_OF_DRAWING-animationTime) - to keep a constant framerate of FPS_WANTED
2. increase framerate if you want
3. Do nothing because you want to get all you can from the CPU
}
}
Of course there can be variations of this but this is the basic algorithm that is valid in any case of animation.
When doing loops for animations, it's best that you find a balance between the speed of the loop, and how much work needs to be done.
For example, if you want to slide a div across the page within a second so it is a nice effect and timely. You would skip coordinates and have a reasonably fast loop time so the effect is noticeable, but not jumpy.
So it's a trial and error thing (by having to put work, time, and browser capability into account). So it doesn't only look nice on one browser.
The number told by fbogner have been tested.
The browsers throttle the js-activity to a certain degree to be usable every time.
If your javascript would be possible to run every 5msec the browser runtime would have much less cpu time to refresh the rendering or react on user input (clicks) because javascript-execution blocks the browser.
I think the chrome-devs allow you to run your javascript at much shorter intervals than the other browsers because their V8-Javascript-Engine compiles the JavaScript and therefore it runs faster and the browser will noch be blocked as long as with interpreted js-code.
But the engine is not only so much faster to allow shorter intervals the devs have certainly tested which is the best possible shortest interval to allow short intervals and don't blocking the browser for to long
Don't know the reasoning behind jQuery's interval time, as 13ms translates to 80fps which is very fast. The "standard" fps that's used in movies and such is 25fps and is fast enough that human eye won't notice any jittering. 25fps translates to 40ms, so to answer your question: anything below 40ms is enough for an animation.
I'm starting on a javascript MMORPG that will actually work smoothly. Currently, I created a demo to prove that I can move characters around and have them chat with each other, as well as see eachother move around live.
http://set.rentfox.net/
Now Javascript timers are something I have not used extensively, but from what I know, correct me if I'm wrong, is that having multiple setIntervals happening at the same time doesn't really work well b/c it's all on a single thread.
Lets say I wanted to have 10 different people nuking fireballs at a monster by using sprite background positioning with setInterval -- that animation would require 10 setIntervals doing repainting of the DOM for sprite background-position shifts. Wouldn't that be a big buggy?
I was wondering if there was a way around all this, perhaps using Canvas, so that animations can all happen concurrently without creating an event queue and I don't have to worry about timers.
Hope that makes sense, and please let me know if I need to clarify further.
The issue with multiple setIntervals is twofold. The first is as you indicate, since all Javascript on browsers is (currently) single-threaded, one timer's execution may hold up the next timer's execution. (Worker threads are coming, though; Firefox already has them, as does Safari 4 [and maybe others].) The second is that the timer happens at a set interval, but if your handler is still running when that interval expires, the second interval is completely skipped. E.g., the timer can interfere with itself.
That last part needs more explanation: Say you have a setInterval at 10ms (which is the fastest you can reasonably expect any implementation to do it; may are clamped so that they don't go faster than that). If your handler takes 13ms, the interval that should have happened 10ms after it began will be completely skipped.
I usually use setTimeout for this kind of thing. When my handler is triggered, I do my work and then schedule the next event at the end of the handler. Then (within the bounds of what you can be certain of), I know the next event will happen at that interval.
For what you're doing, it seems like a single "pulse" timer would be best, working through whatever it needs to do on the pulse. Whether that pulse timer uses setInterval or setTimeout is a judgment call based on what you're seeing with your actual code.
+1 to T. J. Crowder, the answer was perfect. I strongly recommend learning to use Canvas over DOM nodes for game animation; the latter is slow and buggy, and will hang the browser in any non-trivial situation. OTOH, Canvas is much faster and can be hardware accelerated, and even has a 3D context if you need it.