I'm teaching kids how to add audio to their JavaScript games and webpages. I knew about the HTML5 Audio API and am just now learning about the issues around the various browsers (IE surprisingly doesn't support WAV at all, etc).
However, apparently there's a new Web Audio standard I was unaware of. Chrome fully supports it. Mozilla states the following about Web Audio and Firefox:
Firefox currently supports the incompatible, Mozilla-specific Audio
Data API, but it is marked as "deprecated" in favor of the W3C
standard. Work has started to support the new Web Audio API in
Firefox.
caniuse.com reports the following about support for Web Audio API.
I don't mind at all forcing the kids to use Google Chrome for development and sound not working in many other browsers. However, I don't want to be teaching them something that will not be in other browsers in the future.
I'm also reading about a possible alternative to Web Audio called MediaStreaming... Which is the future that I should be teaching?
Strictly speaking, there's really no HTML5 Audio API. But HTML5 did introduce the <audio/> element, which is great for simple use-cases.
If you need high-precision timing, or the ability to perform DSP -- I'd go with the Web Audio API. It's still a draft as far as the W3C is concerned, but it seems fairly clear that this is the direction they're going in: http://www.w3.org/2011/audio/wiki/PubStatus
There is no promise that the web audio API will be in all browsers but it is available in Chrome,Safari,and supposedly scheduled for Firefox. Opera has just changed their engine to webkit which is the engine used for the current batch of Web Audio API supporting browsers.
Internet Explorer is completely up in the air.
If all you need to do is play audio files, I would strongly recommend using howler.js. It doesn't have a ton of features, but it is very reliable, and works well across browsers.
The Web Audio API is definitely 'the future' of audio on the web, but it can be pretty complicated, especially if you're trying to teach it to children.
I read somewhere that microsoft is also getting engaged in the web audio api and others
Also some features like capturing audio are only supported by canary or has this changed lately? When will the entire API be supported by the standard chrome webbrowser? Another problem I had with the audio API is that when you have to do some calculations for DSP as Kevin Ennis mentioned, then javascript has problems to do so and your browser crashes. I tried to figure out why it crashes but all I could find is that javascript is not the rights choice for heavy calculations. So as far as web audio API supporting DSP I'm not convinced.
I think it will still take a lot of time since you can create a website that uses web audio API and that will be supported in a lot of browsers and that will not crash randomly. But there is not real other choice for the moment so in the future I am guessing that more and more browsers will start supporting it.
If you are expecting the game should run only in the modern browsers you can checkout the audio library musquito. It's relatively new and it's completely based on Web Audio API.
musquito also supports audio sprites that helps very much in games.
If you are looking for a fallback mechanism for older browsers then howler.js is a good option.
[Disclaimer: I'm the creator of musquito]
Related
I have a web program which makes use of Web Audio API. The issue here is that i want to make it compatible for IE. Is there any alternate for the Web Audio API, so that i can make the same code run on IE specifically?
What are your needs? If you need to do dynamic synthesis, audio routing, etc, you will only be able to achieve that with the Web Audio API, so your IE users are out of luck.
However, if all you need to do is play audio files, then I would recommend that you use howler.js. Howler has great compatibility across different browsers and operating systems, including various versions of IE.
Microsoft have been working on implementing the web audio API, it looks like a preview implementation is available. https://status.modern.ie/webaudioapi If you can wait until the next version of IE which I believe is expected to be released with Windows 10, which should be out this year IIRC, it may not be worth taking the time to implement an alternative.
That said, to answer your question, there's no way that I could find to generate audio with an oscillator in the web without the Web Audio API other than writing flash code, which has its own major disadvantages.
Minor note: you could possibly hack the html5 audio node to play back audio buffers that you generate in javascript code, but keeping everything in sync and preventing jumps in the waveform would be an awful task.
I'm working on HTML5 app that lets several users to work on one document. I need to add a possibility for users (editing the same document) to talk to each other. And I just don't know how to start with that. Here are my questions
Is there an HTML5 lib allowing to transfer sound from microphone between clients?
What about streaming video from camera?
What is an easiest server-side solution for that?
Any thoughts are strongly appreciated! So don't be shy! :)
UPD: please note that I need an abbility for more then two users to talk.
For this you can use WebRTC.
However, this is a very young and unfinished technology that as already stated is currently available only in Chrome stable and Firefox beta. This means there will probably come changes to the current spec, something to be aware of in case of early implementation. But it allow you to use video and audio communication directly in the browser.
Quick-start here:
http://www.html5rocks.com/en/tutorials/webrtc/basics/
Other options are Flash based plugins such as flash-videoio. This is an open source plugin but will naturally require Adobe Flash installed. This may or may not be a problem depending on the company's security policy.
For technical details on implementation please see examples on the provided links.
For many-to-many you can use either:
"Mesh" - everybody connects to everybody. This however is costly on CPU and mobiles are often left out.
"Star" - everybody goes through the most capable device. However, with many connections this will soon run slow for the device handling all connections.
MCU. Specialized server to handle all connections. If mixes audio and video and handles drop-outs as well without affecting the other callers.
Examples of MCU's:
http://sourceforge.net/projects/mcumediaserver/ (open source)
http://www.medooze.com/products/mcu.aspx (commercial)
you are searching for navigator.getUserMedia()
that allows the various users to share video audio and data.
the support is very low... only chrome and the latest verions of opera and firefox support it.
and totally no support on mobile devices... maybe in the next android chrome... dunno
as there is much to talk about and i have no clue on how u wanna setup everything i suggest u read a little more about that on the urls...
http://caniuse.com/stream
http://www.html5rocks.com/en/tutorials/getusermedia/intro/
http://dev.w3.org/2011/webrtc/editor/getusermedia.html
https://developer.mozilla.org/en-US/docs/WebRTC/navigator.getUserMedia
http://my.opera.com/core/blog/2011/03/23/webcam-orientation-preview
http://simpl.info/getusermedia/
and SERVERSIDE solution nahh... thats not a good solution
clientside is the way to go.
Not sure if you're required to do it yourself from scratch or are able to use third party libraries/tools.
In which case I would recommend using Tokbox which has support for WebRTC and SDK for iOS.
Their API is simple and easy to use.
I understand that the best bet when streaming a video as a source in an HTML5 tag would be .mp4. But let's say that I have a source that only outputs fragmented to an ism/manifest.
Is there any way at all, whether through other libraries or messy hacks, that I can bring this video into something rendered as a tag onscreen? The closest I have found is Walkthrough: Building Your First HTML5 Smooth Streaming Player because it allows this to be done - but I neither have Windows 8, or want to have this running a server capable of .NET. I was hoping there was something, messy or not, that I could achieve this with entirely within javascript and executable locally without a deploy.
Thanks
Firstly, W3C does not provide a standard for adaptive bitrate streaming, yet. So for the time being most browsers only support simple progressive download playback.
Hence, there is no JS implementation of a Smooth Streaming player and Microsoft is not working on one, as far as I know.
The example you provide uses the "Microsoft Smooth Streaming Client SDK Beta 2 for Windows 8" which is a C++ library and is only available for Windows Store Apps development. It has nothing to do with browsers.
So, unfortunately this is not yet possible. Even more, I doubt that this will ever happen, because everybody is waiting for MPEG DASH to be finalized.
UPDATE.
Please, notice that you always can use Siverlight application for playing SmoothStreaming. The referenced HTML5 Player framework is capable of falling back to Silverlight.
no luck for Microsoft Smooth Streaming, but regarding MPEG-DASH which is similar (see http://blog.johndeutscher.com/2013/06/10/mpeg-dash-preview-from-windows-azure-media-services):
"Dash.js is permissively licensed (under the BSD license) and can therefore be studied and reused by anyone seeking to provide their own DASH-AVC/264 compliant player. The goal is to make it easier for third-parties to build adaptive streaming video players."
http://msopentech.com/blog/2013/06/20/ms-open-tech-contributes-to-open-source-adaptive-streaming-video-player/
also see:
https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html
which is required for Dash.js
I want to create a simple visualization tool that would allow to represent my playing a midi keyboard on the screen. I play a relatively novel instrument type, called the harmonic table:
http://en.wikipedia.org/wiki/Harmonic_table_note_layout
http://www.soundonsound.com/newspix/image/axis49.jpg
And want to build tools to ease their use and to teach others how to use them.
However, I can't find a good way to get get midi into javascript environment (or, for that matter, Flash, or Java without a large helping of jiggery-pokery slightly beyond my reach, and the use of code from what look to be rather stale and abandoned open source projects. Neither language I am too enthused to work in for this project in any case).
Is there a suitable library or application that I have missed, that will enable me to do this?
While searching around for another solution (Flash based, using the functions of the Red5 Open source flash server - really ugly, but I'm desperate at this point) I found someone who had done exactly what I needed using Java to interface with the hardware. They had started with a flash solution and recently ported to Javascript. Yay!
http://www.abumarkub.net/abublog/?p=505
Don't let the caveats about 'proof of concept' discourage you: the basic functionality appears solid, at least with everything I have been able to throw at it.
So now I'm on my way, and so is anyone else who want to build javascript based midi interfaces/synths/what have you.
I can manipulate real-time midi in javascript! This is much better than flying cars and jetboots.
I have made a NPAPI browser plugin that lets you communicate in Javascript with MIDI devices.
Currently it works on OSX in Chrome and Safari, and on Windows in Chrome.
I am working on support for other browsers (Firefox, Internet Explorer) and other operating systems (Linux, Android, iOs)
See: http://abumarkub.net/abublog/?p=754
EDIT:
I recently published this module https://github.com/hems/midi2funk it's a node.js module that listens to midi and broadcast it through socket.io so if you have the luxury of running a node.js service locally together with your client side you might get some fun out of it...
~~~~~
A few others handy links, i kinda ordered in what i think would be most important for you:
midibridge.js - A Javascript API for interacting with MIDI devices
midi.js sequencing in javascript
jasmid - A Javascript MIDI file reader and synthesiser
Second web midi api working draft published - 11/12/2012
Jazz Soft - MIDI IN / OUT PLUGIN FOR BROWSER
edit: just realised the thread is old, hopefully the links will help someone ( :
The Web MIDI API is now real in Google Chrome 43+. I even wrote a library to make it easier to work with it. If you are still interested and do not care that it currently only works in Chrome, check it out: https://github.com/cotejp/webmidi
Nowadays browsers supports MIDI listening. All you need is
navigator.requestMIDIAccess().then(requestMIDIAccessSuccess, requestMIDIAccessFailure);
and listen keys
function requestMIDIAccessSuccess(midi) {
var inputs = midi.inputs.values();
for (var input = inputs.next(); input && !input.done; input = inputs.next()) {
console.log('midi input', input);
input.value.onmidimessage = midiOnMIDImessage;
}
midi.onstatechange = midiOnStateChange;
}
See working example here
Most browsers don't allow access to any hardware except the keyboard and mouse - for obvious security reasons, so it's unlikely that you could access a midi device unless it's plugged in as one of those devices.
You could try finding a driver that would translate midi output to key presses, and then deal with those in the browser, but this would be a single-computer solution only.
I am really excited by the upcoming Web MIDI API. As far as I know, its only under discussion and hasn't made it into any browsers yet.
Combined with the Web Audio API which has started to be implemented in some browsers already, it will be possible to have complete software synthesis in the browser. Exciting times.
Since Web MIDI API is still a draft, there is no way of direct access to MIDI events in the browser.
A simple workaround could be to write a small server where you register MIDI events and communicate them to your javascript using a websocket. This could be done quite easily in Python.
I am a little new to embedding videos on a website. What is the best way to go about it? I just started reading up on it.
I read that I should use HTML5 and the browser's native player. -- Does anyone know of a javascript library that will use the browser's player and detect if the browser is not using HTML5 to downgrade to a flash player?
As far as encoding what is the best approach? I am planning on writing a little console app to use FFMPEG to convert the videos. But what encoding should I use? I heard H.264 is promising? But
This is what you're looking for: http://videojs.com/
Encoding wise, I think H.264 and WebM is a wise choice. That way you'll satisfy all modern browsers and mobile devices (incl. iOS and Android), and only fall back to Flash for older, pre-HTML5 browsers (and FF 3.6, which only supports Theora.)
The DIY approach would be to use the Video For Everybody embed code. That will lead with HTML5 video and fallback to flash.
Then for video encoding, you can use FFmpeg like you said.
To make your life easier, you should check out VideoJS for your HTML5 video player. It handles a lot of cross browser issues and some extra features that browsers don't support yet.
For video encoding, if you're planning to use FFmpeg specifically for automating encoding, you should check out Zencoder's video encoding API. It could save you some set up time and compatibility issues.
For codecs, #1 priority should be h.264, specifically using the baseline profile so it will work on smartphones. After that, ogg and webm are both good choices. WebM is higher quality for file size, but Ogg can be viewed by way more users because older version of Firefox support it. And just to note, if you only do h.264, the Video For Everybody code won't fallback to Flash in Firefox and Opera (that don't support h.264) so you'll need to use a player like VideoJS.
Use Handbrake for video encoding, especially for HTML 5 support.
HTML 5 is an awesome way of embedding video, however it hasn't been standardized yet. You definitely want to use it if you're supporting iOS devices, as they of course don't support Flash which is the most mainstream way of embedding video.
HTML5 is pretty awesome because this is how simple it is:
<video src="video.m4v" />
Here is a fantastic link you will find useful:
http://www.robwalshonline.com/posts/tutorial-serving-html5-video-cross-browser-including-ipad/
I read that I should use HTML5 and the
browser's native player
Which idiot told you that? Given that you will loose out on a significant portion of the client population you either run a monoculture of HTML 5 browsers, of whoever said that should be serving burgers at McDonalds, not give advice to people. HTML 5 and the integrated player are nice, but not there yet for the general public. In a couple of years, yes. Now - if I hire a web company for putting videos on a website and they do that, they get sued for gross neglect and incompetence.
Today, playback is mostly flash - that has a high penetration. A flash video player is also what sites like youtube currently use. If you go for less (penetration), Silverlight is very good and has the complete backend infrastructure.
I heard H.264 is promising?
So what? FIRST you need to fix the playback side, then you use whatever format is most efficient given your playback decision.