I'm wondering if any new HTML5 functions or existing JS library would allow me to access information about the sound that's currently playing in an Audio object. For example, I'd like to be able to access an array of ranges the a song is currently playing at (that is, low values appear for deep bass sounds and higher values appear for shriller sounds). I'm not a sound engineer, so I'm not quite sure what the correct terminology is.
A comparable library would be the C++ BASS library (http://www.un4seen.com/), although I certainly don't need the same breadth of functionality.
I did a little more digging around and found this: chromium.googlecode.com/svn/trunk/samples/audio/visualizer-gl.html
It's pretty much what I'm looking for, but I can't figure out how it works. Thoughts?
The chromium visualizer uses the Web Audio API.
Firefox offers the Audio Data API.
These are the two options available at the moment, and they're not compatible with each other. Eventually an agreement will be reached.
If you intend to do something cross-browser, you're condemned to using Flash for now, there is a pretty good library called SoundManager2 that gives you the necessary data. Check out their visualization demos.
Related
So, I have thought about designing a WebGL Graphics Engine, which will facilitate designing 3D Interactive Graphics for web. Now, my question is :
WebGL is Javascript API, so in order to design an engine for WebGL graphics, do I need to have a JavaScript compiler or anything ? What I want is a system which will let users see what they are creating (for example, like blender workspace, if you draw up a scene you can see and make changes simultaneously)
You would have to create some kind of engine, or framework that you'll build your system onto.
Creating only framework/engine would take at least 2-3 months, and if you plan creating something really big and advanced, that supports various effects rather than simple rendering primitives, than that might come down to 5-6 months. After that you could start creating your web application. So 6-7 months time for that? That shouldn't be a problem.
I don't know how advanced you are, how many people are you working with, but that seems very plausible and doable. But is worth of it? In a year, many different things will change, maybe new openGL ES version for webGL, changing api, supported browsers (IE recently joined the game),... it's really questionable.
You wouldn't need any kind of JS compiler or anything like it, just knowledge of advanced JS and many different techniques used in 3d, and since you plan building system that is far beyond just-graphic-stuff, then it adds even more to overall complexity and time consumption.
So, to answer your question: yes, it's very doable in a year, but will it pay off?
Similar things already exist in some form:
http://errolschwartz.com/projects/threescene/
http://badassjs.com/post/12885773103/threenodes-js-a-visual-webgl-scene-editor
CopperLicht engine has its own real-time editor
there are more lab-playground-like editors
I'd like to create animated heads in my web apps. It seems that CSS3 transition, animation and background features with a little help of javascript web API is all I need. Using xface looks like an overkill to me, cartoon solutions is almost all I need. I need to make it cartoon.
I've made some progress already (beeing able to create voice controlled web app), but this time I need mp3/wav input, not direct voice from microphone using google servers through x-webkit-speech.
I am considering this approach:
record speech into mp3 or wav and write it's string contents
play the mp3 in browser and detect end of words using AnalyserNode to synchronize position in the string (I use Czech language which, unlike the English, has almost constant speech speed).
display the cartoon heads (see the link above) according to actual spoken letter
The question: Is there any lower effort (shorter development time for coder and designer) approach? Especially step 2 and English language in the future makes me worried. Maybe some karaoke tool could produce some speech sync file (which can I parse into CSS3 keyframes)? I am not aware of any such tool.
For something more involved you might try:
Step 1. Web speech API to text to voice...
http://updates.html5rocks.com/2013/01/Voice-Driven-Web-Apps-Introduction-to-the-Web-Speech-API
Step 2 try porting "papagayo" to js (uses dictionary to relate words to phonemes to mouth poses I believe)
http://anime.smithmicro.com/papagayo.html
The GNU source is available here:
http://anime.smithmicro.com/update_files/papagayo/papagayo_1.2_source.zip
You might also refer to:
http://www.adobe.com/devnet/flash/articles/lip-sync-smartmouth.html
for an overview of what you're trying to achieve
Maybe you could do something really quick and dirty with spectrum analysis:
http://0xfe.muthanna.com/wavebox/
I'm building a little sequencer like this one, but want to use HTML/JavaScript exclusively (i.e: no Flash).
As you play around with the applet in the link above, you'll notice the potential for several sounds to play simultaneously and/or in rapid succession. I'm using an array of audio objects simulating different 'slots' or channels to achieve the desired polyphony and, as a concept, this seems to work. I'm not experiencing any clipping or cutouts in my audio.
However, this process becomes prohibitively resource-intensive very quickly, regardless of source filetype/compression. It's not uncommon for the web browser itself to crash as the number of sounds playing increases.
So, I"m wondering: What, if any, is the best way to facilitate a fairly heavy audio load without choking the browser? Or, is trying to make HTML/JS work for this purpose a fool's errand?
Any insight would be much-appreciated. Here is a version without sound.
I cant say myself, but here's some examples making use of the new web audio api.
http://chromium.googlecode.com/svn/trunk/samples/audio/index.html
The drum machine is pretty taxing on my system, though. In any case, I figure you'd be interested in the link.
I'm trying to build an mp3 player for my site using JavaScript (and any plugins/frameworks(jQuery)/libraries that are relevant) & html5. So I built the player (more accurately, I implemented jPlayer), and now I want to make a visualizer.
Ok maybe it's not a visualizer (all the names for ways to visualize sound always confused me), I guess what I want is something like this:
(source: anthonymattox.com)
Or just something that graphs the amplitude (loudness) of an MP3.
So to start, does anyone know an API that can do this?
If you don't that's ok; I guess I'll build my own. For which I need to know:
Does anybody know a way to get the amplitude/loudness of an mp3 at any given point using JavaScript?
EDIT
Changed to a question about php: Visualization of MP3 - PHP
You would need to be able to decode the MP3 yourself. The html5 audio element, and the browsers's implementations of it, don't expose this sort of data. For example, look at Firefox's exposed methods for JavaScript. The closest thing to what you want is the "volumechange" event. But that is in reference to the volume mixer on the browser's rendered control (i.e. output volume). It has nothing to do with the actual dB of the audio source.
I imagine that the only feasible way to do this is to render your waveform to a graphic ahead of time, and then "reveal" it as the song plays (e.g. with the "timeupdate" event).
This question already has answers here:
How to write a web-based music visualizer?
(4 answers)
Closed 5 years ago.
I'm trying to use JavaScript to display the waveform for and audio file, but I don't even know how to get started. I found the Audio Data API, but am unfamiliar with most audio terms and don't really know what is provided or how to manipulate it. I found examples of waveforms in JavaScript, but they are too complicated/I can't comprehend what is going on. Then my question is: how can you use JavaScript to create a waveform of a song on canvas, and what exactly is the process behind it?
Here's some sample code from my book (HTML5 Multimedia: Develop and Design) that does exactly that; Audio Waveform. It uses the Mozilla Audio Data API.
The code simply takes snapshots of the audio data and uses it to draw on the canvas.
Here's an article from the BBC's R&D team showing how they did exactly that to build a couple of JS libraries and more besides. The results all seem to be openly available and rather good.
Rather than use the Audio Data API, which you cannot be sure is supported by all your users' browsers, it might be better if you generate your waveform data server-side (the BBC team created a C++ app to do that) and then at least you are decoupling the client-side display aspect from the playback aspect. Also, bear in mind that the entire audio file has to reach the browser before you can calculate peaks and render a waveform. I am not sure if streaming files (eg MP3) can be used to calculate peaks as the file is coming in. But overall it is surely better to calculate your peaks once, server-side, then simply send the data via JSON (or even create + cache your graphics server-side - there are numerous PHP chart libraries or you can do it natively with GD).
For playback on the browser, there are several good (non-Flash!) options. Personally I like SoundManager 2 as the code is completely decoupled from display, meaning that I am free to create whatever UI / display that I like (or that the client wants). I have found it robust and reliable although I had some initial difficulty on one project with multiple players on the same page. The examples on their site are not great (imho) but with imagination you can do some cool things. SM2 also has an optional Flash fallback option for antique browsers.
I did just that with the web audio api and I used a project called wavesurfer.
http://www.html5audio.org/2012/10/interactive-navigable-audio-visualization-using-webaudio-api-and-canvas.html
What it does is, it draws tiny rectangles and uses an audio buffer to determine the height of each rectangle. Also possible in wavesurfer is playing and pausing using space bar and clicking on the wave to start playing at that point.
Update: This POC website no longer exists.
To check out what I made go to this site:
Update: This POC website no longer exists.
This only works in a google chrome browser and maybe safari but I'm not sure about that.
Let me know if you want more info.
Not well supported yet but take a look at this Firefox tone generator.