I was wondering if there is a way to control audio output device patching in HTML5/JavaScript? Like, if the user wanted to have one sound in my web app to go out of one audio device, and another sound out of a different audio device. I know the user can set the default output device on their computer, but for the web app I'm working on, I would like them to be able to send individual sounds to individual outputs while other sounds are playing, similar to the interface below (from a program called QLab).
I feel like the obvious answer is NO, and I do not want to resort to using flash or java. I MIGHT be okay with having to write some sort of browser plugin that interfaces with javascript.
So, after receiving basically zero helpful answers - and finding no further helpful information online, I think I figured out something that we, as developers, NEED to start requesting from browser vendors and w3c. We need to be able to request hardware access from users, in a similar fashion that we can request to access a user's location, or how we can request to send a user push notifications.
Until web developers are allowed the same control as native application developers over hardware, we will be left at a huge disadvantage over what we can offer our users. I don't want to have my users install third/fourth party plugins to enable a little more control/access to their I/O. Users should not have to be inundated with keeping more software than just their web browser updated to have websites run well and securely. And I, for one, do not feel like it should be necessary to write in more languages than HTML, JavaScript, CSS, and PHP to get the same experience a user would get from a native application.
I have no idea how we approach browser vendors about this, but I feel like it would be good to start doing this.
I know this is a little old, but just this year a method was added called "setSinkId" that you can apply to a media element (video, audio) to set the device that audio will be outputted to.
$('#video-element').setSinkId('default');
https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/setSinkId
Though currently it seems only Chrome supports it. I haven't tested on Firefox or other web browsers.
I suggest you take a look at the Web Audio API:
Specs --- Tutorial
There is the destination property in the Web audio API. However it is a readonly property ... so not settable.
Here:
The destination property always correlates to the default hardware output of sound, whether it’s through speakers, attached headphones, or a Bluetooth headset.
I'm working on a sound installation based off web audio and have run into the same problem. I want to map different channel outputs to different speakers. have you had any progress on this?
This gentleman seems to have managed to do it: http://www.brucewiggins.co.uk/?p=311
I tested this out on a apogee quartet and it worked - outputting to 8 different channels.
I also found this article useful: http://www.html5audio.org/2013/03/surround-audio-comes-to-the-web.html
if (context.destination.maxChannelCount >= 4) {
context.destination.channelCount = 4;
}
// otherwise, let's down-mix to 2.0
else {
context.destination.channelCount = 2;
}
context.destination.channelCountMode = "explicit";
context.destination.channelInterpretation = "discrete";
context.destination.numberOfOutputs = 4;
While you can certainly use the splitter and merger nodes to assign to specific channels on the output, the actual devices you output are abstracted by the browser and inaccessible by your code.
I have done some experiments with 8-channel virtual audio cables and relaying that data to other sound devices outside of the browser. Unfortunately, I can't find a browser that will actually open an 8-channel sound card with more than 2 channels.
Hopefully, browsers in the future will provide more options. This flexibility will never come directly to JavaScript... and nor should it. This is an abstraction done for you, and if the browser uses it correctly, it won't be an issue.
Related
Trying out Windows Universal apps with JavaScript I noticed the WinJS.Utilities.isPhone property is no longer available, which makes sense since there would be no reason to ask for that at runtime.
I do want to know just for testing purposes if there is a proper way of detecting the device my app is running in.
EDIT: My question is NOT about detecting a mobile browser. I'm talking about about a brand new Universal Windows App for Window 10 that can run on phones, desktops, tablets, Xbox, HoloLEns, IoT devices et all. WinJS had a property that would tell me whether I was running on the phone or not. That property is now gone. Please don't close this question due to duplicate with "detecting mobile browser". That is NOT what I need.
Caveat: Any form of device detection is fragile due to the dynamic nature of hardware - a new device could come along tomorrow that breaks your app's logic. It is best to use these APIs only for telemetry / analytics rather than to trigger runtime behaviour.
More often than not, what you really want to know is some attribute of the device or the app that is not inherently tied to the device family (does this device support SystemTray API? Is there a keyboard attached? Is the window more than 500px wide? etc.).
That said, in Windows 10 you can query the DeviceFamily via AnalyticsInfo.VersionInfo.DeviceFamily and it will tell you things like "Mobile" or "Desktop" or "Xbox" etc. (where "Mobile" could be any class of device - phone, tablet, etc.). There is also a property DeviceForm that might be helpful, but again you can't really rely on it at runtime to deterministically say you're running on "a phone."
So basically the answer is "you can use these APIs for telemetry but don't hard-code any values into your app lest it break when a new device arrives on the market." At the very least, always make sure you handle the case where the returned value isn't one you know about a-priori.
You can also check out the following links
http://www.abeautifulsite.net/detecting-mobile-devices-with-javascript/
http://www.sitepoint.com/detect-mobile-devices-jquery/
and of course a similar post here on stackoverflow with a good answer
Detecting a mobile browser
And talking about Windows 10, extracting from Winjs Github repo, here is the answer.
https://github.com/winjs/winjs/issues/601#issuecomment-87137485
There are numerous JS libs to detect which platform/device is used.
I personally love using this lib: https://github.com/kaimallea/isMobile
You will then be able to detect mobile device in such a way:
isMobile.apple.tablet
isMobile.android.phone
and so on.
In case you have an idea to implement such lib yourself, keep in mind that it takes some efforts to keep it up-to-date, because ways of detecting mobile device may change over time.
In general, common way of detecting user device is checking query headers.
Keep in mind, though, that you can't absolutely rely on this information - headers may be easily modified. Google for User Agent for more info.
So "omitting auth process for users with phones" is extremely bad idea
Is there any way to consistently detect PhantomJS/CasperJS? I've been dealing with a spat of malicious spambots built with it and have been able to mostly block them based on certain behaviours, but I'm curious if there's a rock-solid way to know if CasperJS is in use, as dealing with constant adaptations gets slightly annoying.
I don't believe in using Captchas. They are a negative user experience and ReCaptcha has never worked to block spam on my MediaWiki installations. As our site has no user registrations (anonymous discussion board), we'd need to have a Captcha entry for every post. We get several thousand legitimate posts a day and a Captcha would see that number divebomb.
I very much share your take on CAPTCHA. I'll list what I have been able to detect so far, for my own detection script, with similar goals. It's only partial, as they are many more headless browsers.
Fairly safe to use exposed window properties to detect/assume those particular headless browser:
window._phantom (or window.callPhantom) //phantomjs
window.__phantomas //PhantomJS-based web perf metrics + monitoring tool
window.Buffer //nodejs
window.emit //couchjs
window.spawn //rhino
The above is gathered from jslint doc and testing with phantom js.
Browser automation drivers (used by BrowserStack or other web capture services for snapshot):
window.webdriver //selenium
window.domAutomation (or window.domAutomationController) //chromium based automation driver
The properties are not always exposed and I am looking into other more robust ways to detect such bots, which I'll probably release as full blown script when done. But that mainly answers your question.
Here is another fairly sound method to detect JS capable headless browsers more broadly:
if (window.outerWidth === 0 && window.outerHeight === 0){ //headless browser }
This should work well because the properties are 0 by default even if a virtual viewport size is set by headless browsers, and by default it can't report a size of a browser window that doesn't exist. In particular, Phantom JS doesn't support outerWith or outerHeight.
ADDENDUM: There is however a Chrome/Blink bug with outer/innerDimensions. Chromium does not report those dimensions when a page loads in a hidden tab, such as when restored from previous session. Safari doesn't seem to have that issue..
Update: Turns out iOS Safari 8+ has a bug with outerWidth & outerHeight at 0, and a Sailfish webview can too. So while it's a signal, it can't be used alone without being mindful of these bugs. Hence, warning: Please don't use this raw snippet unless you really know what you are doing.
PS: If you know of other headless browser properties not listed here, please share in comments.
There is no rock-solid way: PhantomJS, and Selenium, are just software being used to control browser software, instead of a user controlling it.
With PhantomJS 1.x, in particular, I believe there is some JavaScript you can use to crash the browser that exploits a bug in the version of WebKit being used (it is equivalent to Chrome 13, so very few genuine users should be affected). (I remember this being mentioned on the Phantom mailing list a few months back, but I don't know if the exact JS to use was described.) More generally you could use a combination of user-agent matching up with feature detection. E.g. if a browser claims to be "Chrome 23" but does not have a feature that Chrome 23 has (and that Chrome 13 did not have), then get suspicious.
As a user, I hate CAPTCHAs too. But they are quite effective in that they increase the cost for the spammer: he has to write more software or hire humans to read them. (That is why I think easy CAPTCHAs are good enough: the ones that annoy users are those where you have no idea what it says and have to keep pressing reload to get something you recognize.)
One approach (which I believe Google uses) is to show the CAPTCHA conditionally. E.g. users who are logged-in never get shown it. Users who have already done one post this session are not shown it again. Users from IP addresses in a whitelist (which could be built from previous legitimate posts) are not shown them. Or conversely just show them to users from a blacklist of IP ranges.
I know none of those approaches are perfect, sorry.
You could detect phantom on the client-side by checking window.callPhantom property. The minimal script is on the client side is:
var isPhantom = !!window.callPhantom;
Here is a gist with proof of concept that this works.
A spammer could try to delete this property with page.evaluate and then it depends on who is faster. After you tried the detection you do a reload with the post form and a CAPTCHA or not depending on your detection result.
The problem is that you incur a redirect that might annoy your users. This will be necessary with every detection technique on the client. Which can be subverted and changed with onResourceRequested.
Generally, I don't think that this is possible, because you can only detect on the client and send the result to the server. Adding the CAPTCHA combined with the detection step with only one page load does not really add anything as it could be removed just as easily with phantomjs/casperjs. Defense based on user agent also doesn't make sense since it can be easily changed in phantomjs/casperjs.
I'm developing a map based web application and I would like to offer users the ability to "Use current location"
I can easily check for the availability of the geolocation api as follows:
if (navigator && navigator.geolocation) {}
And once I've actually called navigator.geoLocation.getCurrentPosition I can easily check the accuracy of the result, or if the result is available at all.
However, I would like to only display the "Use current location" button when there is a good chance that it might work well, i.e. produce a result accurate to within a couple of hundred meters. I'd like to make this decision without asking the user to share their location.
My best guess so far is to basically attempt to always hide this button on a desktop or laptop PC, probably by using Modernizr touchscreen detection. This clearly won't be 100% effective due to touchscreen laptops, but maybe it's better than nothing.
I'd also like to avoid doing anything too elaborate on the server side. This is the only functionality in my app so far that requires device detection of any sort, so WURFL or similar user agent look ups are out of the question. I'd like to do it all client side without too much bloat
Thanks for any help
Unfortunately, no. Getting the geolocation accuracy of a user is private information so you need to make the request to the end-user first (as in navigator.geolocation.getCurrentPosition). The accuracy of the gps is device specific information, so the ONLY way you could get it would be on the client and not on your server.
Sorry there isn't a better solution for you. If you're making a mobile app, you could always use phonegap/cordova and use or create a plugin that could get you this information.
I'm working on HTML5 app that lets several users to work on one document. I need to add a possibility for users (editing the same document) to talk to each other. And I just don't know how to start with that. Here are my questions
Is there an HTML5 lib allowing to transfer sound from microphone between clients?
What about streaming video from camera?
What is an easiest server-side solution for that?
Any thoughts are strongly appreciated! So don't be shy! :)
UPD: please note that I need an abbility for more then two users to talk.
For this you can use WebRTC.
However, this is a very young and unfinished technology that as already stated is currently available only in Chrome stable and Firefox beta. This means there will probably come changes to the current spec, something to be aware of in case of early implementation. But it allow you to use video and audio communication directly in the browser.
Quick-start here:
http://www.html5rocks.com/en/tutorials/webrtc/basics/
Other options are Flash based plugins such as flash-videoio. This is an open source plugin but will naturally require Adobe Flash installed. This may or may not be a problem depending on the company's security policy.
For technical details on implementation please see examples on the provided links.
For many-to-many you can use either:
"Mesh" - everybody connects to everybody. This however is costly on CPU and mobiles are often left out.
"Star" - everybody goes through the most capable device. However, with many connections this will soon run slow for the device handling all connections.
MCU. Specialized server to handle all connections. If mixes audio and video and handles drop-outs as well without affecting the other callers.
Examples of MCU's:
http://sourceforge.net/projects/mcumediaserver/ (open source)
http://www.medooze.com/products/mcu.aspx (commercial)
you are searching for navigator.getUserMedia()
that allows the various users to share video audio and data.
the support is very low... only chrome and the latest verions of opera and firefox support it.
and totally no support on mobile devices... maybe in the next android chrome... dunno
as there is much to talk about and i have no clue on how u wanna setup everything i suggest u read a little more about that on the urls...
http://caniuse.com/stream
http://www.html5rocks.com/en/tutorials/getusermedia/intro/
http://dev.w3.org/2011/webrtc/editor/getusermedia.html
https://developer.mozilla.org/en-US/docs/WebRTC/navigator.getUserMedia
http://my.opera.com/core/blog/2011/03/23/webcam-orientation-preview
http://simpl.info/getusermedia/
and SERVERSIDE solution nahh... thats not a good solution
clientside is the way to go.
Not sure if you're required to do it yourself from scratch or are able to use third party libraries/tools.
In which case I would recommend using Tokbox which has support for WebRTC and SDK for iOS.
Their API is simple and easy to use.
I want to create a simple visualization tool that would allow to represent my playing a midi keyboard on the screen. I play a relatively novel instrument type, called the harmonic table:
http://en.wikipedia.org/wiki/Harmonic_table_note_layout
http://www.soundonsound.com/newspix/image/axis49.jpg
And want to build tools to ease their use and to teach others how to use them.
However, I can't find a good way to get get midi into javascript environment (or, for that matter, Flash, or Java without a large helping of jiggery-pokery slightly beyond my reach, and the use of code from what look to be rather stale and abandoned open source projects. Neither language I am too enthused to work in for this project in any case).
Is there a suitable library or application that I have missed, that will enable me to do this?
While searching around for another solution (Flash based, using the functions of the Red5 Open source flash server - really ugly, but I'm desperate at this point) I found someone who had done exactly what I needed using Java to interface with the hardware. They had started with a flash solution and recently ported to Javascript. Yay!
http://www.abumarkub.net/abublog/?p=505
Don't let the caveats about 'proof of concept' discourage you: the basic functionality appears solid, at least with everything I have been able to throw at it.
So now I'm on my way, and so is anyone else who want to build javascript based midi interfaces/synths/what have you.
I can manipulate real-time midi in javascript! This is much better than flying cars and jetboots.
I have made a NPAPI browser plugin that lets you communicate in Javascript with MIDI devices.
Currently it works on OSX in Chrome and Safari, and on Windows in Chrome.
I am working on support for other browsers (Firefox, Internet Explorer) and other operating systems (Linux, Android, iOs)
See: http://abumarkub.net/abublog/?p=754
EDIT:
I recently published this module https://github.com/hems/midi2funk it's a node.js module that listens to midi and broadcast it through socket.io so if you have the luxury of running a node.js service locally together with your client side you might get some fun out of it...
~~~~~
A few others handy links, i kinda ordered in what i think would be most important for you:
midibridge.js - A Javascript API for interacting with MIDI devices
midi.js sequencing in javascript
jasmid - A Javascript MIDI file reader and synthesiser
Second web midi api working draft published - 11/12/2012
Jazz Soft - MIDI IN / OUT PLUGIN FOR BROWSER
edit: just realised the thread is old, hopefully the links will help someone ( :
The Web MIDI API is now real in Google Chrome 43+. I even wrote a library to make it easier to work with it. If you are still interested and do not care that it currently only works in Chrome, check it out: https://github.com/cotejp/webmidi
Nowadays browsers supports MIDI listening. All you need is
navigator.requestMIDIAccess().then(requestMIDIAccessSuccess, requestMIDIAccessFailure);
and listen keys
function requestMIDIAccessSuccess(midi) {
var inputs = midi.inputs.values();
for (var input = inputs.next(); input && !input.done; input = inputs.next()) {
console.log('midi input', input);
input.value.onmidimessage = midiOnMIDImessage;
}
midi.onstatechange = midiOnStateChange;
}
See working example here
Most browsers don't allow access to any hardware except the keyboard and mouse - for obvious security reasons, so it's unlikely that you could access a midi device unless it's plugged in as one of those devices.
You could try finding a driver that would translate midi output to key presses, and then deal with those in the browser, but this would be a single-computer solution only.
I am really excited by the upcoming Web MIDI API. As far as I know, its only under discussion and hasn't made it into any browsers yet.
Combined with the Web Audio API which has started to be implemented in some browsers already, it will be possible to have complete software synthesis in the browser. Exciting times.
Since Web MIDI API is still a draft, there is no way of direct access to MIDI events in the browser.
A simple workaround could be to write a small server where you register MIDI events and communicate them to your javascript using a websocket. This could be done quite easily in Python.