I know a webapp can access media devices like microphones and webcams, and other hardware like a smartphone's GPS. As far as I know, that's done with tightly constrained protocols for each specific type of device.
However, I have an advanced scientific camera. It is only useful with a computer - it ships with a desktop application for controlling it and taking photos. It also ships with a C/C++ SDK to interface with it in your own applications.
The browser obviously doesn't recognize it as a webcam. Even if it did, all but the most basic functionality would be ignored. The camera is on the client side. Is it possible to write a webapp that can interface with that camera and use all of its features? I'm not looking for a full solution, I just don't even know what to google.
Any amount of hand-rolled solution is fair game here. Anything from plain JavaScript to browser plug-ins to a custom desktop middleware psuedo-driver to sit between the hardware and the camera and the browser. Even binding the client to a specific OS is fine.
You can do basic video capture and screen grabs with Silverlight:
https://msdn.microsoft.com/en-us/library/ff602282(v=vs.95).aspx
It also scriptable by Javascript:
https://msdn.microsoft.com/en-us/library/cc645085(v=vs.95).aspx
Problem is, Silverlight is going away. Officially not until October 2021 though so that might still be an option until the browser vendors come online with HTML 5 Media Capture and Streams:
http://w3c.github.io/mediacapture-main/getusermedia.html
Anything beyond basic capture though, your probably looking at a custom browser extension to control the camera's functions thought it's provided API
Related
Trying out Windows Universal apps with JavaScript I noticed the WinJS.Utilities.isPhone property is no longer available, which makes sense since there would be no reason to ask for that at runtime.
I do want to know just for testing purposes if there is a proper way of detecting the device my app is running in.
EDIT: My question is NOT about detecting a mobile browser. I'm talking about about a brand new Universal Windows App for Window 10 that can run on phones, desktops, tablets, Xbox, HoloLEns, IoT devices et all. WinJS had a property that would tell me whether I was running on the phone or not. That property is now gone. Please don't close this question due to duplicate with "detecting mobile browser". That is NOT what I need.
Caveat: Any form of device detection is fragile due to the dynamic nature of hardware - a new device could come along tomorrow that breaks your app's logic. It is best to use these APIs only for telemetry / analytics rather than to trigger runtime behaviour.
More often than not, what you really want to know is some attribute of the device or the app that is not inherently tied to the device family (does this device support SystemTray API? Is there a keyboard attached? Is the window more than 500px wide? etc.).
That said, in Windows 10 you can query the DeviceFamily via AnalyticsInfo.VersionInfo.DeviceFamily and it will tell you things like "Mobile" or "Desktop" or "Xbox" etc. (where "Mobile" could be any class of device - phone, tablet, etc.). There is also a property DeviceForm that might be helpful, but again you can't really rely on it at runtime to deterministically say you're running on "a phone."
So basically the answer is "you can use these APIs for telemetry but don't hard-code any values into your app lest it break when a new device arrives on the market." At the very least, always make sure you handle the case where the returned value isn't one you know about a-priori.
You can also check out the following links
http://www.abeautifulsite.net/detecting-mobile-devices-with-javascript/
http://www.sitepoint.com/detect-mobile-devices-jquery/
and of course a similar post here on stackoverflow with a good answer
Detecting a mobile browser
And talking about Windows 10, extracting from Winjs Github repo, here is the answer.
https://github.com/winjs/winjs/issues/601#issuecomment-87137485
There are numerous JS libs to detect which platform/device is used.
I personally love using this lib: https://github.com/kaimallea/isMobile
You will then be able to detect mobile device in such a way:
isMobile.apple.tablet
isMobile.android.phone
and so on.
In case you have an idea to implement such lib yourself, keep in mind that it takes some efforts to keep it up-to-date, because ways of detecting mobile device may change over time.
In general, common way of detecting user device is checking query headers.
Keep in mind, though, that you can't absolutely rely on this information - headers may be easily modified. Google for User Agent for more info.
So "omitting auth process for users with phones" is extremely bad idea
I'm working on HTML5 app that lets several users to work on one document. I need to add a possibility for users (editing the same document) to talk to each other. And I just don't know how to start with that. Here are my questions
Is there an HTML5 lib allowing to transfer sound from microphone between clients?
What about streaming video from camera?
What is an easiest server-side solution for that?
Any thoughts are strongly appreciated! So don't be shy! :)
UPD: please note that I need an abbility for more then two users to talk.
For this you can use WebRTC.
However, this is a very young and unfinished technology that as already stated is currently available only in Chrome stable and Firefox beta. This means there will probably come changes to the current spec, something to be aware of in case of early implementation. But it allow you to use video and audio communication directly in the browser.
Quick-start here:
http://www.html5rocks.com/en/tutorials/webrtc/basics/
Other options are Flash based plugins such as flash-videoio. This is an open source plugin but will naturally require Adobe Flash installed. This may or may not be a problem depending on the company's security policy.
For technical details on implementation please see examples on the provided links.
For many-to-many you can use either:
"Mesh" - everybody connects to everybody. This however is costly on CPU and mobiles are often left out.
"Star" - everybody goes through the most capable device. However, with many connections this will soon run slow for the device handling all connections.
MCU. Specialized server to handle all connections. If mixes audio and video and handles drop-outs as well without affecting the other callers.
Examples of MCU's:
http://sourceforge.net/projects/mcumediaserver/ (open source)
http://www.medooze.com/products/mcu.aspx (commercial)
you are searching for navigator.getUserMedia()
that allows the various users to share video audio and data.
the support is very low... only chrome and the latest verions of opera and firefox support it.
and totally no support on mobile devices... maybe in the next android chrome... dunno
as there is much to talk about and i have no clue on how u wanna setup everything i suggest u read a little more about that on the urls...
http://caniuse.com/stream
http://www.html5rocks.com/en/tutorials/getusermedia/intro/
http://dev.w3.org/2011/webrtc/editor/getusermedia.html
https://developer.mozilla.org/en-US/docs/WebRTC/navigator.getUserMedia
http://my.opera.com/core/blog/2011/03/23/webcam-orientation-preview
http://simpl.info/getusermedia/
and SERVERSIDE solution nahh... thats not a good solution
clientside is the way to go.
Not sure if you're required to do it yourself from scratch or are able to use third party libraries/tools.
In which case I would recommend using Tokbox which has support for WebRTC and SDK for iOS.
Their API is simple and easy to use.
We have a large web product with 11 million users (widgets that are embedded into third party web sites, and now adding iOS/Android apps). Most of our core interface is written in ActionScript 2.0. We are planning to release a revamped version. We are now facing a choice of porting it to ActionScript 3.0 or rewriting it in HTML5. We already have about 30% of our code written in AS 3.0. Can you help us make this decision?
HTML5
Pros
Future proof
Works in browser in iOS
No plugin required, presumably faster loading
Native browser rich text rendering, including spell check and copy/paste
Appealing to webmasters (contemporary technology)
Cons
Product relies on WebSocket - no standard yet, not available in IE
Highly custom UI, not consistent across all browsers
No video/audio hardware interaction for stream capture
Harder to convert into iOS apps (e.g. need hybrid apps, UIWebView cannot easily call functions in the native wrapper)
JavaScript is like AS 1.0, less sophisticated than AS 3.0
More work for us given our existing codebase and skills
Harder to embed JS widgets into third party sites than Flash widgets
Flash
Pros
Already have some code base, experience
Video/audio stream capture, socket connection
Good for widgets on third party websites
Consistent UI across browsers and platforms
Easier to use the same codebase for AIR apps on iOS/Android
Cons
Adobe's commitment to Flash is unclear
Adobe feels free to force developers to change technologies often, unreliable : AS1.0, AS2.0 have been phased out, Text Layout Framework is not supported on mobile and is being phased out, not clear if our AS3.0 codebase will still be relevant in 3 years)
Shoddy support of Flash on Android (e.g. text input is broken in 25% of devices)
Does not work in browser on iOS and Metro, so need a lite HTML5 version anyway
Slower in browser on Android
Public perception of Flash as obsolete?
We are in an identical situation and I truly wish that Adobe would take the lead in answering this very question; Flash or move on?
We have an AS2 Flash RIA (RIA, a phrase from the past?) that allows our clients to edit their sites from their browser, similar to Wix. Although the sites are viewable on iOS devices via ColdFusion (example: dolcedesignsonline.com - try in IE then in an iOS borwser), we are wondering what to do next. We are in the middle of an AS3 version of our RIA and although nothing in HTML5 will ever come close to the functinality we now enjoy at our fingertips in AS3, is AS3 going to there down the road?!
I do not think that Adobe would just let flash die. A more logical next step would be make the Flash IDE open source and free to the public. I am not endorsing that idea and I do not see that happening but I believe that would happen long before Adobe would just close the doors on Flash altogether.
Tick tock... at some point we are just going to have to flip a coin.
John Glynn
My thoughts on your points:
HTML5
Pros
Future proof - this is also true of Flash (you can still run an AS1 movie in FP11)
Works in browser in iOS - but webGL requires iAds, and canvas knackers the battery, in that sense support for actual HTML5 features is pretty limited
No plugin required, presumably faster loading - depends what's loading!
Native browser rich text rendering, including spell check and copy/paste - good points, not sure what/if any support these have in AIR
Appealing to webmasters (contemporary technology) - depends on the webmaster, beside Canvas/webGL it's just plain old HTML/JS
Cons
Product relies on WebSocket - not sure how you could get around this
Highly custom UI, not consistent across all browsers - this will always be the bane of web developers - look into jQuery UI if you haven't already
No video/audio hardware interaction for stream capture - correct AFAIK
Harder to convert into iOS apps (e.g. need hybrid apps, UIWebView cannot easily call functions in the native wrapper) - correct, but it is possible to do
Javascript is like AS 1.0, less sophisticated than AS 3.0
More work for us given our existing codebase and skills
Harder to embed JS widgets into third party sites than Flash widgets - JS is a very cool language but it's not really suited to building large applications which will need to be maintained by many people, debugging it can be an absolute bitch.
Flash
Pros
Already have some code base, experience - code can be organised into classes, easier to maintain and debug, AS3 also has most of the standout JS features (closures, objects, 1st-class functions) as it is derived from it.
Video/audio stream capture, socket connection - true
Good for widgets on third party websites - not sure what relevance this has to you, as not entirely sure what kind of application you're making
Consistent UI across browsers and platforms - to me, this wins by itself!
Easier to use the same codebase for AIR apps on iOS/Android - true, related to above
Cons
Adobe's commitment to Flash is unclear - this is true, but Flash player will be around for years, if not decades, yet, Stage3D has breathed a new lease of life into the platform
Adobe feels free to force developers to change technologies often, unreliable : AS1.0, AS2.0 have been phased out, Text Layout Framework is not supported on mobile and is being phased out, not clear if our AS3.0 codebase will still be relevant in 3 years) - This isn't really true as the Flash Player provides legacy support for everything ever built in Flash, it might make no sense to code in AS1 now but if you do, you can still export and run on any machine with Flash Player.
Shoddy support of Flash on Android (e.g. text input is broken in 25% of devices) - this is the major drawback, I'm not sure how well AIR addresses these problems but in my experiments with it, the only issue I've found with AIR on android is performance for intensive graphics.
Does not work in browser on iOS and Metro, so need a lite HTML5 version anyway - This may be insurmountable unless you can persuade your clients to use an app (shouldn't be too hard whilst the buzz remains around the word), then you can use AIR - and your existing codebase!
Slower in browser on Android - again, you may need to go with the AIR option, even as a Web/Flash developer I've never been over-keen on the idea of it running in mobile browser, at least with current technology
Public perception of Flash as obsolete? - This has been a real pain, but now Steve Jobs is no longer with us I think a lot of the hysteria has died down, people are looking more objectively and realise that his 'thoughts on Flash' were motivated by a desire to protect App-store revenues rather than facts, and that in terms of development speed, maintenance and cross-browser support, Flash is still miles ahead of HTML/5. Provided you can cross the mobile bridge - and that is a pain, there's really no choice but to go with AIR apps - I'd go for Flash.
I want to create a simple visualization tool that would allow to represent my playing a midi keyboard on the screen. I play a relatively novel instrument type, called the harmonic table:
http://en.wikipedia.org/wiki/Harmonic_table_note_layout
http://www.soundonsound.com/newspix/image/axis49.jpg
And want to build tools to ease their use and to teach others how to use them.
However, I can't find a good way to get get midi into javascript environment (or, for that matter, Flash, or Java without a large helping of jiggery-pokery slightly beyond my reach, and the use of code from what look to be rather stale and abandoned open source projects. Neither language I am too enthused to work in for this project in any case).
Is there a suitable library or application that I have missed, that will enable me to do this?
While searching around for another solution (Flash based, using the functions of the Red5 Open source flash server - really ugly, but I'm desperate at this point) I found someone who had done exactly what I needed using Java to interface with the hardware. They had started with a flash solution and recently ported to Javascript. Yay!
http://www.abumarkub.net/abublog/?p=505
Don't let the caveats about 'proof of concept' discourage you: the basic functionality appears solid, at least with everything I have been able to throw at it.
So now I'm on my way, and so is anyone else who want to build javascript based midi interfaces/synths/what have you.
I can manipulate real-time midi in javascript! This is much better than flying cars and jetboots.
I have made a NPAPI browser plugin that lets you communicate in Javascript with MIDI devices.
Currently it works on OSX in Chrome and Safari, and on Windows in Chrome.
I am working on support for other browsers (Firefox, Internet Explorer) and other operating systems (Linux, Android, iOs)
See: http://abumarkub.net/abublog/?p=754
EDIT:
I recently published this module https://github.com/hems/midi2funk it's a node.js module that listens to midi and broadcast it through socket.io so if you have the luxury of running a node.js service locally together with your client side you might get some fun out of it...
~~~~~
A few others handy links, i kinda ordered in what i think would be most important for you:
midibridge.js - A Javascript API for interacting with MIDI devices
midi.js sequencing in javascript
jasmid - A Javascript MIDI file reader and synthesiser
Second web midi api working draft published - 11/12/2012
Jazz Soft - MIDI IN / OUT PLUGIN FOR BROWSER
edit: just realised the thread is old, hopefully the links will help someone ( :
The Web MIDI API is now real in Google Chrome 43+. I even wrote a library to make it easier to work with it. If you are still interested and do not care that it currently only works in Chrome, check it out: https://github.com/cotejp/webmidi
Nowadays browsers supports MIDI listening. All you need is
navigator.requestMIDIAccess().then(requestMIDIAccessSuccess, requestMIDIAccessFailure);
and listen keys
function requestMIDIAccessSuccess(midi) {
var inputs = midi.inputs.values();
for (var input = inputs.next(); input && !input.done; input = inputs.next()) {
console.log('midi input', input);
input.value.onmidimessage = midiOnMIDImessage;
}
midi.onstatechange = midiOnStateChange;
}
See working example here
Most browsers don't allow access to any hardware except the keyboard and mouse - for obvious security reasons, so it's unlikely that you could access a midi device unless it's plugged in as one of those devices.
You could try finding a driver that would translate midi output to key presses, and then deal with those in the browser, but this would be a single-computer solution only.
I am really excited by the upcoming Web MIDI API. As far as I know, its only under discussion and hasn't made it into any browsers yet.
Combined with the Web Audio API which has started to be implemented in some browsers already, it will be possible to have complete software synthesis in the browser. Exciting times.
Since Web MIDI API is still a draft, there is no way of direct access to MIDI events in the browser.
A simple workaround could be to write a small server where you register MIDI events and communicate them to your javascript using a websocket. This could be done quite easily in Python.
I need to implement a paint type program that will run on all major browsers as well as can be packaged as an app on the iPad/iPhone. After some investigation I have learned the following facts (correct me if any are wrong):
Javascript is entirely too slow to handle an app of this type by itself
HTML5 seems like a good solution, it has a canvas tag and everything and can easily be packaged into a UIWebView for a Cocoa app. However, I need to be able to reach the majority market for web users, which unfortunately includes most IE users who aren't using the public beta for IE 9.
Flash vs. Apple
Right now I'm debating just making two completely separate apps, one in OpenGL ES for iPad/iPhone and another in something like Flash for web browsers. I was just wondering if anyone had an immediate solutions in mind to make one app instead?
I'd question the accuracy of your first point (Javascript is too slow). Since it's possible to run Quake II in pure Javascript, it's likely that your paint application is less resource-intensive and thus could also run. Of course, one could ask how much optimisation effort and/or experience is required to write performant Javascript of this standard - I can't give you an answer there.
But I'm certainly sure that it's possible to get enough performance out of Javascript to implement such a paint tool, in any modern browser. That doesn't preclude the other options, though; using JS along with HTML 5 sounds like the ideal solution here for "out of the box" cross-platform compatibility.