Develop Reference

Intercept calls to HTML5 canvas element

I have a WEB application, that renders it's entire User Interface in an HTML5 canvas.
Note that I can't change the current application.
Currently, this application is being tested using Selenium.
This is done by simulating a click event at a given location in the browser window.
After the click has been executed, a sleep of 2 seconds is being performed to ensure that the entire UI is ready before moving to the next step.
Due to all the 'wait' statements, testing the application is very slow.
Therefore, I thought it was an idea to intercept all calls to the HTML5 canvas.
That way I can rely on the triggered events to know if the UI is ready to move to the next step.
Assume that I have the following code in my application that renders the canvas.
var canvas = document.getElementById("canvasElement");
var ctx = canvas.getContext("2d");
ctx.fillStyle = "green";
ctx.fillRect(10, 10, 100, 100);
Is there a way to intercept the 'fillRect' event?
I tought something along the lines:
var canvasProxy = document.getElementById("canvasElement");
canvasProxy.addEventListener("getContext", function(event) {
console.log("Hello");
});
var canvas = document.getElementById("canvasElement");
var ctx = canvas.getContext("2d");
ctx.fillStyle = "green";
ctx.fillRect(10, 10, 100, 100);
Unforuntately this is not working.
I've created a JSFiddle to play with the example.
https://jsfiddle.net/5cknym74/4/
Amy toughts?

I played a bit around with the JS API and it seems that the following might be working:
// SECTION: Store a reference to all the HTML5 'canvas' element methods.
HTMLCanvasElement.prototype._captureStream = HTMLCanvasElement.prototype.captureStream;
HTMLCanvasElement.prototype._getContext = HTMLCanvasElement.prototype.getContext;
HTMLCanvasElement.prototype._toDataURL = HTMLCanvasElement.prototype.toDataURL;
HTMLCanvasElement.prototype._toBlob = HTMLCanvasElement.prototype.toBlob;
HTMLCanvasElement.prototype._transferControlToOffscreen = HTMLCanvasElement.prototype.transferControlToOffscreen;
HTMLCanvasElement.prototype._mozGetAsFile = HTMLCanvasElement.prototype.mozGetAsFile;
// SECTION: Patch the HTML5 'canvas' element methods.
HTMLCanvasElement.prototype.captureStream = function(frameRate) {
console.log('INTERCEPTING: HTMLCanvasElement.prototype.captureStream');
return this._captureStream(frameRate);
}
HTMLCanvasElement.prototype.getContext = function(contextType, contextAttributes) {
console.log('INTERCEPTING: HTMLCanvasElement.prototype.getContext');
console.log('PROPERTIES:');
console.log(' contextType: ' + contextType);
return this._getContext(contextType, contextAttributes);
}
HTMLCanvasElement.prototype.toDataURL = function(type, encoderOptions) {
console.log('INTERCEPTING: HTMLCanvasElement.prototype.toDataURL');
return this._toDataURL(type, encoderOptions);
}
HTMLCanvasElement.prototype.toBlob = function(callback, mimeType, qualityArgument) {
console.log('INTERCEPTING: HTMLCanvasElement.prototype.toBlob');
return this._toBlob(callback, mimeType, qualityArgument);
}
HTMLCanvasElement.prototype.transferControlToOffscreen = function() {
console.log('INTERCEPTING: HTMLCanvasElement.prototype.transferControlToOffscreen');
return this._transferControlToOffscreen();
}
HTMLCanvasElement.prototype.mozGetAsFile = function(name, type) {
console.log('INTERCEPTING: HTMLCanvasElement.prototype.mozGetAsFile');
return this._mozGetAsFile(name, type);
}
Now that I can intercept the calls, I can find out which calls are responsible that draw a button and react accordingly.

Phonegap - ios9 File input

I have an issue with phonegap and iOS 9, this was working fine with ios 8, When you click on the file input it shows a cancel button in the middle of the screen that does nothing when tapped. The file input works fine in safari but in my built app, it doesn't.
I realize that there is a phonegap file uploader API, but I am using my app as a web browser to the mobile version of my website, so I haven't built a fully native app, this was a quick solution for me.
This could be because of the new action sheet style or the new option added to the action sheet in ios 9.
Does anyone have a solution that I can skip the action sheet and when clicking the input file I go directly to the camera roll? Or any fix for this?

I moved to the plugin - cordova-plugin-camera and this solved the problem.
After click on input type file:
e.preventDefault();
navigator.notification.confirm(
'Please select image', // message
function (buttonIndex) {
if (buttonIndex === 1) {
photoFromSource(navigator.camera.PictureSourceType.CAMERA);
} else {
photoFromSource(navigator.camera.PictureSourceType.PHOTOLIBRARY);
}
}, // callback to invoke with index of button pressed
'Image capture', // title
['Camera', 'Gallery'] // buttonLabels
);
function photoFromSource(source) {
var targetWidth = 640,
targetHeight = 640,
onSuccess = function (imageURI) {
var image = new Image(),
canvas = document.createElement('canvas'),
canvasContext = canvas.getContext('2d');
image.onload = function () {
canvas.width = image.width;
canvas.height = image.height;
canvasContext.drawImage(image, 0, 0, image.width, image.height);
var dataURL = canvas.toDataURL();
self.model.set('imageData', dataURL);
self.model.setDataAttr('image', true);
self.render();
};
image.src = imageURI;
},
onFail = function (message) {
// Ignore if no image seleted
if (!/^no image selected$/.test(message))
alert('Failed because: ' + message);
},
opts = {
quality: 50,
destinationType: navigator.camera.DestinationType.FILE_URI,
sourceType: source,
mediaType: navigator.camera.MediaType.PICTURE,
targetWidth: targetWidth,
targetHeight: targetHeight,
encodingType: navigator.camera.EncodingType.JPEG,
correctOrientation: true,
cameraDirection: navigator.camera.Direction.BACK,
saveToPhotoAlbum: false
};
try {
navigator.camera.getPicture(onSuccess, onFail, opts);
}
catch (e) {
alert('Could not capture image: ' + e);
}
};

Black image bug when exporting highcharts in Firefox

I am using highcharts to display several graphs on a webpage which display fine.
I have an export function that tries to combine the charts into a pdf. I am getting the svg of the chart and converting it to a jpeg image to be included in a pdf created by jsPDF.
Here is the code I am using to generate the images:
if ($('.chart').length > 0) {
var chartSVG = $('.chart').highcharts().getSVG(),
chartImg = new Image();
chartImg.src = "data:image/svg+xml;base64," + window.btoa(unescape(encodeURIComponent(chartSVG)));
var chartCanvas = document.createElement("canvas");
chartCanvas.width = 600;
chartCanvas.height = 400;
chartCanvas.getContext("2d").drawImage(chartImg, 0, 0, 600, 400);
var chartImgData = chartCanvas.toDataURL("image/jpeg");
}
This works perfectly in Chrome but in Firefox it just returns a black image.
Does anyone know what might be going wrong or has seen a similar issue?
Thanks for your help.
UPDATE
I've updated the code but now no image is appended to the pdf document, either in Chrome or Firefox.
if ($('.sales').length > 0) {
var chartSVG = $('.sales').highcharts().getSVG(),
chartImg = new Image();
chartImg.onload = function () {
var chartCanvas = document.createElement("canvas");
chartCanvas.width = 600;
chartCanvas.height = 400;
chartCanvas.getContext("2d").drawImage(chartImg, 0, 0, 600, 400);
var chartImgData = chartCanvas.toDataURL("image/jpeg");
}
chartImg.src = "data:image/svg+xml;base64," + window.btoa(unescape(encodeURIComponent(chartSVG)));
}
Not sure if I have the code in the correct place.
If I log 'chartImgData' to the console, both browsers generate a dataURI, but Firefox's version differs to Chromes.
UPDATE
Fixed the issue with black images. Now i'm struggling with how to return multiple images - how to nest multiple callbacks or is there another way?
Example: jsfiddle.net/wmuk489c/2
SOLVED
Thanks for your help #RobertLangson. fiddle updated with final working code should anyone need it: http://jsfiddle.net/wmuk489c/3/
FURTHER ISSUES:
My charts are dynamic and so may not always be present. I need to get an image from each graph that exists. If the graph does not exist, the 'getSVG' function fails, see example: http://jsfiddle.net/wmuk489c/4/
How should the img.onload work if the chart doesn't exist? The first chart in the callback may not be present either, so how would this work? Is there a better way to get the images?

setting chartImg.src causes an asynchronous load so you then need to do this...
chartImg.onload = function() {
var chartCanvas = document.createElement("canvas");
chartCanvas.width = 600;
chartCanvas.height = 400;
chartCanvas.getContext("2d").drawImage(chartImg, 0, 0, 600, 400);
var chartImgData = chartCanvas.toDataURL("image/jpeg");
doc.addImage(chartImgData, 'JPEG', 0, 0, 200, 100);
// You can only do this bit after you've added the image so it needs
// to be in the callback too
doc.save('test.pdf');
}
chartImg.src = ...
You've a race condition otherwise and I imagine you just happen to get away with it with the Chrome browser on your PC.
Here's your fiddle fixed.

How to pick(choose) multiple images using camera API at the same time in phonegap?

How to choose or pick multiple images at the same time in phonegap camera API when using Camera.DestinationType.FILE_URI. I am able to pick only one images at a time. I am able to pick multiple files(including txt,pdf..) in sdcard using this. So i want same like for images.
navigator.camera.getPicture(function(imageData) {
window.resolveLocalFileSystemURI(imageData, function(fileEntry) {
fileEntry.file(function(fileObj) {
}, onFail, {
quality : 50,
destinationType : Camera.DestinationType.FILE_URI
});
My cordova version 3.3, Jquery Mobile 1.3.2.
Please suggest any plugins are available to do this.

Use this Cordova multiple image selector plugin to choose multiple image at a time. It is good plugin for choose multiple images.
Download the above plugin and copy paste the java classes. Set the required permission. Don't forget to copy the res folder just copy and paste in your res folder.
Inside assets/www create imagepicker.js copy and paste the dowloaded imagepicker.js
In your index.html set like this:
<script type="text/javascript" src="cordova.js"></script>
<script type="text/javascript" src="imagepicker.js"></script>
<script type="text/javascript">
document.addEventListener("deviceready",onDeviceReady,false);
function onDeviceReady(){
window.imagePicker.getPictures(
function(results) {
for (var i = 0; i < results.length; i++) {
alert('Image URI: ' + results[i]);
// read file type and size and file name like below(in comment)
/* window.resolveLocalFileSystemURI(results[i], function(fileEntry){
fileEntry.file(function(fileObj) {
alert(fileEntry.name);
alert(fileObj.size);
alert(fileObj.type);
});
}, function (error) {
alert('Error: ' + error);
});*/
}
}, function (error) {
alert('Error: ' + error);
}
);
}
</script>
Note: This should work only cordova 3.0 and above and android 4.0 and above

Open CameraLauncher.java file and replace these lines
String resizePath = getTempDirectoryPath() + "/resize.jpg";
this.callbackContext.success("file://" + resizePath + "?" + System.currentTimeMillis());
to
String resizePath = getTempDirectoryPath() + "/resize"+System.currentTimeMillis()+".jpg";
this.callbackContext.success("file://" + resizePath);

var x=0;
function onPhotoDataSuccess(imageURI)
{
x++;
// Uncomment to view the base64-encoded image data
console.log(imageURI);
alert(imageURI);
// Get image handle
//
var y = 'smallImage'+x;
var smallImage = document.getElementById(y);
alert(smallImage);
smallImage.src = "data:image/jpeg;base64," + imageURI;
// Unhide image elements
//
smallImage.style.display = 'block';
// Show the captured photo
// The in-line CSS rules are used to resize the image
//
//var fso=new ActiveXObject("Scripting.FileSystemObject");
//fso.CopyFile("data:image/jpeg;base64," + imageURI,"file:///storage/sdcard/DCIM/");
alert(smallImage.src)
}
where x is the loop for doing the multiple image from camera as well as from photogallery

WebRTC: Is it possible to control the microphone and volume levels

I am working on a demo site which includes a slide-out widget that allows a user to place a call.
I am using the SIPml5 tool along with the webrtc2sip back end for handling the call. That part is all set up and properly working. So now I am looking at seeing if I can control the microphone and volume levels using sliders in the widget. Is this even possible? I look everywhere online and haven't had much luck.
I did find a couple sites that showed me how I can control the volume of the audio tag within the jQuery slider code. So I tried setting it up like the code below:
$(function() {
$( "#slider-spkr" ).slider({
orientation: "vertical",
range: "min",
min: 0,
max: 100,
value: 60,
slide: function( event, ui ) {
var value = $("#slider-spkr").slider("value");
document.getElementById("audio_remote").volume = (value / 100);
},
change: function() {
var value = $("#slider-spkr").slider("value");
document.getElementById("audio_remote").volume = (value / 100);
}
});
});
Unfortunately, this isn't working either. So I'm not sure if I am allowed to do this when using SIPml5, or if my jQuery code needs adjusted.
Has anyone else had any luck with adding microphone/volume controls? Thanks for your help.

Afaik it's impossible to adjust microphone volume. But you can switch it on/off by using stream api:
function toggleMic(stream) { // stream is your local WebRTC stream
var audioTracks = stream.getAudioTracks();
for (var i = 0, l = audioTracks.length; i < l; i++) {
audioTracks[i].enabled = !audioTracks[i].enabled;
}
}
This code is for native webrtc api, not sipML5. It seems they haven't implemented it yet. Here is not so clear receipt for it.

Well it is possible, but currently only in Chrome and with some assumptions.
I am not the auther, you can find inspiration for this code in this open-source library (SimpleWebRtc).
navigator.webkitGetUserMedia(constraints,
function(webRTCStream){
var context = new window.AudioContext();
var microphone = context.createMediaStreamSource(webRTCStream);
var gainFilter = context.createGain();
var destination = context.createMediaStreamDestination();
var outputStream = destination.stream;
microphone.connect(gainFilter);
gainFilter.connect(destination);
var filteredTrack = outputStream.getAudioTracks()[0];
webRTCStream.addTrack(filteredTrack);
var originalTrack = webRTCStream.getAudioTracks()[0];
webRTCStream.removeTrack(originalTrack);
},
function(err) {
console.log("The following error occured: " + err);
}
);
The trick is to modify the stream and then replace the audio track of current stream with audio track of modified stream (taken from MediaStreamDestination stream).
DISCLAIMER:
This doesn't work in FireFox as of version 35, since they merely didn't implement MediaStream.addTrack/removeTrack. I use this check currently
this.micVolumeIsSupported = function() {
var MediaStream = window.webkitMediaStream || window.MediaStream;
return !!MediaStream.prototype.addTrack && !!MediaStream.prototype.removeTrack;
};
_gainSupported = this.micVolumeIsSupported();
This has a limitation in Chrome due to a bug with stopping stream with mixed up tracks. You might wish to restore these tracks before closing connection or on connection interruption;
this.restoreTracks = function(){
if(_gainSupported && _tracksSubstituted){
webRTCStream.addTrack(originalTrack);
webRTCStream.removeTrack(filteredTrack);
_tracksSubstituted = false;
}
};
This works for me