Develop Reference

How to send blob appended in formData to php

Issue : While uploading large image files i recognized that while uploading on my AWS server having 1gb memory uses it's full capacity, it goes upto 932 mb usage which causes crash to the process. I was saving that image in the form of DataURI and then I read somewhere that saving it in the form of blob can solve my problem. So i want to append that blob to formData and send to server and this is the reason i come up with this question. However if any else suggestion regarding the same problem to save image more efficient way when memory is concerned, will be appreciated.
Motive
I want to send an image to the server side as in the form of a blob.
What I have done
I am currently having a dataURI which I have converted into a blob. Further, i append that blob to formData and try to send it to server side/php using ajax.
JAVASCRIPT:
function convertURIToImageData(dataURI) {
// convert base64/URLEncoded data component to raw binary data held in a string
var byteString;
if (dataURI.split(',')[0].indexOf('base64') >= 0)
byteString = atob(dataURI.split(',')[1]);
else
byteString = unescape(dataURI.split(',')[1]);
// separate out the mime component
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0];
// write the bytes of the string to a typed array
var ia = new Uint8Array(byteString.length);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
return new Blob([ia], {type:mimeString});
}
//
const dataURIconverter = () =>{
let img;
var image = new Image();
image.crossOrigin = 'anonymous'; // cross domain
// create an empty canvas element
var canvas = document.createElement("canvas"),
canvasContext = canvas.getContext("2d");
image.onload = function () {
//Set canvas size is same as the picture
canvas.width = image.width;
canvas.height = image.height;
// draw image into canvas element
canvasContext.drawImage(image, 0, 0, image.width, image.height);
// get canvas contents as a data URL (returns png format by default)
var dataURL = canvas.toDataURL();
// console.log(dataURL)
let blob = convertURIToImageData(dataURL)
console.log(blob)
var formData = new FormData();
formData.append('blobImage',blob)
$.ajax({
type: 'POST',
url: 'check.php',
data: formData,
processData: false
}).done(function(data) {
console.log(data);
})
}
image.src = "https://static.pexels.com/photos/248797/pexels-photo-248797.jpeg"
}
dataURIconverter()
PHP
<?php
var_dump($_POST['blobImage'])
var_dump($_POST);
//var_dump($_FILES['image']);
//$name = $_FILES['image']['tmp_name'];
//echo $name;
//echo $_FILES['image']['tmp_name'];
//$status = move_uploaded_file($name, $_FILES['image']['name']);
//echo 'successfully stored at '.$_SERVER['HTTP_HOST'];
?>
Error
I am receiving null as in console and i also checked the headers where i see formData with the name
As you can see, $_POST showing the blob but $_POST['blobImage'] is showing null.
Solution I require:
i am not that quick to php so i am not sure if i am sending the blob in the right way or receiving it.
I have provided my all possible efforts i have taken to achieve my motive.
Thanks to the community for help.

Add the following three properties on your jQuery Ajax call , they are required for blobs :
cache: false,
contentType: false,
processData: false
Then do not use formData in the data property of your Ajax Call , you simply need to add your created blob.
Also add a small rendering callback (apart from the console.log you already use) to print the Image. Your AJAX call gets like this :
$.ajax({
type: 'POST',
url: 'check.php',
data: blob,
cache: false,
contentType: false,
processData: false
}).done(function(data) {
document.write("<img src='"+data+"'></img>");
})
Change your PHP code to the following :
<?php
$res = file_get_contents("php://input");
echo "data:image/jpg;base64,".base64_encode($res);
?>
As far as the "php://input" use is concerned. It returns all the raw data that come after the headers of your request and it does not care what type they are which is pretty handy in most cases. Whereas $_POST will only wrap the data that have been passed with the following Content-Types :
application/x-www-form-urlencoded
multipart/form-data
If you really want to use FormData then you can change the request to the following :
$.ajax({
type: 'POST',
url: 'check.php',
data: formData,
cache: false,
contentType: false,
processData: false
}).done(function(data) {
console.log(data);
})
And you should also change your PHP file to get the $_FILE. Sending data this way , the Content-Type of the Request will be "multipart/form-data" which will have blobs , images and generally files on the $_FILES and the rest on the $_POST so the "php://input" will not be helpful.
<?php
var_dump($_FILES);
?>
Also keep in mind that when uploading blobs this way , they will get a random name , if you are not going to be generating filenames on the Server-Side (which you probably should in most cases) and want a specific name designated by the uploader , then you can pass it along with the FormData like :
formData.append('blobImage',blob, "MyBloBName");

If you set contentType: false in your jQuery Ajax call , you can still use the same code with formData and then access the file on the server through $_FILES['blobImage']

The problem is that $_REQUEST, and therefore $_GET and $_POST objects have a limitation to the number of characters available to them.
post_max_size
in PHP.ini controls the maximum size of post.
Browsers and their implementations of $_GET control the limit of a $_GET request. As it appears in their URL bar. For example IE9's limit is 2000 characters so if your blob ends up as anything more than 2000 characters in the $_GET Request. The general consensus is that $_GET requests should be much less than 255 bytes. And if you approach this limit be careful because older browsers and protocols are completely unprepared for this.

Javascript: converting binary data in Uint8Array to string corrupting file

I'm trying to upload a video file in chunks to a server via a standard jQuery $.ajax call. The process is simple:
Use the slice() method on the file object to read a chunk
Use FileReader.readAsArrayBuffer to read the resulting blob
Create a Uint8Array of that result in the FileReader.onload callback
Use String.fromCharCode.apply(null, intArrayName) to convert it into a binary string
Upload that binary string as a chunk in the AJAX call's data property
When I finish uploading all the chunks, the server's API complains that the file upload is incomplete. To test this method, I converted the concatenated binary strings into a Blob and had Chrome save it to my downloads folder, only to find out my media player said the file was unplayable.
I've read posts on this site and other articles suggesting that directly converting binary data represented by integers to strings results in a loss of data, and that the suggestion is convert it to a base64-encoded string, since Javascript doesn't have a StreamContent object like C# does.
The problem is that even if I set Content-Transfer-Encoding to base64 I don't think the API (which wasn't written by us) picks up on it and can tell that a base64 decoding is needed.
My question is: is this the only sure-fire way of sending video data over an AJAX call safely? If not, how else can I send it in the data field of my AJAX request? There are probably thousands of uploaders like this on the Internet but nowhere can I find solid documentation on how to do this. If it turns out that the server needs to expect base64 encoding, we might have to write a middle-man API to do this decoding for us.

Thanks to Patrick Evans, it turns out all I needed to do was upload the blob itself:
function upload(file) {
var offset = 0; // what's been sent already
var fileSize = file.size;
var chunkSize = 64 * 1024; // bytes
while (offset < fileSize) {
var blob = file.slice(offset, chunkSize + offset);
var urlToUse = ((offset + blob.size) >= fileSize) ? videoFinishBaseUrl : videoContinueBaseUrl;
$.ajax({
url: urlToUse,
method: 'POST',
data: blob,
headers: requestHeaders,
contentType: false,
processData: false,
cache: false,
async: false,
success: function(data) {
jQuery('#uploadmsg').text('Finished offset ' + offset);
offset += blob.size;
},
error: function(err) {
jQuery('#uploadmsg').text(err);
}
});
}
};

Javascript Blob changes file

I really have no prior knowledge to Blob objects in JavaScript aside from what I've read here but need to use them to convert a string of binary data into a .xls file which I then make available to the user. The catch is when I construct a blob, get the location, and open up the file to look at it, it opens up saying
The file you are trying to open is in a different format that
specified by the file extension. Verify that the file is not
corrupted and is from a trusted source before opening the file.
(I know this data is incorrect because when I submit the form normally I get the file correctly and don't have this issue)
This is done in an ajax call and the success functions parameter data is the binary data.
$("#fileForm").submit(function(){
var fileData = $("#fileInputElmt").prop("files")[0];
var data = new FormData();
data.append("upload",fileData);
var url = "process.action?" + $("#fileForm").serialize();
$.ajax({
type: "POST",
url:url,
data:data,
cache:false,
contentType:false,
processData:false,
success:function(data){
var bb = new Blob([data],
{ type: 'application/vnd.ms-excel',endings:'native'});
var bUrl = URL.createObjectURL(bb);
window.open(bUrl,"_self");
hideProgressBar();
},error:function(data){
hideProgressBar();
}
});
return false;
});
Am I doing something wrong? or is there a better way of doing this?

jQuery `ajax` call throws an error when an XML containing escaped (!) ampersand is retrieved

A client-side script takes the text within a text input, "wraps" it within an XML block and sends it to a server that stores the information in a MySQL database.
As a first step before wrapping the input value, I escape the "&" characters like so:
var copyright = copyright.replace(/&/g,"&");
The resulting XML data block is sent to the server using jquery's ajax method:
var copyright = copyright.replace(/&/g,"&"),
xml = "<request><session>"+session+"</session><space>"+space_id+"</space><view>"+view_id+"</view><copyright>"+copyright+"</copyright></request>",
url = "hidden URL";
$.ajax({
type: "POST",
url: url,
contentType: "text/xml; charset=UTF-8",
dataType: "xml;charset=UTF-8",
data: xml
});
Later after this operation, the content that was previously saved within the database needs to be retrieved and displayed within a web page:
$.ajax({
type: "POST",
url: url,
dataType: 'xml',
data: xmlString,
success: function(xml) {
var XML = $(xml);
// Process the data retrieved
},
error: function(jqXHR, textStatus, errorThrown) {
var XML = $(jqXHR.responseText);
console.log("error: "+textStatus+"\n"+errorThrown);
}
});
If an ampersand was typed in the input field and then saved, when trying to load the page that displays the same previously saved content, the ajax call breaks and runs down the error event handler, with the following error:
error: parsererror
Error: Invalid XML: <?xml version="1.0" encoding="UTF-8"?><response><target>
<target_id>2095466</target_id>
<plot>20029/13</plot>
<builder>Lemminkäinen</builder>
<housing_form>vm</housing_form>
<block_name></block_name>
<finnish_year>2013</finnish_year>
<target_name>As Oy Helsingin Saukonranta</target_name>
<target_address>Saukonpaadenranta 8</target_address>
<office_space></office_space>
<purpose></purpose>
<reservations></reservations>
<contacts></contacts>
<infoflag>2</infoflag>
<views>
<view>
<view_id>2095468</view_id>
<copyright>B&M</copyright>
</view>
</views>
</target>
<status>OK</status><errormsg></errormsg></response>
What is it that I'm doing wrong? Am I escaping the characters wrongly, or is it something else?
This question may seem to be a duplicate, but to me it doesn't seem like it since the ampersand characters have been escaped prior to being stored. I even tried adding additional (1, then two) amp;s to the escape string, but the result is EXACTLY the same.

It turns out that the problem actually came from the server (to which I did not have access), the script that handled the requests did not escape the ampersand characters correctly, even though they were on the client-side.
Bellow is a JavaScript function that escapes all (?) special characters used with XML, just in case someone needs it:
function escapeXML(string){
var str = string;
str = str.replace(/\&/g,"&");
str = str.replace(/\>/g,">");
str = str.replace(/\</g,"<");
str = str.replace(/\"/g,""");
str = str.replace(/\'/g,"&apos;");
return str;
}

The problem is the ä character of the Lemminkäinen in the builder node, as pointed by Shahid. When the Lemminkäinen text is UTF-8 decoded, the ä would be part of a two-characters UTF-8 encoding. So the UTF-8 decoder would try to decode äi, which is not a valid character sequence. The correct UTF-8 encoded character for ä is ä, or 0xC3, 0xA4 in binary. Thus, the full UTF-8 encoded text should be Lemminkäinen.
When the reported XML data is saved in an XML file then opened with a web browser, it'll fail on all major web browsers: Chrome ("Encoding error"), Firefox ("not well-formed"), Safari ("Encoding error"), MSIE ("An invalid character was found in text content."), and Opera ("illegal byte sequence in encoding").
Since the XML data came from the server, it's likely that the script that posted the builder data didn't specify an UTF-8 character set (there's no indication that the provided codes are the one that does it). It may have caused by old script which by now, is already fixed, but the damage has already been done. i.e.: incorrect data format was added into database. Manual input into database is also a possible cause during server maintenance.

Save chart image with open flash chart2

I am using Open Flash Chart 2 to create some graphs. I want to be able to save an image of the graph, which OFC2 supplies some methods to accomplish this. I used the example on the OFC2 site to directly display the raw image data on the page, but that does not work in IE6, which most of our users are using (I know, I know).
I switched to using the OFC2 method, post_image to post the raw image data to the server. I use a Perl script to receive the image data, save it to a file, and I can view the image. The unfortunate part about using the post_image method is that ActionScript throws an error when saving the image:
Error #2101: The String passed to URLVariables.decode() must be a URL-encoded query string containing name/value pairs.
Which apparently is a bug in Adobe - see this page. Because of this error, the post_image method does not complete successfully, so the javascript callback won't ever fire - I basically don't have a way to tell if the image was saved successfully.
So, I thought I would use the get_img_binary method of OFC2 to get the binary data of the image, and use jQuery to post the binary data to my Perl script.
I cannot figure out how to send the binary data correctly, or how to let my Perl script receive the binary data correctly, or both.
Here is my jQuery function:
var chartObj = $("#chart_object").get(0);
$.ajax({
type: "POST",
url: 'download_image.pl',
//contentType: 'application/octet-stream',
contentType: 'image/png',
//processData: false,
//data: { imgData: chartObj.get_img_binary() },
data: chartObj.get_img_binary(),
dataType: "text",
success: function(data) {
console.log( data );
}
});
You can see from some of my commented out lines that I have tried various contentTypes and other settings of the Ajax call.
The Ajax call is sending some data, but it doesn't appear to be binary. I think it is a base64 representation of the binary data.
Does anyone have any ideas on how to send binary data from javascript to the server?
The Perl script I have works fine for the post_image method, so I don't think the problem is there?
Thanks in advance!

I seem to have stumbled onto the solution.
Here is my ajax call:
var chartObj = $("#chart_object").get(0);
$.ajax({
type: "POST",
url: 'download_image.pl',
contentType: 'application/octet-stream',
processData: false,
data: imgData,
dataType: "text",
success: function(data) {
console.log( data );
}
});
And here is my Perl snippet to process/save the image:
use CGI qw(:standard);
use MIME::Base64;
...
my $img64 = param('POSTDATA');
my $img = decode_base64( $img64 );
...
#then print $img out to a file in binary mode
I had to decode the base64 representation of the PNG file, and then save it to a file.

i've got trouble too with using IE6 and OFC2 for saving image... So here are the scripts i use (javascript + PHP)
i know it's not very beautifull but jQuery doesn't want to work in a popup created via window.open('') on my IE6 so i decided to use a "old school method" to get it...
// javascript in the page displaying the flash chart
OFC = {};
OFC.jquery = {
name: "jQuery",
image: function(src) { return '<img src="data:image/png;base64,' + $('#'+src)[0].get_img_binary() + '" \/>'},
popup: function(src) {
var img_tag = OFC.jquery.image(src);
var img_win = window.open('', 'imagesave');
img_win.document.write('<html><head><title>imagesave<\/title><\/head><body>'+ img_tag + '<\/body><\/html>');
img_win.document.close();
},
popupie: function(src) {
var img_data = "image/png;base64,"+$("#"+src)[0].get_img_binary();
var img_win = window.open('', 'imagesave');
with(img_win.document) {
write('<html>');
write('<head>');
write('<title>imagesave<\/title>');
write('<\/head>');
write('<body onload="document.forms[0].submit()">');
write('<form action="\/ofc\/base64post.php" method="post">');
write('<input type="hidden" name="d" id="data" value="'+img_data+'" \/>');
write('<\/form>');
write('<div><img src="\/ofc\/ajax-loader.gif" border="0" alt="" \/><\/div>');
write('<div style="font-family: Verdana;">Please wait<br \/>After you can save the image<\/div>');
write('<\/body>');
write('<\/html>');
}
img_win.document.close();
}
}
function save_image() { // this function is automatically called
if ($.browser.msie)
OFC.jquery.popupie("my_chart"); // only for IE navigators
else
OFC.jquery.popup("my_chart"); // for the others
}
so, when we use the save_image() function (which is automaticaly called when you right clic dans select "Save Image Locally" on the flahs chart)
the image of the chart is tranfered to the popup and the data (base64 binary image) are posted to a php script /ofc/base64post.php that rander the picture :
<?php
// base64post.php
$data = split(';', $_POST['d']);
$type = $data[0];
$data64 = split(',', $data[1]);
$pic = base64_decode($data64[1]);
header("Content-type: $type");
echo $pic;
?>
hope that help someone !