I built a simple project using javascript and FastAPI. I would like to POST canvas image data in javascript to the server(FastAPI).
But, I get the 422 Unprocessable Entity error
# FastAPI
#app.post("/post")
async def upload_files(input_data: UploadFile = File(...)):
return JSONResponse(status_code = 200, content={'result' : 'success'})
// javascript
let canvas = document.getElementById("canvas");
let ctx = canvas.getContext("2d");
imageData = ctx.getImageData(0, 0, 112, 112);
fetch("http://{IP}/post", {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
input_data: imageData,
}),
})
.then((response) => response.json())
here is my code. I am new one in javascript. so I don't know how can post data to server.
How can I do? Thanks in advance.
MatsLindh's way is correct but you can try upload image by formdata so you haven't convert to base64 then convert back to image
const dataURItoBlob = function(dataURI : string) {
// convert base64/URLEncoded data component to raw binary data held in a string
var byteString : string;
if (dataURI.split(',')[0].indexOf('base64') >= 0)
byteString = atob(dataURI.split(',')[1]);
else
byteString = unescape(dataURI.split(',')[1]);
// separate out the mime component
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0];
// write the bytes of the string to a typed array
var ia = new Uint8Array(byteString.length);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
return new Blob([ia], {type:mimeString});
}
const url = canvas.toDataURL()
const file = dataURItoBlob(url)
//add file to formdata
const form = new FormData()
form.append('file', item.file)
//post this form in body
UploadFile expects a file being posted to the endpoint in the multipart/form-data format - i.e. as a regular POST with a bit of added metadata.
Since you're retrieving the image data and submitting it as JSON body, that won't match the expected format. You also want to use canvas.toDataURL() to convert the content to base64 before trying to submit it through your API to avoid any UTF-8 issues with JSON.
const pngData = canvas.toDataURL();
// then in your request:
body: JSON.stringify({
input_data: pngData,
}),
To handle this request change your view function signature to receive an input_data parameter (and use a better endpoint name than post):
#app.post("/images")
async def upload_files(input_data: str = Body(...)):
# check the expected format (content-type;<base64 content>)
# .. and decode it with base64.b64decode()
return {'content': input_data}
Related
I'm using code from this and that answer to send a base64 encoded image to a python FastAPI backend.
The client side looks like this:
function toDataURL(src, callback, outputFormat) {
var img = new Image();
img.crossOrigin = 'Anonymous';
img.onload = function() {
var canvas = document.createElement('CANVAS');
var ctx = canvas.getContext('2d');
var dataURL;
canvas.height = this.naturalHeight;
canvas.width = this.naturalWidth;
ctx.drawImage(this, 0, 0);
dataURL = canvas.toDataURL(outputFormat);
callback(dataURL);
};
img.src = src;
if (img.complete || img.complete === undefined) {
img.src = "data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///ywAAAAAAQABAAACAUwAOw==";
img.src = src;
}
}
function makeBlob(dataURL) {
var BASE64_MARKER = ';base64,';
if (dataURL.indexOf(BASE64_MARKER) == -1) {
var parts = dataURL.split(',');
var contentType = parts[0].split(':')[1];
var raw = decodeURIComponent(parts[1]);
return new Blob([raw], { type: contentType });
}
var parts = dataURL.split(BASE64_MARKER);
var contentType = parts[0].split(':')[1];
var raw = window.atob(parts[1]);
var rawLength = raw.length;
var uInt8Array = new Uint8Array(rawLength);
for (var i = 0; i < rawLength; ++i) {
uInt8Array[i] = raw.charCodeAt(i);
}
return new Blob([uInt8Array], { type: contentType });
}
...
toDataURL(
images[0], // images is an array of paths to images
function(dataUrl) {
console.log('RESULT:', dataUrl);
$.ajax({
url: "http://0.0.0.0:8000/check/",
type: 'POST',
processData: false,
contentType: 'application/octet-stream',
data: makeBlob(dataUrl)
}).done(function(data) {console.log("success");}).fail(function() {console.log("error");});
}
);
And the server side is as follows:
#app.post("/check")
async def check(file: bytes = File(...)) -> Any:
// do something here
I'm only showing the signature of the endpoint because for now nothing much is happening in it anyway.
Here is the output of the backend when I call it as shown above:
172.17.0.1:36464 - "OPTIONS /check/ HTTP/1.1" 200
172.17.0.1:36464 - "POST /check/ HTTP/1.1" 307
172.17.0.1:36464 - "OPTIONS /check HTTP/1.1" 200
172.17.0.1:36464 - "POST /check HTTP/1.1" 422
So, in short, I keep getting 422 error codes, which means that there is a mismatch between what I send and what the endpoint expects, but even after some reading I'm still not clear on what exactly is the issue. Any help would be most welcome!
As described in this answer, uploaded files are sent as form data. As per FastAPI documentation:
Data from forms is normally encoded using the "media type"
application/x-www-form-urlencoded when it doesn't include files.
But when the form includes files, it is encoded as
multipart/form-data. If you use File, FastAPI will know it has to get
the files from the correct part of the body.
Regardless of what type you used, either bytes or UploadFile, since...
If you declare the type of your path operation function parameter as
bytes, FastAPI will read the file for you and you will receive the
contents as bytes.
Hence, the 422 Unprocessable entity error. In your example, you send binary data (using application/octet-stream for content-type), however, your API's endpoint expects form data (i.e., multipart/form-data).
Option 1
Instead of sending a base64 encoded image, upload the file as is, either using an HTML form, as shown here, or Javascript, as shown below. As others pointed out, it is imperative that you set the contentType option to false, when using JQuery. Using Fetch API, as shown below, it is best to leave it out as well, and force the browser to set it (along with the mandatory multipart boundary). For async read/write in the FastAPI backend, please have a look at this answer.
app.py:
import uvicorn
from fastapi import File, UploadFile, Request, FastAPI
from fastapi.templating import Jinja2Templates
app = FastAPI()
templates = Jinja2Templates(directory="templates")
#app.post("/upload")
def upload(file: UploadFile = File(...)):
try:
contents = file.file.read()
with open("uploaded_" + file.filename, "wb") as f:
f.write(contents)
except Exception:
return {"message": "There was an error uploading the file"}
finally:
file.file.close()
return {"message": f"Successfuly uploaded {file.filename}"}
#app.get("/")
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
templates/index.html
<script>
function uploadFile(){
var file = document.getElementById('fileInput').files[0];
if(file){
var formData = new FormData();
formData.append('file', file);
fetch('/upload', {
method: 'POST',
body: formData,
})
.then(response => {
console.log(response);
})
.catch(error => {
console.error(error);
});
}
}
</script>
<input type="file" id="fileInput" name="file"><br>
<input type="button" value="Upload File" onclick="uploadFile()">
If you would like to use Axioslibrary for the upload, please have a look at this answer.
Option 2
If you still need to upload a base64 encoded image, you can send the data as form data, using application/x-www-form-urlencoded as the content-type; while in your API's endpoint, you can define a Form field to receive the data. Below is a complete working example, where a base64 encoded image is sent, received by the server, decoded and saved to the disk. For base64 encoding, the readAsDataURL method is used on client side. Please note, the file writing to the disk is done using synchronous writing. In scenarios where multiple (or large) files need to be saved, it would be best to use async writing, as described here.
app.py
from fastapi import Form, Request, FastAPI
from fastapi.templating import Jinja2Templates
import base64
app = FastAPI()
templates = Jinja2Templates(directory="templates")
#app.post("/upload")
def upload(filename: str = Form(...), filedata: str = Form(...)):
image_as_bytes = str.encode(filedata) # convert string to bytes
img_recovered = base64.b64decode(image_as_bytes) # decode base64string
with open("uploaded_" + filename, "wb") as f:
f.write(img_recovered)
return {"message": f"Successfuly uploaded {filename}"}
#app.get("/")
def main(request: Request):
return templates.TemplateResponse("index.html", {"request": request})
templates/index.html
<script type="text/javascript">
function previewFile() {
const preview = document.querySelector('img');
const file = document.querySelector('input[type=file]').files[0];
const reader = new FileReader();
reader.addEventListener("load", function () {
preview.src = reader.result; //show image in <img tag>
base64String = reader.result.replace("data:", "").replace(/^.+,/, "");
uploadFile(file.name, base64String)
}, false);
if (file) {
reader.readAsDataURL(file);
}
}
function uploadFile(filename, filedata){
var formData = new FormData();
formData.append("filename", filename);
formData.append("filedata", filedata);
fetch('/upload', {
method: 'POST',
body: formData,
})
.then(response => {
console.log(response);
})
.catch(error => {
console.error(error);
});
}
</script>
<input type="file" onchange="previewFile()"><br>
<img src="" height="200" alt="Image preview...">
I have a page to download a file from the Postgres database. Now I can hit the following URL to view the file content present in the database(stored as bytes)-HTTP://sandbox4.wootz.io:8080/api/blob/1/UploadFile/hope%20real.txt
Since the data is stored in a column of type bytes(byte array) when I click the download button it downloads the file and when I see its contents it is displayed as a byte array.
file.txt(contents)
[\x58595a5052415445454b3123473b4c534e44204e474f49574853474849444748445348474d70253335]
download functionality
axios({
url: 'api/store/blob/UploadFile/' + data.chosenfile,
method: 'GET',
headers: {'session_id': data.sessionid},
responseType: 'arraybuffer'
}).then(response => {
console.log(response.data); //displays nothing (empty)
var fileURL = window.URL.createObjectURL(new Blob([response.data]));
console.log('fileURL is'+fileURL)
var fileLink = document.createElement('a');
console.log('fileLink is'+fileLink)
fileLink.href = fileURL;
fileLink.setAttribute('download', data.chosenfile);
document.body.appendChild(fileLink);
fileLink.click();
)
console.log of the response object
{"data":{},"status":200,"statusText":"OK","headers":{"access-control-allow-origin":"*","connection":"keep-alive","content-length":"86","content-type":"text/html; charset=utf-8","date":"Mon, 06 Jul 2020 18:22:23 GMT","etag":"W/\"56-Vaz0hG1/FIgtEurgvK+wOU+4F4M\"","x-powered-by":"Express"},"config":{"url":"api/store/blob/UploadFile/hope real.txt","method":"get","headers":{"Accept":"application/json, text/plain, */*","Access-Control-Allow-Origin":"http://localhost","session_id":"c5b3b878-771e-4472-84eb-6de15686effa"},"transformRequest":[null],"transformResponse":[null],"timeout":0,"responseType":"arraybuffer","xsrfCookieName":"XSRF-TOKEN","xsrfHeaderName":"X-XSRF-TOKEN","maxContentLength":-1},"request":{}}
Uploadfile part of my code(this is how the files was uploaded to database)
function readFileAsync(file) {
return new Promise((resolve, reject) => {
let reader = new FileReader();
reader.onload = () => {
var base64Url = reader.result;
console.log(base64Url) //ENITRE BASE64 URL
resolve(base64Url.substr(base64Url.indexOf(',') + 1)); //will return only the base64string part from the base64 url
};
reader.onerror = reject;
reader.readAsDataURL(file);
})
}
async function uploadFile(path, data) {
try {
let base64string = await readFileAsync(data.chosenfile);
console.log('base64 content is'+ base64string)
let response = await axios({
method: 'post',
url: 'api/store/blob' + path,
headers: {'session_id': data.sessionid},
data: {"id":data.chosenfile.name, "file": base64string }
});
if (response.status == 200) {
console.log(response.status);
}
return response.data;
} catch (err) {
console.error(err);
}
}
what am i doing wrong? why am i getting the file contents as [\x58595a5052415445454b3123473b4c534e44204e474f49574853474849444748445348474d70253335] ? What should I do to get the actual file's content in the downloaded file?
NOTE:With respect to the upload part, I am using the same strategy for all kind of files(excel documents,txt files etc) which is encoding it to base64 encoded string before passing it in the axios post payload.Now this payload is passed to another project called data-manager (interacts with postgres database).so when this data-manager project recieves the payload that i sent, it converts it to bytes [] before inserting to the table column of type bytea.So ultimately when i download any file from this table i will get the file contents also in bytea format.
Your file is a correct ASCII text file with content [\x58595a5052415445454b3123473b4c534e44204e474f49574853474849444748445348474d70253335] literally.
That you get this content when you download it is perfectly fine, since it's what this file's content is.
What you want is to parse that content from the Hex representation they used to an actual ArrayBuffer to finally read that again as UTF-8 text (or any encoding respecting ASCII).
This has to be done after you do download the file and read it as text.
You first extract the actual bytes sequence as Hex from that [\x - ] wrapper, then you split the resulting string at every two chars to get hex values of every bytes, and finally you parse it into an Uint8Array to get back the original data:
// for StackSnippet we need to hardcode the response
// OP would have to make its request return that string
const response = { data: String.raw`[\x58595a5052415445454b3123473b4c534e44204e474f49574853474849444748445348474d70253335]` };
// .then( (response) => {
const encoded_text = response.data;
// remove leading "[\x" and final "]"
const encoded_data = encoded_text.slice( 3, -1 );
// split at every two chars, so we can get 0xNN, 0xNN
const hex_bytes = encoded_data.match( /.{2}/g );
// as numbers (0 - 255)
const num_bytes = hex_bytes.map( (hex) => parseInt( hex, 16 ) );
// wrap in an Uint8Array
const view = new Uint8Array( num_bytes );
// from there you can generate the Blob to save on disk
download( new Blob( [ view ] ), "file.txt" );
// but if you want to read it as UTF-8 text, you can:
const as_text = new TextDecoder().decode( view );
console.log( as_text );
// } );
function download( blob, filename ) {
const anchor = document.createElement( "a" );
anchor.href = URL.createObjectURL( blob );
anchor.download = filename;
anchor.textContent = "click to download";
document.body.append( anchor );
}
I have a pdf file which is generated into my local server with my server side code. I want to send a request to the another server requesting POST. The post method take parameter as FormData where formdata types
one is string and another is file type.
content-type
form-data
Body
PDF file (file type)
string value
Is it possible to make the POST request without browsing the file location?
Doing some R&D I have overcome this problem with following some steps, as there is no way to get the file object from the physical location automatically in client side (basically in js) except browsing for security reason.
In my local server I have created a REST service. which response base64 string of the desired file.
Than I call the REST api from my javaScript and as a response I receive the base64 string. And than I convert it into bytes array and than Blob object and than File object.
base64 string==>bytes array==>Blob object==>File object
var base64 = this.getpdfFromLocal() //get the base64 string
var byteArray= this.base64ToByte(base64 );
var file = this.getFileFromByteArray(byteArray);
//return the byte array form the base64 string
MyApi.prototype.base64ToByte= function(base64) {
var binaryString = window.atob(base64);
var binaryLen = binaryString.length;
var bytes = new Uint8Array(binaryLen);
for (var i = 0; i < binaryLen; i++) {
var ascii = binaryString.charCodeAt(i);
bytes[i] = ascii;
}
return bytes;
};
MyApi.prototype.getFileFromByteArray=function(byteArray) {
var blob = new Blob([byteArray]);
var file = new File([blob], "resource.pdf");
return file;
};
Lastly I make from data using file object and send request the another server REST web services.
var formdata = new FormData();
formdata.append("some_value", "Some String");
formdata.append("file", file);
var url = "http://yoururl.com";
var result =$.ajax({
url : url ,
type : 'POST',
data : formdata,
contentType : false,
cache : false,
processData : false,
scriptCharset : 'utf-8',
async : false
}).responseText;
I am using the File reader in JavaScript,i need to Post my image to WebApi and convert it into byte Array and save it in server,Its working fine,Now my problem is base64 string increasing the size of image, Let say if i upload image of 30Kb, it is storing has 389Kb in server,How i can save in same size or reduce size of image need help
//File Reader
function OnFileEditImageEntry(file) {
var reader = new FileReader();
reader.onloadend = function (evt) {
var ImageBase64 = evt.target.result;
return ImageBase64 ;
};
reader.readAsDataURL(file);
}
//WEB API//
public IHttpActionResult UpdateUserDetails(ImageModel model)
{
try
{
if (model.ImageBase64 != "")
{
var PicDataUrl = "";
string ftpurl = "ftp://xxx.xxxxx.xxxx/";
var username = "xxx";
var password = "xxxxx";
string UploadDirectory = "xxxx/xx";
string FileName =model.ImageFileName;
String uploadUrl = String.Format("{0}{1}/{2}", ftpurl, UploadDirectory,FileName);
FtpWebRequest req = (FtpWebRequest)FtpWebRequest.Create(uploadUrl);
req.Proxy = null;
req.Method = WebRequestMethods.Ftp.UploadFile;
req.Credentials = new NetworkCredential(username, password);
req.EnableSsl = false;
req.UseBinary = true;
req.UsePassive = true;
byte[] data =Convert.FromBase64String(model.ImageBase64);
req.ContentLength = data.Length;
Stream stream = req.GetRequestStream();
stream.Write(data, 0, data.Length);
stream.Close();
}
}
}
Send the raw binary instead of increasing the size ~30% with base64/FileReader
with fetch
// sends the raw binary
fetch('http://example.com/upload', {method: 'post', body: file})
// Append the blob/file to a FormData and send it
var fd = new FormData()
fd.append('file', file, file.name)
fetch('http://example.com/upload', {method: 'post', body: fd})
With XHR
// xhr = new ...
// xhr.open(...)
xhr.send(file) // or
xhr.send(fd) // send the FormData
Normally when uploading files, try to avoid sending a json as many developers tends to to wrong. Binary data in json is equal to bad practice (and larger size) eg:
$.post(url, {
name: '',
data: base64
})
Use the FormData#append as much as possible or if you feel like it:
fd.append('json', json)
I'm working on a project in browser that receives a multipart response from a server.
The first part of the response is a bit of JSON metadata
The second part of the response is a binary MP3 file.
I need a way to extract that MP3 file from the multipart response and play it in an HTML 5 audio element.
Has anyone encountered this or something similar?
I ended up solving my problem. I'm providing the issue and my solution to anyone needing to do this in the future.
Background info:
I'm using AngularJS when I make this ajax request, but the idea is the same for both that, jquery ajax, and regular xhr.
Code:
//creating a form object and assigning everything
//to it is so that XHR can automatically
//generate proper multipart formatting
var form = new FormData();
var data = {};
data['messageHeader'] = {};
var jsonData = JSON.stringify(data);
var jsonBlob = new Blob([jsonData],{type: "application/json"});
//assign json metadata blob and audio blob to the form
form.append("request", jsonData);
form.append("audio",response); //Response is the audio blob
//make the post request
//Notes:
//content-type set to undefined so angular can auto assign type
//transformRequest: angular.identity allows for angular to create multipart
//response: arraybuffer so untouched binary data can be received
$http({method:"POST",
url: endpoint + path,
headers: {
'Authorization': 'Bearer ' + $cookies.get('token'),
'Content-Type': undefined
},
transformRequest: angular.identity,
data: form,
responseType: "arraybuffer"
})
.success(function(data){
//data: ArrayBuffer of multipart response
//toss ArrayBuffer into Uint8Array
//lets you iterate over the bytes
var audioArray = new Uint8Array(data);
//toss a UTF-8 version of the response into
//a variable. Used to extract metadata
var holder = "";
for (var i = 0; i < audioArray.length; i++){
holder += String.fromCharCode(audioArray[i]);
}
//get the boundary from the string. Eg contents of first line
var boundary = holder.substr(0, holder.indexOf("\n"));
//break response into array at each boundary string
var temp = holder.split(boundary);
var parts = [];
//loop through array to remove empty parts
for (var i = 0; i < temp.length; i++){
if (temp[i] != ""){
parts.push(temp[i]);
}
}
//PARSE FIRST PART
//get index of first squiggly, indicator of start of JSON
var jsonStart = parts[0].indexOf('{');
//string to JSON on { index to end of part substring
var JSONResponse = JSON.parse(parts[0].substring(jsonStart));
//PARSE SECOND PART
var audioStart = holder.indexOf('mpeg') + 8;
//get an ArrayBuffer from UInt8Buffer from the audio
//start point to the end of the array
var audio = audioArray.buffer.slice(audioStart);
//hand off audio to AudioContext for automatic decoding
audio_context.decodeAudioData(audio, function(buffer) {
var audioBuffer = buffer;
//create a sound source
var source = audio_context.createBufferSource();
//attach audioBuffer to sound source
source.buffer = audioBuffer;
//wire source to speakers
source.connect(audio_context.destination);
//on audio completion, re-enable mic button
source.onended = function() {
console.log("ended");
$scope.$apply(function(){
$scope.playing = false;
});
}
//start playing audio
source.start(0);
}, function (){
//callback for when there is an error
console.log("error decoding audio");
});
})
Overview:
You need to accept the response as pure binary data (ArrayBuffer). Most libraries will give it to you as a string, which is cool for normal requests but bad for binary data.
You then step through the data to find the multipart boundaries.
You then split at the boundaries.
Get the index of the boundary you know is binary data
and then retrieve the original binary from the ArrayBuffer.
In my case I send that binary into the speakers, however if its an image you can build a blob, get a url from FileReader and then set that as a source of an image.