Develop Reference

Retrieving file data in chunks using Web API for display in browser (WIP)

I have this working but I want to share this out to see if I missed anything obvious and to solve a mystery as to why my file chunk size has to be a multiple of 2049. The main requirements are:
Files uploaded from website must be stored in SQL server, not as files
Website must be able to download and display file data as a file (opened in a separate window.
Website is angularjs/javascript SPA, no server side code, no MVC
API is Web API 2 (again not MVC)
I'm just going to focus on the download part here. Basically what I'm doing is:
Read a chunk of data from SQL server varbinary field
Web API 2 api returns file name, mime type and byte data as a base64 string. NOTE - tried returning byte array but Web API just serializes it into base64 string anyway.
concatenate the chunks, convert the chunks to a blob and display
VB library function that returns a dataset with the chunk (I have to use this library which handles the database connection but doesn't support parameter queries)
Public Function GetWebApplicationAttachment(ByVal intId As Integer, ByVal intChunkNumber As Integer, ByVal intChunkSize As Integer) As DataSet
' the starting number is NOT 0 based
Dim intStart As Integer = 1
If intChunkNumber > 1 Then intStart = ((intChunkNumber - 1) * intChunkSize) + 1
Dim strQuery As String = ""
strQuery += "SELECT FileName, "
strQuery += "SUBSTRING(ByteData," & intStart.ToString & "," & intChunkSize.ToString & ") AS ByteData "
strQuery += "FROM FileAttachments WHERE Id = " + intId.ToString + " "
Try
Return Query(strQuery)
Catch ex As Exception
...
End Try
End Function
Web API business rules bit that creates the file object from the dataset
...
result.FileName = ds.Tables[0].Rows[0]["FileName"].ToString();
// NOTE: Web API converts a byte array to base 64 string so the result is the same either way
// the result of this is that the returned data will be about 30% bigger than the chunk size requested
result.StringData = Convert.ToBase64String((byte[])ds.Tables[0].Rows[0]["ByteData"]);
//result.ByteData = (byte[])ds.Tables[0].Rows[0]["ByteData"];
... some code to get the mime type
result.MIMEType = ...
Web API controller (simplified - all security and error handling removed)
public IHttpActionResult GetFileAttachment([FromUri] int id, int chunkSize, int chunkNumber) {
brs = new Files(...);
fileResult file = brs.GetFileAttachment(appID, chunkNumber, chunkSize);
return Ok(file);
}
angularjs Service that gets the chunks recurively and puts them together
function getFileAttachment2(id, chunkSize, chunkNumber, def, fileData, mimeType) {
var deferred = def || $q.defer();
$http.get(webServicesPath + "api/files/get-file-attachment?id=" + id + "&chunkSize=" + chunkSize + "&chunkNumber=" + chunkNumber).then(
function (response) {
// when completed string data will be empty
if (response.data.StringData === "") {
response.data.MIMEType = mimeType;
response.data.StringData = fileData;
deferred.resolve(response.data);
} else {
if (chunkNumber === 1) {
// only the first chunk computes the mime type
mimeType = response.data.MIMEType;
}
fileData += response.data.StringData;
chunkNumber += 1;
getFileAttachment2(appID, detailID, orgID, GUID, type, chunkSize, chunkNumber, deferred, fileData, mimeType);
}
},
function (response) {
... error stuff
}
);
return deferred.promise;
}
angular controller method that makes the calls.
function viewFile(id) {
sharedInfo.getWebPortalSetting("FileChunkSize").then(function (result) {
// chunk size must be a multiple of 2049 ???
var chunkSize = 0;
if (result !== null) chunkSize = parseInt(result);
fileHelper.getFileAttachment2(id, chunkSize, 1, null, "", "").then(function (result) {
if (result.error === null) {
if (!fileHelper.viewAsFile(result.StringData, result.FileName, result.MIMEType)) {
... error
}
result = {};
} else {
... error;
}
});
});
}
And finally the bit of javascript that displays the file as a download
function viewAsFile(fileData, fileName, fileType) {
try {
fileData = window.atob(fileData);
var ab = new ArrayBuffer(fileData.length);
var ia = new Uint8Array(ab); // ia provides window into array buffer
for (var i = 0; i < fileData.length; i++) {
ia[i] = fileData.charCodeAt(i);
}
var file = new Blob([ab], { type: fileType });
fileData = "";
if (window.navigator.msSaveOrOpenBlob) // IE10+
window.navigator.msSaveOrOpenBlob(file, fileName);
else { // Others
var a = document.createElement("a"),
url = URL.createObjectURL(file);
a.href = url;
a.download = fileName;
document.body.appendChild(a);
a.click();
setTimeout(function () {
document.body.removeChild(a);
window.URL.revokeObjectURL(url);
}, 0);
}
return true;
} catch (e) {
... error stuff
}
}
I see already that a more RESTful approach would be to use headers to indicate chunk range and to separate the file meta data from the file chunks. Also I could try returning a data stream rather than Base64 encoded string. If anyone has tips on that let me know.

Well that was entirely the wrong way to go about that. In case it helps here's what I ended up doing.
Dynamically create the href address of an anchor tag to return a file (security token and parameters in query string)
get byte array from database
web api call return response message (see code below)
This is much faster and more reliable, but provides less in the way of progress monitoring.
business rule method uses...
...
file.ByteData = (byte[])ds.Tables[0].Rows[0]["ByteData"];
...
web api controller
public HttpResponseMessage ViewFileAttachment([FromUri] int id, string token) {
HttpResponseMessage response = new HttpResponseMessage();
... security stuff
fileInfoClass file = ... code to get file info
response.Content = new ByteArrayContent(file.ByteData);
response.Content.Headers.ContentDisposition =
new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment") {
FileName = file.FileName
};
response.Content.Headers.ContentType = new MediaTypeHeaderValue(file.MIMEType);
return response;
This could even be improved with streaming

Retrieving XML as downloadable attachment from MVC Controller action

I am rewriting an existing webform to use js libraries instead of using the vendor controls and microsoft ajax tooling (basically, updating the web app to use more contemporary methodologies).
The AS-IS page (webform) uses a button click handler on the server to process the submitted data and return a document containing xml, which document can either then be saved or opened (opening opens it up as another tab in the browser). This happens asynchronously.
The TO-BE page uses jquery ajax to submit the form to an MVC controller, where virtually the same exact code is executed as in the server-side postback case. I've verified in the browser that the same data is being returned from the caller, but, after returning it, the user is NOT prompted to save/open - the page just remains as if nothing ever happened.
I will put the code below, but I think I am just missing some key diferrence between the postback and ajax/controller contexts to prompt the browser to recognize the returned data as a separate attachment to be saved. My problem is that I have looked at and tried so many ad-hoc approaches that I'm not certain what I am doing wrong at this point.
AS-IS Server Side Handler
(Abridged, since the SendXml() method is what generates the response)
protected void btnXMLButton_Clicked(object sender, EventArgs e)
{
//generate server side biz objects
//formattedXml is a string of xml iteratively generated from each selected item that was posted back
var documentStream = MemStreamMgmt.StringToMemoryStream(formattedXml);
byte[] _documentXMLFile = documentStream.ToArray();
SendXml(_documentXMLFile);
}
private void SendXml(byte[] xmlDoc)
{
string _xmlDocument = System.Text.Encoding.UTF8.GetString(xmlDoc);
XDocument _xdoc = XDocument.Parse(_xmlDocument);
var _dcpXMLSchema = new XmlSchemaSet();
_dcpXMLSchema.Add("", Server.MapPath(#"~/Orders/DCP.xsd"));
bool _result = true;
try
{
_xdoc.Validate(_dcpXMLSchema, null);
}
catch (XmlSchemaValidationException)
{
//validation failed raise error
_result = false;
}
// return error message
if (!_result)
{
//stuff to display message
return;
}
// all is well .. download xml file
Response.ClearContent();
Response.Clear();
Response.ContentType = "text/plain";
Response.AddHeader("Content-disposition", "attachment; filename=" + "XMLOrdersExported_" + string.Format("{0:yyyy-MM-dd_hh-mm-ss-tt}.xml", DateTime.Now));
Response.BinaryWrite(xmlDoc);
Response.Flush();
Context.ApplicationInstance.CompleteRequest();
Response.End();
}
TO-BE (Using jquery to submit to a controller action)
Client code: button click handler:
queueModel.getXmlForSelectedOrders = function () {
//create form to submit
$('body').append('<form id="formXmlTest"></form>');
//submit handler
$('#formXmlTest').submit(function(event) {
var orderNbrs = queueModel.selectedItems().map(function (e) { return e.OrderId() });
console.log(orderNbrs);
var ordersForXml = orderNbrs;
var urlx = "http://localhost:1234/svc/OrderServices/GetXml";
$.ajax({
url: urlx,
type: 'POST',
data: { orders: ordersForXml },
dataType: "xml",
accepts: {
xml: 'application/xhtml+xml',
text: 'text/plain'
}
}).done(function (data) {
/*Updated per comments */
console.log(data);
var link = document.createElement("a");
link.target = "blank";
link.download = "someFile";//data.name
console.log(link.download);
link.href = "http://localhost:23968/svc/OrderServices/GetFile/demo.xml";//data.uri;
link.click();
});
event.preventDefault();
});
$('#formXmlTest').submit();
};
//Updated per comments
/*
[System.Web.Mvc.HttpPost]
public void GetXml([FromBody] string[] orders)
{
//same code to generate xml string
var documentStream = MemStreamMgmt.StringToMemoryStream(formattedXml);
byte[] _documentXMLFile = documentStream.ToArray();
//SendXml(_documentXMLFile);
string _xmlDocument = System.Text.Encoding.UTF8.GetString(_documentXMLFile);
XDocument _xdoc = XDocument.Parse(_xmlDocument);
var _dcpXMLSchema = new XmlSchemaSet();
_dcpXMLSchema.Add("", Server.MapPath(#"~/Orders/DCP.xsd"));
bool _result = true;
try
{
_xdoc.Validate(_dcpXMLSchema, null);
}
catch (XmlSchemaValidationException)
{
//validation failed raise error
_result = false;
}
Response.ClearContent();
Response.Clear();
Response.ContentType = "text/plain";
Response.AddHeader("Content-disposition", "attachment; filename=" + "XMLOrdersExported_" + string.Format("{0:yyyy-MM-dd_hh-mm-ss-tt}.xml", DateTime.Now));
Response.BinaryWrite(_documentXMLFile);
Response.Flush();
//Context.ApplicationInstance.CompleteRequest();
Response.End();
}
}*/
[System.Web.Mvc.HttpPost]
public FileResult GetXmlAsFile([FromBody] string[] orders)
{
var schema = Server.MapPath(#"~/Orders/DCP.xsd");
var formattedXml = OrderXmlFormatter.GenerateXmlForSelectedOrders(orders, schema);
var _result = validateXml(formattedXml.DocumentXmlFile, schema);
// return error message
if (!_result)
{
const string message = "The XML File(s) are not valid! Please check with your administrator!.";
return null;
}
var cd = new System.Net.Mime.ContentDisposition
{
FileName = "blargoWargo.xml",
Inline = false
};
System.IO.File.WriteAllBytes(Server.MapPath("~/temp/demo.xml"),formattedXml.DocumentXmlFile);
return File(formattedXml.DocumentXmlFile,MediaTypeNames.Text.Plain,"blarg.xml");
}
[System.Web.Mvc.HttpGet]
public FileResult GetFile(string fileName)
{
var cd = new System.Net.Mime.ContentDisposition
{
// for example foo.bak
FileName = fileName,
Inline = false
};
Response.AppendHeader("Content-Disposition", cd.ToString());
var fName = !string.IsNullOrEmpty(fileName)?fileName:"demo.xml";
var fArray = System.IO.File.ReadAllBytes(Server.MapPath("~/temp/" + fName));
System.IO.File.Delete(Server.MapPath("~/temp/" + fName));
return File(fArray, MediaTypeNames.Application.Octet);
}
UPDATE:
I just put the AS-IS/TO-BE side by side, and in the dev tools verified the ONLY difference (at least as far as dev tools shows) is that the ACCEPT: header for TO-BE is:
application/xhtml+xml, /; q=0.01
Whereas the header for AS-IS is
text/html, application/xhtml+xml, image/jxr, /
Update II
I've found a workaround using a 2-step process with a hyperlink. It is a mutt of a solution, but as I suspected, apparently when making an ajax call (at least a jQuery ajax call, as opposed to a straight XmlHttpRequest) it is impossible to trigger the open/save dialog. So, in the POST step, I create and save the desired file, then in the GET step (using a dynamically-created link) I send the file to the client and delete it from the server. I'm leaving this unanswered for now in the hopes someone who understands the difference deeply can explain why you can't retrieve the file in the course of a normal ajax call.

Creating zip file in memory out of byte[]. Zip file is never suggest to download

I have a problem with my created zip file. I am using Java 8. I tried to create a zip file out of a byte array, which contains two or more Excel files. . So, I thought everything is alright. I do an ajax call for create and download my file but i don't have the popup for download my zip and i don't have error.
This is my javascript:
function getFile() {
$.ajax({
type: "POST",
url: "/support-web/downloadCSV",
dataType: "json",
contentType: 'application/json;charset=UTF-8',
data: jsonfile,
success: function (data) {
console.log("in sucess");
window.location.href="/support-web/downloadCSV/"+data
},
error:function (xhr, ajaxOptions, thrownError){
console.log("in error")
}
});
}
This is my Controller:
#Controller
#RequestMapping(value = "/downloadCSV")
public class DownloadCSVController {
private static final int BUFFER_SIZE = 4096;
#RequestMapping(method = RequestMethod.POST)
#ResponseBody
public void downloadCSV(HttpServletRequest request, HttpServletResponse response, #RequestBody String json)
throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (ZipOutputStream zos = new ZipOutputStream(baos)) {
int i = 0;
for (String url : parts) {
i++;
URL uri = new URL(url);
HttpURLConnection httpConn = (HttpURLConnection) uri.openConnection();
int responseCode = httpConn.getResponseCode();
if (responseCode == HttpURLConnection.HTTP_OK) {
String fileName = "";
String disposition = httpConn.getHeaderField("Content-Disposition");
String contentType = httpConn.getContentType();
int contentLength = httpConn.getContentLength();
if (disposition != null) {
// extracts file name from header field
int index = disposition.indexOf("filename=");
if (index > 0) {
fileName = disposition.substring(index + 9, disposition.length());
}
} else {
// extracts file name from URL
fileName = url.substring(url.lastIndexOf("/") + 1, url.length());
}
System.out.println("Content-Type = " + contentType);
System.out.println("Content-Disposition = " + disposition);
System.out.println("Content-Length = " + contentLength);
System.out.println("fileName = " + fileName);
// opens input stream from the HTTP connection
InputStream inputStream = httpConn.getInputStream();
ZipEntry entry = new ZipEntry(fileName + i + ".csv");
int length = 1;
zos.putNextEntry(entry);
byte[] b = new byte[BUFFER_SIZE];
while ((length = inputStream.read(b)) > 0) {
zos.write(b, 0, length);
}
zos.closeEntry();
inputStream.close();
System.out.println("File downloaded");
}
}
} catch (IOException ioe) {
ioe.printStackTrace();
}
// this is the zip file as byte[]
int size = baos.toByteArray().length;
byte[] reportContent = baos.toByteArray();
// Write file to response.
OutputStream output = response.getOutputStream();
output.write(reportContent);
output.close();
response.setContentType("application/force-download");
response.setContentLength((int)size);
response.setHeader("Content-Transfer-Encoding", "binary");
response.setHeader("Content-Disposition","attachment; filename=\"test.zip\"");//fileName)
System.out.println("FIN TELECHARGEMENT");
}
}
Problem:
The Browser not should open a download box
The response isn't handled in the error or in the success (ajax)
So what do I wrong or what is the proper way to do this?
In my navigator you can see the response with my file but download box not should open

You need to do two things:
Set headers before writing anything to response stream.
Remove output.close(); you should not do that. Stream is opened and closed by container.
Second point actually not affecting your problem, its just an advice. You can read more about it here Should one call .close() on HttpServletResponse.getOutputStream()/.getWriter()?.

Upload Document to Web API

First Step: On my MVC project I have a Div that allow to upload a file.
<div id="modalUpload" style="display:none">
Select a file:
<input id="upload" type="file" name="upload" /><br />
Description:
<input id="documentDescription" type="text" /><br />
<input type="button" value="Submit" id="submitUpload" data-bind="click:function(){ $root.upload() }">
</div>
Second Step: In JavaScript I upload the Document to the web API with a POST request: (for example, with the XMLHttpRequest method)
self.upload = function () {
var file = document.getElementById("upload").files[0];
var xhr = new XMLHttpRequest();
xhr.open('post', "/API/document/addDocument/", true);
xhr.send(file);
}
Third Step: This is my addDocument function on the Web API that expects a Document object, like this:
[HttpPost]
[Route("{controller}/addDocument")]
public string Post([FromBody]Document doc)
{
.......
}
My question is, how can I post the Document to the API as a Document object? Is it possible to set the type of the POST data?
On my JavaScript code, I have a Document object:
var Document = function (data) {
this.id = ko.observable(data.id);
this.name= ko.observable(data.name);
this.size= ko.observable(data.zise);
}
But I'm not sure how to connect between them.
When I tried this:
var file = new Document(document.getElementById("upload").files[0]);
I got the following error:
415 (Unsupported Media Type)
Any help would be very much appreciated.

One possible issue is that you'll need to make sure your XHR request declares the correct MIME type in the accept header.
Using XHR directly can be a bit of a pain. You may want to consider using a client-side library (such as jquery) to make things easier and handle any cross-browser inconsistencies.

You could try getting it as an HttpPostedFileBase
[HttpPost]
[Route("{controller}/addDocument")]
public string Post(HttpPostedFileBase doc)
{
//Parse it to a doc and do what you gotta do
}

Finally, I didn't send the document as an object, and this is what I ended up doing,
self.upload = function () {
if (document.getElementById("upload").value != "") {
var file = document.getElementById("upload").files[0];
var filePath = "....";
if (window.FormData !== undefined) {
var data = new FormData();
data.append("file", file);
var encodedString = Base64.encode(filePath);
$.ajax({
type: "POST",
url: "/API/document/upload/" + file.name + "/" + encodedString ,
contentType: false,
processData: false,
data: data,
success: function (result) {
console.log(result);
},
error: function (xhr, status, p3, p4) {
var err = "Error " + " " + status + " " + p3 + " " + p4;
if (xhr.responseText && xhr.responseText[0] == "{")
err = JSON.parse(xhr.responseText).Message;
console.log(err);
}
});
}
}
}
And this is my API function:
[HttpPost]
[Route("{controller}/upload/{fileName}/{filePath}")]
public async Task<HttpResponseMessage> UploadFile(string fileName, string filePath)
{
if (!Request.Content.IsMimeMultipartContent())
{
return Request.CreateErrorResponse(HttpStatusCode.UnsupportedMediaType, "The request doesn't contain valid content!");
}
byte[] data = Convert.FromBase64String(filePath);
filePath = Encoding.UTF8.GetString(data);
try
{
var provider = new MultipartMemoryStreamProvider();
await Request.Content.ReadAsMultipartAsync(provider);
foreach (var file in provider.Contents)
{
var dataStream = await file.ReadAsStreamAsync();
// use the data stream to persist the data to the server (file system etc)
using (var fileStream = File.Create(filePath + fileName))
{
dataStream.Seek(0, SeekOrigin.Begin);
dataStream.CopyTo(fileStream);
}
var response = Request.CreateResponse(HttpStatusCode.OK);
response.Content = new StringContent("Successful upload", Encoding.UTF8, "text/plain");
response.Content.Headers.ContentType = new MediaTypeWithQualityHeaderValue(#"text/html");
return response;
}
return Request.CreateResponse(HttpStatusCode.ExpectationFailed);
}
catch (Exception e)
{
return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, e.Message);
}
}
That works perfectly, I got the idea from here.

Downloading a file in MVC app using AngularJS and $http.post

Any help is most welcomed and really appreciated.
I have an MVC action which retries a file content from a web service. This action is invoked from a Angular service (located in services.js) using $http.post(action, model), and the action is returning a FileContentResult object, which contains the byte array and the content type.
public ActionResult DownloadResults(DownloadResultsModel downloadResultsModel)
{
downloadResult = ... // Retrieving the file from a web service
Response.ClearHeaders();
Response.AddHeader("Content-Disposition", string.Format("attachment; filename={0}", downloadResult.FileName));
Response.BufferOutput = false;
return new FileContentResult(downloadResult.Contents, downloadResult.ContentType);
}
The issue I'm having is about the browser not performing the default behavior of handing a file (for example, prompting to open it, saving it or cancel). The action is completed successfully with having the content of the file and the file name (injected to the FileContentResult object), but there s no response from the browser.
When I'm replacing the post with $window.location.href, and construct the URI myself, I'm hitting the action and after it completes the browser is handling the file as expected.
Does anyone can think of any idea how to complete the 'post' as expected?
Thanks,
Elad

I am using below code to download the file, given that the file does exist on the server and client is sending server the full path of the file...
as per you requirement change the code to specify path on server itself.
[HttpGet]
public HttpResponseMessage DownloadFile(string filename)
{
filename = filename.Replace("\\\\", "\\").Replace("'", "").Replace("\"", "");
if (!char.IsLetter(filename[0]))
{
filename = filename.Substring(2);
}
var fileinfo = new FileInfo(filename);
if (!fileinfo.Exists)
{
throw new FileNotFoundException(fileinfo.Name);
}
try
{
var excelData = File.ReadAllBytes(filename);
var result = new HttpResponseMessage(HttpStatusCode.OK);
var stream = new MemoryStream(excelData);
result.Content = new StreamContent(stream);
result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
result.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = fileinfo.Name
};
return result;
}
catch (Exception ex)
{
return Request.CreateResponse(HttpStatusCode.ExpectationFailed, ex);
}
}
and then on client side in angular:
var downloadFile = function (filename) {
var ifr = document.createElement('iframe');
ifr.style.display = 'none';
document.body.appendChild(ifr);
ifr.src = document.location.pathname + "api/GridApi/DownloadFile?filename='" + escape(filename) + "'";
ifr.onload = function () {
document.body.removeChild(ifr);
ifr = null;
};
};