I worked on a ASP.NET MVC 4 project in which users can apply filter on UI & extract an excel report. The data was stored in MS SQL server. To improve performances, i decided to adopt the async in my application to:
Reduce the time of extraction
Allow users to have parallel extraction
To do this, my approach are:
When user click extraction button on UI -> an ajax call will be made from client to async function on server -> This async function in turn will create an Command object & do ExecuteReaderAsync(). Using this DbDatareader to generated an Excel file using NPOI and save the file content to TempData. The handler to retrieve file will be return to client for later download using window.location. I adopted these techniques from this post Download Excel file via AJAX MVC
After the first extraction, if users want to extract another datasets in parallel, they can click extraction button again and application will repeat step 1.
The results are 2 or more data extractions can happened on the same time.
My problem is, take example, 4 extractions currently running in parallels, if any of these extractions finished & 1 file is downloaded (using window.location). The next time user click on extraction button (which repeat step 1), it doesn't async anymore & later extractions will wait for previous extraction finish before execute.
On debugging, if i restart the ISS server, the problem gone for a while until 1 file is downloaded, so I doubted that window.location do something that blocked the threads on server when any of file is downloaded.
UPDATE 1
Class:
public class QUERYREADER
{
public DbConnection CONNECTION { get; set; }
public DbDataReader READER { get; set; }
}
Model:
public async Task<QUERYREADER> GET_DATA(CancellationToken ct)
{
//Create the query reader
QUERYREADER qr = new QUERYREADER();
//Set up the database instances
DbProviderFactory dbFactory = DbProviderFactories.GetFactory(db.Database.Connection);
//Defined the query
var query = "SELECT * FROM Table";
//Set up the sql command object
using (var cmd = dbFactory.CreateCommand())
{
//Try to open the database connection
try
{
//Check if SQL connection is set up
if (cmd.Connection == null)
{
cmd.CommandType = CommandType.Text;
cmd.Connection = db.Database.Connection;
}
//Open connection to SQL if current state is closed
if (cmd.Connection.State == ConnectionState.Closed)
{
//Change the connection string to set the packet size to max. value = 32768 to improve efficiency for I/O transmit to SQL server
cmd.Connection.ConnectionString = cmd.Connection.ConnectionString + ";Packet Size=20000";
//Open connection
await cmd.Connection.OpenAsync(ct);
}
//Save the connection
qr.CONNECTION = cmd.Connection;
} catch (Exception ex) {
//If errors throw, close the connection
cmd.Connection.Close();
};
//Retrieve the database reader of provided sql query
cmd.CommandText = query;
DbDataReader dr = await cmd.ExecuteReaderAsync(ct);
qr.READER.Add(dr);
}
//Return the queryreader
return qr;
}
Controller:
public async Task<JsonResult> SQL_TO_EXCEL()
{
//Set up the subscription to client for "cancellation request, browser closing"
CancellationToken disToken = Response.ClientDisconnectedToken;
//Get the datareader
try
{
qr = await GET_DATA(disToken);
}
catch(Exception ex) { }
//Open the connection to SQL server
using (qr.CONNECTION)
{
using (var dr = qr.READER)
{
while (await dr.ReadAsync(disToken))
{
for (int k = 0; k < dr.FieldCount; k++)
{
//.... using NPOI to write Excel file to MemoryStream
}
}
dr.Close();
}
}
//Generate XL file if controller action is still running (no "cancellation request, browser closing")
if (!disToken.IsCancellationRequested)
{
string file_id = Guid.NewGuid().ToString();
//... Write the NPOI excel file to TempData and then create a handler for later download at client
//This line caused trouble
TempData["file_id"] = XLMemoryStream.ToArray();
HANDLER["file_id"] = file_id;
HANDLER["file_name"] = FILE["FILE_NAME"].ToString().NonUnicode() + FILE["FILE_TYPE"].ToString() ;
}
//Return JSON to caller
var JSONRESULT = Json(JsonConvert.SerializeObject(HANDLER), JsonRequestBehavior.AllowGet);
JSONRESULT.MaxJsonLength = int.MaxValue;
return JSONRESULT;
}
public async Task<ActionResult> DOWNLOAD_EXCEL(string file_id, string file_name)
{
if (TempData[file_id] != null)
{
byte[] data = await Task.Run(() => TempData[file_id] as byte[]);
return File(data, "application/vnd.ms-excel", file_name);
}
else
{
return new EmptyResult();
}
}
Javascript
$.ajax({
type: 'POST',
async: true,
cache: false,
url: 'SQL_TO_EXCEL',
success: function (data)
{
var response = JSON.parse(data);
window.location =
(
"DOWNLOAD_EXCEL" +
'?file_id=' + response.file_id +
'&file_name=' + response.file_name
);
},
error: function (XMLHttpRequest, textStatus, errorThrown) {
console.log(errorThrown);
}
});
UPDATE 2:
After a lot of tests, i figured out window.location has nothing to do with threads on server, the line TempData[file_id] = XLMemoryStream.ToArray() caused the issues. It look likes the problem is similar as described in this post Two parallel ajax requests to Action methods are queued, why?
Related
I am currently getting drag and drop / uploaded images as a data url and displaying them with that url.
What I am now trying to do is send those uploaded images to my backend web api using ASP.Net core to store then in a sqlite database this is a requirement for my application.
Currently I am converting the data url to an arraybuffer using the following code.
async srcToFile(context, asset) {
const files = asset[0].files.fileList;
let results = [];
for (let i = 0; i < files.length; i++) {
const file = files[i];
const data = file.data;
const name = file.name;
const mimeType = file.type;
await fetch(data)
.then(function(res) {
const r = res.arrayBuffer();
console.warn('resource ', r);
return r;
})
.then(function(buf) {
console.warn('buffer: ', [buf]);
const fileData = {data:[buf], name:name, type:mimeType};
results.push(fileData);
console.warn('results of file: ', fileData);
});
}
console.warn(results);
return results;
}
then I put it in an data object to send to my server via axios this is what that data object looks like
const data = {
Name: asset[0].name,
Detail: asset[0].detail,
Files: asset[0].files.fileList
};
When I console out the Files it shows there is Arraybuffer data in it. But when I send it to my server it looks like that data is stripped out of the header call. Cause when I look at the header I no longer have that data in there and I cannot figure out why that is happening.
this is my axios call.
axios.post('https://localhost:5001/api/Assets', data)
.then(res => console.log(res))
.catch(error => console.log(error));
and my back end web api post controller
public async Task<ActionResult> PostAsset([FromBody] AssetSaveRequest request,[FromForm] List<IFormFile> files)
{
foreach (var file in files)
{
if (file.Length > 0)
{
using (var ms = new MemoryStream())
{
file.CopyTo(ms);
var fileBytes = ms.ToArray();
string s = Convert.ToBase64String(fileBytes);
// act on the Base64 data
}
}
}
var assetCreationDto = new AssetCreationDto(request);
//var assetCreationDto = "";
try
{
var asset = _mapper.Map<Asset>(assetCreationDto);
_context.Assets.Add(asset);
//await _context.SaveChangesAsync();
var assetDto = _mapper.Map<AssetDto>(asset);
return CreatedAtAction("GetAsset", new {assetDto.Id}, assetDto);
}
catch (DbUpdateException dbe)
{
var errorCode = ((Microsoft.Data.Sqlite.SqliteException) dbe.InnerException).SqliteErrorCode;
switch (errorCode)
{
case 19:
Console.WriteLine(((Microsoft.Data.Sqlite.SqliteException)dbe.InnerException).Message);
break;
default:
Console.WriteLine("Something went wrong");
break;
}
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
return null;
}
That of which I don't know is working because I never get the file data, I do how ever get the Name and the details which come in fine.
I am looking for advice on what I should do here to get this to work. I have tried converting the arraybuffer to base64 string but that does not come out right any help and suggestions would be great to get me back on track with this project .
UPDATE:
I have modified my srcToFile code to give me a file, now I am using axios to send the file and data to the backend working with one file at this time and all im getting in the header now is [object object]. I've tried JSON.stringify on my data like so
const data = JSON.stringify({
Name: asset[0].name,
Detail: asset[0].detail,
Files: asset[0].files.fileList
});
It stringify's the name and detail but wipes out the file and I get nothing on the backend.
I have tested with postman and made several successful posts. but I can't seem to get the correct data from my Vue front end.
that is where I am at now. any suggestions always helps
I created a service to download a PDF file.
On my server-side(Java) the PDF is generated successfully. But I am unable to download that on the UI side (Using Jquery Ajax call).
Could anyone please help me with this?
$(document).on('click', '.orderView', function(event){
orderId = $(this).attr('data');
$.ajax({
type : 'GET',
contentType : 'application/json',
url : '../service/purchase/generateInventoryPurchasePdf/'+orderId,
success : function(response) {
console.log("Success");
},
error : function(response) {
console.log("Error :" + response);
}
});
});
Java Code:
#RequestMapping(value = "/generateInventoryPurchasePdf/{purchaseId}", method = RequestMethod.GET)
public ResponseEntity<ByteArrayResource> generateInventoryPurchasePdf(HttpServletResponse response,#PathVariable("purchaseId") Long purchaseId) throws Exception {
PurchaseOrder purchaseOrder = null;
purchaseOrder = purchaseService.findByPurchaseOrderId(purchaseId);
// generate the PDF
Map<Object,Object> pdfMap = new HashMap<>();
pdfMap.put("purchaseOrder", purchaseOrder);
pdfMap.put("purchaseOrderDetail", purchaseOrder.getPurchaseOrderDetail());
pdfMap.put("vendorName", purchaseOrder.getInvVendor().getName());
pdfMap.put("vendorAddrs", purchaseOrder.getInvVendor().getVenAddress().get(0));
File file = util.generatePdf("email/purchasepdf", pdfMap);
MediaType mediaType = MediaTypeUtils.getMediaTypeForFileName(this.servletContext, file.getName());
System.out.println("fileName: " + file.getName());
System.out.println("mediaType: " + mediaType);
//Path path = Paths.get(file.getAbsolutePath() + "/" + file.getName());
Path path = Paths.get(file.getAbsolutePath());
byte[] data = Files.readAllBytes(path);
ByteArrayResource resource = new ByteArrayResource(data);
return ResponseEntity.ok()
// Content-Disposition
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment;filename=" + path.getFileName().toString())
// Content-Type
.contentType(mediaType) //
// Content-Lengh
.contentLength(data.length) //
.body(resource);
}
mediaUtil class:
public class MediaTypeUtils {
public static MediaType getMediaTypeForFileName(ServletContext servletContext, String fileName) {
// application/pdf
// application/xml
// image/gif, ...
String mineType = servletContext.getMimeType(fileName);
try {
MediaType mediaType = MediaType.parseMediaType(mineType);
return mediaType;
} catch (Exception e) {
return MediaType.APPLICATION_OCTET_STREAM;
}
}
}
PDF Generation code:
public File generatePdf(String templateName, Map<Object, Object> map) throws Exception {
Assert.notNull(templateName, "The templateName can not be null");
Context ctx = new Context();
if (map != null) {
Iterator<Entry<Object, Object>> itMap = map.entrySet().iterator();
while (itMap.hasNext()) {
Map.Entry<Object, Object> pair = itMap.next();
ctx.setVariable(pair.getKey().toString(), pair.getValue());
}
}
String processedHtml = templateEngine.process(templateName, ctx);
FileOutputStream os = null;
String fileName = "POLIST";
try {
final File outputFile = File.createTempFile(fileName, ".pdf",new File(servletContext.getRealPath("/")));
outputFile.mkdir();
os = new FileOutputStream(outputFile);
ITextRenderer renderer = new ITextRenderer();
renderer.setDocumentFromString(processedHtml);
renderer.layout();
renderer.createPDF(os, false);
renderer.finishPDF();
System.out.println("PDF created successfully");
return outputFile;
} finally {
if (os != null) {
try {
os.close();
} catch (IOException e) {
}
}
}
}
I'm not getting any error, PDF generate successfully in the server side. But In UI side not working.
Downloading files via AJAX isn't really a logical thing to do. When you make an AJAX call, the data returned from the server is returned into your page's JavaScript code (in the response callback value), rather than being returned to the browser itself to decide what to do. Therefore the browser has no way to initiate a download, because the browser is not directly in control of the response - your JavaScript code is in control instead.
As you've indicated in your comment below the question, there are workarounds you can use, but really the best approach is simply to use a regular non-AJAX request to download
For instance you could replace your jQuery code with something like
$(document).on('click', '.orderView', function(event){
orderId = $(this).attr('data');
window.open('../service/purchase/generateInventoryPurchasePdf/'+orderId);
});
This will download the document from a new tab without navigating away from the current page.
I'm working on a legacy system and I'm trying to call an HTTP handler which I have added some logic which retrieves audio blob from an Azure service.
The thing is, I can't seem to get the content back to the client so I can play it.
The response that I get from a jQuery call is:
"System.Threading.Tasks.Task`1[System.String]"
This is the processRequest code:
public void ProcessRequest(HttpContext context)
{
var text = "walk";
Authentication auth = new Authentication("subscriptionID");
context.Response.Write(auth.getVoice(text));
}
Here's the getVoice function:
public async Task<string> getVoice(string text)
{
using (var client = new HttpClient())
{
try
{
client.DefaultRequestHeaders
.Add("User-Agent", "uagent");
client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);
client.DefaultRequestHeaders.Add("Authorization", "Bearer " + this.token);
client.DefaultRequestHeaders.Add("host", "westeurope.tts.speech.microsoft.com");
client.DefaultRequestHeaders.Add("X-MICROSOFT-OutputFormat", "audio-16khz-32kbitrate-mono-mp3");
UriBuilder uriBuilder = new UriBuilder(VoiceUri);
// send xml post
var voiceTest = "<speak version='1.0' xml:lang='en-US'><voice xml:lang='en-US' xml:gender='Female'\n\rname='Microsoft Server Speech Text to Speech Voice (en-US, ZiraRUS)'>\n\rWalk\n\r</voice></speak>";
var data = new StringContent(voiceTest, Encoding.UTF8, "application/xml");
data.Headers.ContentType = new MediaTypeHeaderValue("application/ssml+xml");
var result = await client.PostAsync(uriBuilder.Uri.AbsoluteUri, data);
return await result.Content.ReadAsStringAsync();
}
catch (Exception e)
{
return null;
}
}
The server API call is successful but I can't seem to receive it on the client side in order to play it to the user afterward.
You have to unwrap the result of auth.getVoice. You can do it by using await key word. Also you need you handler to implement HttpTaskAsyncHandler class in order to make it work
public class TestHandler : HttpTaskAsyncHandler
{
public async override Task ProcessRequestAsync(HttpContext context)
{
var text = "walk";
Authentication auth = new Authentication("subscriptionID");
context.Response.Write(await auth.getVoice(text)); //added await here
}
//..
}
I am rewriting an existing webform to use js libraries instead of using the vendor controls and microsoft ajax tooling (basically, updating the web app to use more contemporary methodologies).
The AS-IS page (webform) uses a button click handler on the server to process the submitted data and return a document containing xml, which document can either then be saved or opened (opening opens it up as another tab in the browser). This happens asynchronously.
The TO-BE page uses jquery ajax to submit the form to an MVC controller, where virtually the same exact code is executed as in the server-side postback case. I've verified in the browser that the same data is being returned from the caller, but, after returning it, the user is NOT prompted to save/open - the page just remains as if nothing ever happened.
I will put the code below, but I think I am just missing some key diferrence between the postback and ajax/controller contexts to prompt the browser to recognize the returned data as a separate attachment to be saved. My problem is that I have looked at and tried so many ad-hoc approaches that I'm not certain what I am doing wrong at this point.
AS-IS Server Side Handler
(Abridged, since the SendXml() method is what generates the response)
protected void btnXMLButton_Clicked(object sender, EventArgs e)
{
//generate server side biz objects
//formattedXml is a string of xml iteratively generated from each selected item that was posted back
var documentStream = MemStreamMgmt.StringToMemoryStream(formattedXml);
byte[] _documentXMLFile = documentStream.ToArray();
SendXml(_documentXMLFile);
}
private void SendXml(byte[] xmlDoc)
{
string _xmlDocument = System.Text.Encoding.UTF8.GetString(xmlDoc);
XDocument _xdoc = XDocument.Parse(_xmlDocument);
var _dcpXMLSchema = new XmlSchemaSet();
_dcpXMLSchema.Add("", Server.MapPath(#"~/Orders/DCP.xsd"));
bool _result = true;
try
{
_xdoc.Validate(_dcpXMLSchema, null);
}
catch (XmlSchemaValidationException)
{
//validation failed raise error
_result = false;
}
// return error message
if (!_result)
{
//stuff to display message
return;
}
// all is well .. download xml file
Response.ClearContent();
Response.Clear();
Response.ContentType = "text/plain";
Response.AddHeader("Content-disposition", "attachment; filename=" + "XMLOrdersExported_" + string.Format("{0:yyyy-MM-dd_hh-mm-ss-tt}.xml", DateTime.Now));
Response.BinaryWrite(xmlDoc);
Response.Flush();
Context.ApplicationInstance.CompleteRequest();
Response.End();
}
TO-BE (Using jquery to submit to a controller action)
Client code: button click handler:
queueModel.getXmlForSelectedOrders = function () {
//create form to submit
$('body').append('<form id="formXmlTest"></form>');
//submit handler
$('#formXmlTest').submit(function(event) {
var orderNbrs = queueModel.selectedItems().map(function (e) { return e.OrderId() });
console.log(orderNbrs);
var ordersForXml = orderNbrs;
var urlx = "http://localhost:1234/svc/OrderServices/GetXml";
$.ajax({
url: urlx,
type: 'POST',
data: { orders: ordersForXml },
dataType: "xml",
accepts: {
xml: 'application/xhtml+xml',
text: 'text/plain'
}
}).done(function (data) {
/*Updated per comments */
console.log(data);
var link = document.createElement("a");
link.target = "blank";
link.download = "someFile";//data.name
console.log(link.download);
link.href = "http://localhost:23968/svc/OrderServices/GetFile/demo.xml";//data.uri;
link.click();
});
event.preventDefault();
});
$('#formXmlTest').submit();
};
//Updated per comments
/*
[System.Web.Mvc.HttpPost]
public void GetXml([FromBody] string[] orders)
{
//same code to generate xml string
var documentStream = MemStreamMgmt.StringToMemoryStream(formattedXml);
byte[] _documentXMLFile = documentStream.ToArray();
//SendXml(_documentXMLFile);
string _xmlDocument = System.Text.Encoding.UTF8.GetString(_documentXMLFile);
XDocument _xdoc = XDocument.Parse(_xmlDocument);
var _dcpXMLSchema = new XmlSchemaSet();
_dcpXMLSchema.Add("", Server.MapPath(#"~/Orders/DCP.xsd"));
bool _result = true;
try
{
_xdoc.Validate(_dcpXMLSchema, null);
}
catch (XmlSchemaValidationException)
{
//validation failed raise error
_result = false;
}
Response.ClearContent();
Response.Clear();
Response.ContentType = "text/plain";
Response.AddHeader("Content-disposition", "attachment; filename=" + "XMLOrdersExported_" + string.Format("{0:yyyy-MM-dd_hh-mm-ss-tt}.xml", DateTime.Now));
Response.BinaryWrite(_documentXMLFile);
Response.Flush();
//Context.ApplicationInstance.CompleteRequest();
Response.End();
}
}*/
[System.Web.Mvc.HttpPost]
public FileResult GetXmlAsFile([FromBody] string[] orders)
{
var schema = Server.MapPath(#"~/Orders/DCP.xsd");
var formattedXml = OrderXmlFormatter.GenerateXmlForSelectedOrders(orders, schema);
var _result = validateXml(formattedXml.DocumentXmlFile, schema);
// return error message
if (!_result)
{
const string message = "The XML File(s) are not valid! Please check with your administrator!.";
return null;
}
var cd = new System.Net.Mime.ContentDisposition
{
FileName = "blargoWargo.xml",
Inline = false
};
System.IO.File.WriteAllBytes(Server.MapPath("~/temp/demo.xml"),formattedXml.DocumentXmlFile);
return File(formattedXml.DocumentXmlFile,MediaTypeNames.Text.Plain,"blarg.xml");
}
[System.Web.Mvc.HttpGet]
public FileResult GetFile(string fileName)
{
var cd = new System.Net.Mime.ContentDisposition
{
// for example foo.bak
FileName = fileName,
Inline = false
};
Response.AppendHeader("Content-Disposition", cd.ToString());
var fName = !string.IsNullOrEmpty(fileName)?fileName:"demo.xml";
var fArray = System.IO.File.ReadAllBytes(Server.MapPath("~/temp/" + fName));
System.IO.File.Delete(Server.MapPath("~/temp/" + fName));
return File(fArray, MediaTypeNames.Application.Octet);
}
UPDATE:
I just put the AS-IS/TO-BE side by side, and in the dev tools verified the ONLY difference (at least as far as dev tools shows) is that the ACCEPT: header for TO-BE is:
application/xhtml+xml, /; q=0.01
Whereas the header for AS-IS is
text/html, application/xhtml+xml, image/jxr, /
Update II
I've found a workaround using a 2-step process with a hyperlink. It is a mutt of a solution, but as I suspected, apparently when making an ajax call (at least a jQuery ajax call, as opposed to a straight XmlHttpRequest) it is impossible to trigger the open/save dialog. So, in the POST step, I create and save the desired file, then in the GET step (using a dynamically-created link) I send the file to the client and delete it from the server. I'm leaving this unanswered for now in the hopes someone who understands the difference deeply can explain why you can't retrieve the file in the course of a normal ajax call.
In a Java Web Project we are using a Servlet to update an XML tag value using the values coming from a webpage then for further execution the Servlet has to fetch this updated tag value from the XML and proceed, however it is retrieving the old value (before the update) and proceeding.
public void setPeriodID(String bookingsBOPeriodID) throws InterruptedException {
try{
final String FilePath=UtilLib.getEnvVar("ConfigXMLFilePath");
String filepath = FilePath;
String bwperiodid=" and ";
DocumentBuilderFactory docFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder docBuilder = docFactory.newDocumentBuilder();
Document doc = docBuilder.parse(filepath);
// Get the staff element by tag name directly
Node Parameters = doc.getElementsByTagName("Parameters").item(0);
// loop the staff child node
NodeList list = Parameters.getChildNodes();
for (int i = 0; i < list.getLength(); i++) {
Node node = list.item(i);
//BookingsBO
if (bookingsBOPeriodID!=null && bookingsBOPeriodID.length()!=0 && "BookingsBOINPeriodId".equals(node.getNodeName()) && bookingsBOPeriodID.indexOf(bwperiodid)==-1 ){
System.out.println("***** Updating Bookings BO IN Period id ********");
System.out.println("inside updateEnvPeriodID::"+bookingsBOPeriodID);
node.setTextContent(bookingsBOPeriodID);
// node.setNodeValue(bookingsBOPeriodID);
}
}
// write the content into xml file
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer();
DOMSource source = new DOMSource(doc);
StreamResult result = new StreamResult(new File(filepath));
transformer.transform(source, result);
System.out.println("******* Period Id details updated **************");
} catch (ParserConfigurationException pce) {
pce.printStackTrace();
} catch (TransformerException tfe) {
tfe.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
} catch (SAXException sae) {
sae.printStackTrace();
}
System.out.println("in period id after update :"+ UtilLib.getParam("BookingsBOINPeriodId"));
}
New value from webinterface is passed through "bookingsBOPeriodID" . Immediately after this method the new value is not reflecting in the XML
Make your code wait for sometime before reading the data from the xml.
You should have your resources and processes Synchronized for such opertaions.