Develop Reference

How to send a large file from server using Jersey?

I am trying to download and save a large zip file. The zip file is possibly larger than the heap, so I want to use a stream to avoid java.lang.OutOfMemoryError: Java heap space error.
Also, the large zip file is generated on request, so I would like to delete the file after downloading it.
My current code is
#POST
#Path("/downloadLargeZip")
public Response largeZip() throws FileNotFoundException {
File file = generateZipFile(); // generates zip file successfully
FileInputStream input = new FileInputStream(file);
StreamingOutput so = os -> {
try {
int n;
byte[] buffer = new byte[1024];
while ((n = input.read(buffer)) >= 0) {
os.write(buffer, 0, n);
}
os.flush();
os.close();
} catch (Exception e) {
throw new WebApplicationException(e);
}
};
return Response.ok(so).build();
}
My current client-side code is
import { saveAs } from 'browser-filesaver/FileSaver.js';
save() {
this.http.post<any>('url', '', { observe: 'response', responseType: 'blob'})
.subscribe(res => {
this.downloadFile(res);
});
}
downloadFile(response: any) {
const contentDisposition = 'attachment; filename="KNOWN_FILE_NAME"'; // response.headers('content-disposition'); - response object has no headers
// Retrieve file name from content-disposition
let fileName = contentDisposition.substr(contentDisposition.indexOf('filename=') + 9);
fileName = fileName.replace(/\"/g, '');
const contentType = 'application/zip'; // response.headers('content-type');
const blob = new Blob([response.data], { type: contentType });
saveAs(blob, fileName);
}
I have a few problems with my code:
Using dev tools to check the response, it has no headers (normalizedNames is a map with no entries) or data.
Checking the saved zip file, I can't open it using WinRAR. The error is The archive is either in unknown format or damaged.
Trying to open the zip file with Notepad++, the content is the text undefined.
The JSON representation of the response is
{
"headers":{
"normalizedNames":{
},
"lazyUpdate":null
},
"status":200,
"statusText":"OK",
"url":"URL",
"ok":true,
"type":4,
"body":{
}
}
Although the body does contain data {size: 2501157, type: "application/json"}.
Please ignore the number (I am guessing it's the zip file size in bytes, the actual file will be much larger).
What am I doing wrong? How can I read the stream and save the generated zip file?
I think the issue is in my downloadFile function, but I don't know what to change there.
Any help would be appreciated.

I needed to completely change the way I approached the issue.
The server will now generate the file and return a URI for the client. The client will then download the file via given URI.
Server code
#POST
#Path("/create")
public Response createLogs(String data) {
String fileName = generateFileAndReturnName(data);
if (fileName != null) {
return Response.created(URI.create(manipulateUri(fileName))).build();
}
return Response.status(500).build();
}
Client code
save() {
this.http.post<any>(this.baseUrl + '/create', this.data, { observe: 'response'}).subscribe(postResponse => {
if (postResponse.status !== 201) {
this.logger.error('Failed.');
return;
}
postResponse.headers.keys(); // lazy init headers
const uri = postResponse.headers.get('location');
if (!uri) {
this.logger.error('URI not present.');
return;
}
const link = document.createElement('a');
link.href = uri;
link.setAttribute('download', 'fileName.fileExtension');
document.body.appendChild(link);
link.click();
if (link.parentNode) {
link.parentNode.removeChild(link);
}
});
}
Working fine now with 40GB file (32GB RAM, so file is definitely bigger than any allocated heap).

How to create download feature using ASP.NET Core 2.1 and React JS?

I'm using ASP.NET Core and React JS. I'm newbie to this both platforms. I have used Axios for requesting data and getting response from server. But I have not requested images or any kind of file from server. This time I'm working on Download feature where user will click on button and can download desired file which is of .png, .jpg, .pdf format. I'm not understanding how can server will send data? I read, I needed to send base64 data which is converted from blob format. But not understanding how to request data from client and how server will serve desired file. In DB, I have stored only address of file e.g. /images/img1.jpg. This file actually resides in wwwroot/images folder. I have used downloadjs for downloading which is working correctly but after downloading, that image is not readable as it does not have any content.
Please anyone help me to implement this feature.

First you need API to download data something like this
public async Task<IActionResult> Download(string filename)
{
if (filename == null)
return Content("filename not present");
var path = Path.Combine(
Directory.GetCurrentDirectory(),
"wwwroot", filename);
var memory = new MemoryStream();
using (var stream = new FileStream(path, FileMode.Open))
{
await stream.CopyToAsync(memory);
}
memory.Position = 0;
return File(memory, GetContentType(path), Path.GetFileName(path));
}
private string GetContentType(string path)
{
var types = GetMimeTypes();
var ext = Path.GetExtension(path).ToLowerInvariant();
return types[ext];
}
private Dictionary<string, string> GetMimeTypes()
{
return new Dictionary<string, string>
{
{".txt", "text/plain"},
{".pdf", "application/pdf"},
{".doc", "application/vnd.ms-word"},
{".docx", "application/vnd.ms-word"},
{".xls", "application/vnd.ms-excel"},
{".xlsx", "application/vnd.openxmlformats
officedocument.spreadsheetml.sheet"},
{".png", "image/png"},
{".jpg", "image/jpeg"},
{".jpeg", "image/jpeg"},
{".gif", "image/gif"},
{".csv", "text/csv"}
};
}
Then download file like this
axios({
url: 'your url',
method: 'POST', // Worked using POST or PUT. Prefer POST
responseType: 'blob', // important
}).then((response) => {
const url = window.URL.createObjectURL(new Blob([response.data]));
const link = document.createElement('a');
link.href = url;
link.setAttribute('download', 'file.pdf');
document.body.appendChild(link);
link.click();
});
Ref link

Convert base64 to png in meteor app

I have a meteor application and in this one I get a base64 image. I want to save the image on a Digital Ocean instance, so I would convert it in a png or an other image format and send it to the server to get an url of the image.
But I didn't find a meteor package that does this.
Do you know how I can do that ?

I was running into a similar issue.
run the following:
meteor npm install --save file-api
This will allow the following code on the server for example:
import FileAPI from 'file-api';
const { File } = FileAPI;
const getFile = function(name,image){
const i = image.indexOf('base64,');
const buffer = Buffer.from(image.slice(i + 7), 'base64');
const file = new File({buffer: buffer, name, type: 'image/jpeg'});
return file;
}
Simply call it with any name of file you prefer, and the base64 string as the image parameter.
I hope this helps. I have tested this and it works on the server. I have not tested it on the client but I don't see why it wouldn't work.

I solved my problem using fs.writeFile from File System.
This is my javascript code on client side, I got a base64 image (img) from a plugin and when I click on my save button, I do this :
$("#saveImage").click(function() {
var img = $image.cropper("getDataURL")
preview.setAttribute('src', img);
insertionImage(img);
});
var insertionImage = function(img){
//some things...
Meteor.call('saveTileImage', img);
//some things...
}
And on the server side, I have :
Meteor.methods({
saveTileImage: function(fileData) {
var fs = Npm.require('fs');
var path = process.env.PWD + '/var/uploads/';
base64Data = fileData.replace(/^data:image\/png;base64,/, "");
base64Data += base64Data.replace('+', ' ');
binaryData = new Buffer(base64Data, 'base64').toString('binary');
var imageName = "tileImg_" + currentTileId + ".png";
fs.writeFile(path + imageName, binaryData, "binary", Meteor.bindEnvironment(function (err) {
if (err) {
throw (new Meteor.Error(500, 'Failed to save file.', err));
} else {
insertionTileImage(imageName);
}
}));
}
});
var insertionTileImage = function(fileName){
tiles.update({_id: currentTileId},{$set:{image: "upload/" + fileName}});
}
So, the meteor methods saveTileImage transform the base64 image into a png file and insertionTileImage upload it to the server.

BlobUrl, would it be a better option for you?
Save the images to a server as you like in base64 or whatever, and then when you are viewing the image on a page, generate the blobUrl of it. The url being used only at that time, preventing others from using your url on various websites and not overloading your image server ...

How to download multiple files from Google Drive as .zip using JSZip on Salesforce

The case:
On Salesforce platform I use Google Drive to store files (images for this case) with configured Apex Google Drive API Framework. So Google Drive API handles authToken and so on. I can upload and browse images in my application. In my case I want to select multiple files and download them in a single zip file. So far I'm trying to do that using JSZip and FileSaver libraries. With the same code below I can zip and download multiple files stored somewhere else with proper response header, but not from GDrive because of CORS error.
https://xxx.salesforce.com/contenthub/download/XXXXXXXXXX%3Afile%XXXXXX_XXXXXXXXX. No'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://xxx.visual.force.com' is therefore not allowed access. If I just click on this link, file starts to download.
Is there any way to configure GDrive to enable response header: Access-Control-Allow-Origin: * or Access-Control-Allow-Origin: https://*/mydomain.com somehow or I just have to use something else, maybe server side compression? Now I am using the download link provided by Apex Google Drive API (looks like this:
https://xxx.salesforce.com/contenthub/download/XXXXXXXXXXX%3Afile%XXXXXXXX), it works fine when used as src="fileURL" or when pasted directly to the browser. GDrive connector add 'accesToken' and so on.
My code:
//ajax request to get files using JSZipUtils
let urlToPromise = (url) =>{
return new Promise(function(resolve, reject) {
JSZipUtils.getBinaryContent(url, function (err, data) {
if(err) {
reject(err);
} else {
resolve(data);
}
});
});
};
this.downloadAssets = () => {
let zip = new JSZip();
//here 'selectedAssets' array of objects each of them has 'assetFiles'
//with fileURL where I have url. Download and add them to 'zip' one by one
for (var a of this.selectedAssets){
for (let f of a.assetFiles){
let url = f.fileURL;
let name = a.assetName + "." + f.fileType;
let filename = name.replace(/ /g, "");
zip.file(filename, urlToPromise(url), {binary:true});
}
}
//generate zip and download using 'FileSaver.js'
zip.generateAsync({type:"blob"})
.then(function callback(blob) {
saveAs(blob, "test.zip");
});
};
I also tried to change let url = f.fileURL to let url = f.fileURL + '?alt=media'; and &access_token=CURRENT_TOKEN added by GDrive connector.
this link handled by GRDrive connector so if I just enter it in browser it download the image. However, for multiple download using JS I got CORS error.

I think this feature is not yet supported. If you check the Download Files guide from Drive API, there's no mention of downloading multiple files at once. That's because you have to make individual API requests for each file. This is confirmed in this SO thread.
But that selected multiple files are convert into single zip file and download that single zip file which is possible with google drive API. So how can i convert them into single Zip File? please tell me.

According to me, just download all files and store them at temporary directory location and then add that directory to zip file and store that zip to physical device.
public Entity.Result<Entity.GoogleDrive> DownloadMultipleFile(string[] fileidList)
{
var result = new Entity.Result<Entity.GoogleDrive>();
ZipFile zip = new ZipFile();
try
{
var service = new DriveService(new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = "Download File",
});
FilesResource.ListRequest listRequest = service.Files.List();
//listRequest.PageSize = 10;
listRequest.Fields = "nextPageToken, files(id, name, mimeType, fullFileExtension)";
IList<File> files = listRequest.Execute().Files;
if (files != null && files.Count > 0)
{
foreach (var fileid in fileidList)
{
foreach (var file in files)
{
if (file.Id == fileid)
{
result.Data = new Entity.GoogleDrive { FileId = fileid };
FilesResource.GetRequest request = service.Files.Get(fileid);
request.ExecuteAsync();
var stream = new System.IO.FileStream(HttpContext.Current.Server.MapPath(#"~\TempFiles") + "\\" + file.Name, System.IO.FileMode.Create, System.IO.FileAccess.Write);
request.MediaDownloader.ProgressChanged += (IDownloadProgress progress) =>
{
switch (progress.Status)
{
case DownloadStatus.Downloading:
{
break;
}
case DownloadStatus.Completed:
{
break;
}
case DownloadStatus.Failed:
{
break;
}
}
};
request.Download(stream);
stream.Close();
break;
}
}
}
}
zip.AddDirectory(HttpContext.Current.Server.MapPath(#"~\TempFiles"), "GoogleDrive");
string pathUser = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile);
string pathDownload = System.IO.Path.Combine(pathUser, "Downloads");
zip.Save(pathDownload + "\\GoogleDrive.zip");
System.IO.DirectoryInfo di = new System.IO.DirectoryInfo(HttpContext.Current.Server.MapPath(#"~\TempFiles"));
foreach (var file in di.GetFiles())
{
file.Delete();
}
result.IsSucceed = true;
result.Message = "File downloaded suceessfully";
}
catch (Exception ex)
{
result.IsSucceed = false;
result.Message = ex.ToString();
}
return result;
}

My previously published code works. Forgot to post a solution.
Just instead of using content hub link I started to use direct link to Google Drive and CORS issue was solved. Still not sure if CORS might be solved somehow at Salesforce side. Tried different setups with no luck.
Direct download link to GDrive works ok in my case. The only thing I had to change is the prefix to GDrive file ID.

How to create and upload a video to parse server?

I am able to upload image file to S3 using parse server. (by creating parse file from base64 image data and doing save() on parse file)
How can I do the same thing for a video file? I am doing this using parse-server js library in Ionic 2 app with typescript. The below code worked for images.
let file = new Parse.File("thumbnail", { base64: imageData });
file.save().then(() => {
// The file has been saved to Parse.
console.log("File uploaded....");
}, (error) => {
// The file either could not be read, or could not be saved to Parse.
console.log("File upload failed.");
});
In case of a video file, I have the file location received from cordova media capture callback. Help me in uploading the video file.
Thank you

here is my solution after days of research.
it works for iphone.
the important statement is this:
data=data.replace("quicktime","mov");
var options = { limit: 1, duration: 30 };
navigator.device.capture.captureVideo(function(files){
// Success! Audio data is here
console.log("video file ready");
var vFile = files[0];
console.log(vFile.fullPath);
///private/var/mobile/Containers/Data/Application/7A0069EB-F864-438F-A685-A0DAE97F8B2D/tmp/capture-T0x144510b50.tmp.GfXOow/capturedvideo.MOV
self.auctionvideo = vFile.fullPath; //localURL;
console.log(self.auctionvideo);
var fileReader = new FileReader();
var file;
fileReader.onload = function (readerEvt) {
var data = fileReader.result;
data=data.replace("quicktime","mov");
console.log(data);
//data:video/quicktime;base64,AAAAFGZ0
console.log(data.length);
self.auctionvideo=data;
self.videofile = {base64:data};
};
//fileReader.reasAsDataURL(audioFile); //This will result in your problem.
file = new window.File(vFile.name, vFile.localURL,
vFile.type, vFile.lastModifiedDate, vFile.size);
fileReader.readAsDataURL(file); //This will result in the solution.
// fileReader.readAsBinaryString(file); //This will result in the solution.
},
function(error){
},
options);