Related
I want to display OpenOffice files, .odt and .odp at client side using a web browser.
These files are zipped files. Using Ajax, I can get these files from server but these are zipped files. I have to unzip them using JavaScript, I have tried using inflate.js, http://www.onicos.com/staff/iz/amuse/javascript/expert/inflate.txt, but without success.
How can I do this?
I wrote an unzipper in Javascript. It works.
It relies on Andy G.P. Na's binary file reader and some RFC1951 inflate logic from notmasteryet. I added the ZipFile class.
working example:
http://cheeso.members.winisp.net/Unzip-Example.htm (dead link)
The source:
http://cheeso.members.winisp.net/srcview.aspx?dir=js-unzip (dead link)
NB: the links are dead; I'll find a new host soon.
Included in the source is a ZipFile.htm demonstration page, and 3 distinct scripts, one for the zipfile class, one for the inflate class, and one for a binary file reader class. The demo also depends on jQuery and jQuery UI. If you just download the js-zip.zip file, all of the necessary source is there.
Here's what the application code looks like in Javascript:
// In my demo, this gets attached to a click event.
// it instantiates a ZipFile, and provides a callback that is
// invoked when the zip is read. This can take a few seconds on a
// large zip file, so it's asynchronous.
var readFile = function(){
$("#status").html("<br/>");
var url= $("#urlToLoad").val();
var doneReading = function(zip){
extractEntries(zip);
};
var zipFile = new ZipFile(url, doneReading);
};
// this function extracts the entries from an instantiated zip
function extractEntries(zip){
$('#report').accordion('destroy');
// clear
$("#report").html('');
var extractCb = function(id) {
// this callback is invoked with the entry name, and entry text
// in my demo, the text is just injected into an accordion panel.
return (function(entryName, entryText){
var content = entryText.replace(new RegExp( "\\n", "g" ), "<br/>");
$("#"+id).html(content);
$("#status").append("extract cb, entry(" + entryName + ") id(" + id + ")<br/>");
$('#report').accordion('destroy');
$('#report').accordion({collapsible:true, active:false});
});
}
// for each entry in the zip, extract it.
for (var i=0; i<zip.entries.length; i++) {
var entry = zip.entries[i];
var entryInfo = "<h4><a>" + entry.name + "</a></h4>\n<div>";
// contrive an id for the entry, make it unique
var randomId = "id-"+ Math.floor((Math.random() * 1000000000));
entryInfo += "<span class='inputDiv'><h4>Content:</h4><span id='" + randomId +
"'></span></span></div>\n";
// insert the info for one entry as the last child within the report div
$("#report").append(entryInfo);
// extract asynchronously
entry.extract(extractCb(randomId));
}
}
The demo works in a couple of steps: The readFile fn is triggered by a click, and instantiates a ZipFile object, which reads the zip file. There's an asynchronous callback for when the read completes (usually happens in less than a second for reasonably sized zips) - in this demo the callback is held in the doneReading local variable, which simply calls extractEntries, which
just blindly unzips all the content of the provided zip file. In a real app you would probably choose some of the entries to extract (allow the user to select, or choose one or more entries programmatically, etc).
The extractEntries fn iterates over all entries, and calls extract() on each one, passing a callback. Decompression of an entry takes time, maybe 1s or more for each entry in the zipfile, which means asynchrony is appropriate. The extract callback simply adds the extracted content to an jQuery accordion on the page. If the content is binary, then it gets formatted as such (not shown).
It works, but I think that the utility is somewhat limited.
For one thing: It's very slow. Takes ~4 seconds to unzip the 140k AppNote.txt file from PKWare. The same uncompress can be done in less than .5s in a .NET program. EDIT: The Javascript ZipFile unpacks considerably faster than this now, in IE9 and in Chrome. It is still slower than a compiled program, but it is plenty fast for normal browser usage.
For another: it does not do streaming. It basically slurps in the entire contents of the zipfile into memory. In a "real" programming environment you could read in only the metadata of a zip file (say, 64 bytes per entry) and then read and decompress the other data as desired. There's no way to do IO like that in javascript, as far as I know, therefore the only option is to read the entire zip into memory and do random access in it. This means it will place unreasonable demands on system memory for large zip files. Not so much a problem for a smaller zip file.
Also: It doesn't handle the "general case" zip file - there are lots of zip options that I didn't bother to implement in the unzipper - like ZIP encryption, WinZip encryption, zip64, UTF-8 encoded filenames, and so on. (EDIT - it handles UTF-8 encoded filenames now). The ZipFile class handles the basics, though. Some of these things would not be hard to implement. I have an AES encryption class in Javascript; that could be integrated to support encryption. Supporting Zip64 would probably useless for most users of Javascript, as it is intended to support >4gb zipfiles - don't need to extract those in a browser.
I also did not test the case for unzipping binary content. Right now it unzips text. If you have a zipped binary file, you'd need to edit the ZipFile class to handle it properly. I didn't figure out how to do that cleanly. It does binary files now, too.
EDIT - I updated the JS unzip library and demo. It now does binary files, in addition to text. I've made it more resilient and more general - you can now specify the encoding to use when reading text files. Also the demo is expanded - it shows unzipping an XLSX file in the browser, among other things.
So, while I think it is of limited utility and interest, it works. I guess it would work in Node.js.
I'm using zip.js and it seems to be quite useful. It's worth a look!
Check the Unzip demo, for example.
I found jszip quite useful. I've used so far only for reading, but they have create/edit capabilities as well.
Code wise it looks something like this
var new_zip = new JSZip();
new_zip.load(file);
new_zip.files["doc.xml"].asText() // this give you the text in the file
One thing I noticed is that it seems the file has to be in binary stream format (read using the .readAsArrayBuffer of FileReader(), otherwise I was getting errors saying I might have a corrupt zip file
Edit: Note from the 2.x to 3.0.0 upgrade guide:
The load() method and the constructor with data (new JSZip(data)) have
been replaced by loadAsync().
Thanks user2677034
If you need to support other formats as well or just need good performance, you can use this WebAssembly library
it's promised based, it uses WebWorkers for threading and API is actually simple ES module
How to use
Install with npm i libarchive.js and use it as a ES module.
The library consists of two parts: ES module and webworker bundle, ES module part is your interface to talk to library, use it like any other module. The webworker bundle lives in the libarchive.js/dist folder so you need to make sure that it is available in your public folder since it will not get bundled if you're using bundler (it's all bundled up already) and specify correct path to Archive.init() method.
import {Archive} from 'libarchive.js/main.js';
Archive.init({
workerUrl: 'libarchive.js/dist/worker-bundle.js'
});
document.getElementById('file').addEventListener('change', async (e) => {
const file = e.currentTarget.files[0];
const archive = await Archive.open(file);
let obj = await archive.extractFiles();
console.log(obj);
});
// outputs
{
".gitignore": {File},
"addon": {
"addon.py": {File},
"addon.xml": {File}
},
"README.md": {File}
}
I wrote "Binary Tools for JavaScript", an open source project that includes the ability to unzip, unrar and untar: https://github.com/codedread/bitjs
Used in my comic book reader: https://github.com/codedread/kthoom (also open source).
HTH!
If anyone's reading images or other binary files from a zip file hosted at a remote server, you can use following snippet to download and create zip object using the jszip library.
// this function just get the public url of zip file.
let url = await getStorageUrl(path)
console.log('public url is', url)
//get the zip file to client
axios.get(url, { responseType: 'arraybuffer' }).then((res) => {
console.log('zip download status ', res.status)
//load contents into jszip and create an object
jszip.loadAsync(new Blob([res.data], { type: 'application/zip' })).then((zip) => {
const zipObj = zip
$.each(zip.files, function (index, zipEntry) {
console.log('filename', zipEntry.name)
})
})
Now using the zipObj you can access the files and create a src url for it.
var fname = 'myImage.jpg'
zipObj.file(fname).async('blob').then((blob) => {
var blobUrl = URL.createObjectURL(blob)
I am trying to upload file with Google App script and React.
Google Script:
function uploadArquivoParaDrive(base64Data, nomeArq, idPasta) {
try{
var splitBase = base64Data.split(','),
type = splitBase[0].split(';')[0].replace('data:','');
var byteCharacters = Utilities.base64Decode(splitBase[1]);
var ss = Utilities.newBlob(byteCharacters, type);
ss.setName(nomeArq);
var file = DriveApp.getFolderById(idPasta).createFile(ss);
return file.getName();
}catch(e){
return 'Erro: ' + e.toString();
}
}
I can run this ant it works:
function uploadFile() {
var image = UrlFetchApp.fetch('url to some image').getBlob();
var file = {
title: 'google_logo.png',
mimeType: 'image/png'
};
file = Drive.Files.insert(file, image);
Logger.log('ID: %s, File size (bytes): %s', file.id, file.fileSize);
}
This is React script:
onSubmit = (e) => {
e.preventDefault();
axios.get(url, {...this.state}, {
headers: {
'Content-Type': 'multipart/form-data'
}
}, (response) => {
console.log(response);
})
};
setFile = (event) => {
console.log(event.target.files)
this.setState({file: event.target.files[0]});
};
render() {
return (
<form>
<input type="file" id="file" onChange={this.setFile} />
<button onClick={this.onSubmit}>ADD</button>
</form>
)
}
I am trying with POST, but I am getting 400 response. I know that this can't be GET request, but and with it, I am getting - 200 with no response.
I can insert rows in sheets, but I want to upload files to Google Drive with Google App Scripts.
I know that there is a way to upload files via Google Scripts and React, because, there is a way without React (google.script.run).
Here are two different approaches that are used in mixed mode. It's unacceptable in some contexts.
Let's say softly 'React is a dummy'. This is an add-on that you should always avoid when somewhat comes to something that you depend on, but you cannot change. See what SOLID is.
Below it is always assumed that you are working in a browser. Your web-pages are hosted in the web application of the Google Apps Script.
The first approach. Using XMLHttpRequests
On client side you have to use XMLHttpRequests call from your browser.
On server side you have to use doGet doPost reserved functions. Always transfer data in a clear and simple format. This will save time searching for errors.
Example https://stackoverflow.com/a/11300412/1393023
The second approach. Using Client-side API
On client side you have to use google.script.run call from your browser.
On server side you have to use your functions. Always transfer data in a clear and simple format. This will save time searching for errors.
Example https://stackoverflow.com/a/15790713/1393023
Consequence
Your example has signs of mixing approaches. Unfortunately, it cannot be quickly debugged.
There is no reason that React is causing the problem. If so, then your architecture is incorrect.
If you want to use axios, then you need to consider the first approach.
If you want to use google.script.run then you need to catch onSubmit then you need to call an interface that implement google.script.run. Usually asynchronously, since the last call will still be completed with a callback.
I have application store and applications have their url. I want to download apks from those urls to my jaggery server. Although below code(my first solution) create myApp.apk successfully, its not work properly.
First i tried to below code,
var url = "http://img.xxx.com/006/someApp.apk";
var data = get(url, {});
var file = new File("myApp.apk");
file.open("w");
file.write(data.data);
file.close();
when i print data.data value, its look like
i also tried,
var file = new File("http://img.xxx.com/006/someApp.apk");
file.saveAs("myApp.txt");
Can anyone help me?
.apk files are Android application files, and they are expected to start with PK, because they are actually zip archives!
They're not meant to be unzipped, although you can do it to see some of the application resources (but there are better ways for reverse engineering .apk files such as Apktool, if that's what you're looking for).
According to jaggery documentations, file.write is writing the String representation of the object to the file. So that's why you are getting an apk file which cannot be installed.
However you can make it work using copyURLToFile in apache commons-io java library as follows since jaggery supports java itself and all of WSO2 products have apache commons-io library in their class path.
<%
var JFileUtils = Packages.org.apache.commons.io.FileUtils;
var JUrl = Packages.java.net.URL;
var JFile = Packages.java.io.File;
var url = new JUrl("http://img.xxx.com/006/someApp.apk");
JFileUtils.copyURLToFile(url, new JFile("myApp.apk"));
print("done");
%>
Your file will be stored on $CARBON_HOME directory by default, unless you specified relative or absolute path to the file.
I need to get the file type of a file with the help of node.js to set the content type. I know I can easily check the file extension but I've also got files without extension which should have the content type image/png, text/html aso.
This is my code (I know it doesn't make much sense but that's the base I need):
var http = require("http"),
fs = require("fs");
http.createServer(function(req, res) {
var data = "";
try {
/*
* Do not use this code!
* It's not async and it has a security issue.
* The code style is also bad.
*/
data = fs.readFileSync("/home/path/to/folder" + req.url);
var type = "???"; // how to get the file type??
res.writeHead(200, {"Content-Type": type});
} catch(e) {
data = "404 Not Found";
res.writeHead(404, {"Content-Type": "text/plain"});
}
res.write(data);
res.end();
}).listen(7000);
I haven't found a function for that in the API so I would be happy if anyone can tell me how to do it.
There is a helper library for looking up mime types https://github.com/broofa/node-mime
var mime = require('mime');
mime.getType('/path/to/file.txt'); // => 'text/plain'
But it still uses the extension for lookup though
Have a look at the mmmagic module. It is a libmagic binding and seems to do exactly what you want.
You should have a look at the command line tool file (Linux). It attempts to guess the filetype based on the first couple of bytes of the file. You can use child_process.spawn to run it from within node.
You want to be looking up the mime type, and thankfully node has a handy library just for that:
https://github.com/bentomas/node-mime#readme
edit:
You should probably look into a static asset server and not be setting any of this stuff yourself. You can use express to do this really easily, or there's a whole host of static file modules, e.g. ecstatic. On the other hand you should probably use nginx to serve static files anyway.
2018 solution
The accepted answer appears to have a Python dependency and the other answers are either out-of-date or presume the file name has some sort of extension.
Please find my more up-to-date answer here
I used this:
npm install mime-types
And, inside the code:
var mime = require('mime-types');
tmpImg.contentType = mime.lookup(fileImageTmp);
Where fileImageTmp is the copy of image stored on the file system (in this case tmp).
The result I can see is: image/jpeg
The best approach I believe is using the file command of the system, that way you have three advantages:
No dependencies,
You will make sure through magic numbers the type of content the file has,
You will be able to create type of contents through magic files.
Example:
let pathToFile = '/path/to/file';
const child_process = require('child_process');
child_process.exec(`"file" ${path}`, (err, res) => {
let results = res.replace('\n', '').split(':');
let stringPath = results[0].trim();
let typeOfFile = results[1].trim();
console.log(stringPath, typeOfFile);
});
Docs:
https://www.man7.org/linux/man-pages/man1/file.1.html
https://nodejs.org/docs/latest-v13.x/api/child_process.html#child_process_child_process_exec_command_options_callback
I want to display OpenOffice files, .odt and .odp at client side using a web browser.
These files are zipped files. Using Ajax, I can get these files from server but these are zipped files. I have to unzip them using JavaScript, I have tried using inflate.js, http://www.onicos.com/staff/iz/amuse/javascript/expert/inflate.txt, but without success.
How can I do this?
I wrote an unzipper in Javascript. It works.
It relies on Andy G.P. Na's binary file reader and some RFC1951 inflate logic from notmasteryet. I added the ZipFile class.
working example:
http://cheeso.members.winisp.net/Unzip-Example.htm (dead link)
The source:
http://cheeso.members.winisp.net/srcview.aspx?dir=js-unzip (dead link)
NB: the links are dead; I'll find a new host soon.
Included in the source is a ZipFile.htm demonstration page, and 3 distinct scripts, one for the zipfile class, one for the inflate class, and one for a binary file reader class. The demo also depends on jQuery and jQuery UI. If you just download the js-zip.zip file, all of the necessary source is there.
Here's what the application code looks like in Javascript:
// In my demo, this gets attached to a click event.
// it instantiates a ZipFile, and provides a callback that is
// invoked when the zip is read. This can take a few seconds on a
// large zip file, so it's asynchronous.
var readFile = function(){
$("#status").html("<br/>");
var url= $("#urlToLoad").val();
var doneReading = function(zip){
extractEntries(zip);
};
var zipFile = new ZipFile(url, doneReading);
};
// this function extracts the entries from an instantiated zip
function extractEntries(zip){
$('#report').accordion('destroy');
// clear
$("#report").html('');
var extractCb = function(id) {
// this callback is invoked with the entry name, and entry text
// in my demo, the text is just injected into an accordion panel.
return (function(entryName, entryText){
var content = entryText.replace(new RegExp( "\\n", "g" ), "<br/>");
$("#"+id).html(content);
$("#status").append("extract cb, entry(" + entryName + ") id(" + id + ")<br/>");
$('#report').accordion('destroy');
$('#report').accordion({collapsible:true, active:false});
});
}
// for each entry in the zip, extract it.
for (var i=0; i<zip.entries.length; i++) {
var entry = zip.entries[i];
var entryInfo = "<h4><a>" + entry.name + "</a></h4>\n<div>";
// contrive an id for the entry, make it unique
var randomId = "id-"+ Math.floor((Math.random() * 1000000000));
entryInfo += "<span class='inputDiv'><h4>Content:</h4><span id='" + randomId +
"'></span></span></div>\n";
// insert the info for one entry as the last child within the report div
$("#report").append(entryInfo);
// extract asynchronously
entry.extract(extractCb(randomId));
}
}
The demo works in a couple of steps: The readFile fn is triggered by a click, and instantiates a ZipFile object, which reads the zip file. There's an asynchronous callback for when the read completes (usually happens in less than a second for reasonably sized zips) - in this demo the callback is held in the doneReading local variable, which simply calls extractEntries, which
just blindly unzips all the content of the provided zip file. In a real app you would probably choose some of the entries to extract (allow the user to select, or choose one or more entries programmatically, etc).
The extractEntries fn iterates over all entries, and calls extract() on each one, passing a callback. Decompression of an entry takes time, maybe 1s or more for each entry in the zipfile, which means asynchrony is appropriate. The extract callback simply adds the extracted content to an jQuery accordion on the page. If the content is binary, then it gets formatted as such (not shown).
It works, but I think that the utility is somewhat limited.
For one thing: It's very slow. Takes ~4 seconds to unzip the 140k AppNote.txt file from PKWare. The same uncompress can be done in less than .5s in a .NET program. EDIT: The Javascript ZipFile unpacks considerably faster than this now, in IE9 and in Chrome. It is still slower than a compiled program, but it is plenty fast for normal browser usage.
For another: it does not do streaming. It basically slurps in the entire contents of the zipfile into memory. In a "real" programming environment you could read in only the metadata of a zip file (say, 64 bytes per entry) and then read and decompress the other data as desired. There's no way to do IO like that in javascript, as far as I know, therefore the only option is to read the entire zip into memory and do random access in it. This means it will place unreasonable demands on system memory for large zip files. Not so much a problem for a smaller zip file.
Also: It doesn't handle the "general case" zip file - there are lots of zip options that I didn't bother to implement in the unzipper - like ZIP encryption, WinZip encryption, zip64, UTF-8 encoded filenames, and so on. (EDIT - it handles UTF-8 encoded filenames now). The ZipFile class handles the basics, though. Some of these things would not be hard to implement. I have an AES encryption class in Javascript; that could be integrated to support encryption. Supporting Zip64 would probably useless for most users of Javascript, as it is intended to support >4gb zipfiles - don't need to extract those in a browser.
I also did not test the case for unzipping binary content. Right now it unzips text. If you have a zipped binary file, you'd need to edit the ZipFile class to handle it properly. I didn't figure out how to do that cleanly. It does binary files now, too.
EDIT - I updated the JS unzip library and demo. It now does binary files, in addition to text. I've made it more resilient and more general - you can now specify the encoding to use when reading text files. Also the demo is expanded - it shows unzipping an XLSX file in the browser, among other things.
So, while I think it is of limited utility and interest, it works. I guess it would work in Node.js.
I'm using zip.js and it seems to be quite useful. It's worth a look!
Check the Unzip demo, for example.
I found jszip quite useful. I've used so far only for reading, but they have create/edit capabilities as well.
Code wise it looks something like this
var new_zip = new JSZip();
new_zip.load(file);
new_zip.files["doc.xml"].asText() // this give you the text in the file
One thing I noticed is that it seems the file has to be in binary stream format (read using the .readAsArrayBuffer of FileReader(), otherwise I was getting errors saying I might have a corrupt zip file
Edit: Note from the 2.x to 3.0.0 upgrade guide:
The load() method and the constructor with data (new JSZip(data)) have
been replaced by loadAsync().
Thanks user2677034
If you need to support other formats as well or just need good performance, you can use this WebAssembly library
it's promised based, it uses WebWorkers for threading and API is actually simple ES module
How to use
Install with npm i libarchive.js and use it as a ES module.
The library consists of two parts: ES module and webworker bundle, ES module part is your interface to talk to library, use it like any other module. The webworker bundle lives in the libarchive.js/dist folder so you need to make sure that it is available in your public folder since it will not get bundled if you're using bundler (it's all bundled up already) and specify correct path to Archive.init() method.
import {Archive} from 'libarchive.js/main.js';
Archive.init({
workerUrl: 'libarchive.js/dist/worker-bundle.js'
});
document.getElementById('file').addEventListener('change', async (e) => {
const file = e.currentTarget.files[0];
const archive = await Archive.open(file);
let obj = await archive.extractFiles();
console.log(obj);
});
// outputs
{
".gitignore": {File},
"addon": {
"addon.py": {File},
"addon.xml": {File}
},
"README.md": {File}
}
I wrote "Binary Tools for JavaScript", an open source project that includes the ability to unzip, unrar and untar: https://github.com/codedread/bitjs
Used in my comic book reader: https://github.com/codedread/kthoom (also open source).
HTH!
If anyone's reading images or other binary files from a zip file hosted at a remote server, you can use following snippet to download and create zip object using the jszip library.
// this function just get the public url of zip file.
let url = await getStorageUrl(path)
console.log('public url is', url)
//get the zip file to client
axios.get(url, { responseType: 'arraybuffer' }).then((res) => {
console.log('zip download status ', res.status)
//load contents into jszip and create an object
jszip.loadAsync(new Blob([res.data], { type: 'application/zip' })).then((zip) => {
const zipObj = zip
$.each(zip.files, function (index, zipEntry) {
console.log('filename', zipEntry.name)
})
})
Now using the zipObj you can access the files and create a src url for it.
var fname = 'myImage.jpg'
zipObj.file(fname).async('blob').then((blob) => {
var blobUrl = URL.createObjectURL(blob)