JS - how to batch multiple dynamic imports into one? [closed] - javascript

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
I have two separate but dependent dynamic imports requested from my JS application at virtually the same time. How can I avoid make two import calls or batch them into one?

You can combine the power of URL.createObjectURL() and dynamic imports to import multiple files in one HTTP call.
Server setup
You'll obviously need some sort of api to fetch multiple files in one API call. For this you'll need to make the server somehow send multiple files in one HTTP response. The syntax can vary, but for this example I'm using the syntax GET /a.js+b.js, which will return a string.
Example: 16 24;export default 3export default [2, 3, 5]. This has two files, one with a length of 16 characters, and one with 24. The numbers before the ; are like metadata for the contents of the files. Idk you might put the metadata in the headers or something, but this example uses the ; seperating the metadata and contents.
Client side code
I created a function called fetchMultiple, which is like fetch, but it returns a Promise<Array<Promise< the data exported by the files >>>.
// I created a syntax where it goes
// {length of file 1} {length of file 2} {...lengths of files};(no \n)
// {contents of file 1} {contents of file 2} {...contents of files}
const mockServerResponses = new Map()
.set('a.js+b.js', '16 24;export default 3export default [2, 3, 5]')
// The real thing would fetch the files from the server
const fetchMultiple = async (...urls) =>
mockServerResponses.get(urls.join('+'))
// You could probably optimize this function to load the first script as soon as it is streamed, so that the first script will be loaded while the second one is still being streamed.
const importMultiple = async (...urls) => {
const result = await fetchMultiple(...urls)
const semi = result.indexOf(';')
const lengths = result.slice(0, semi).split(' ').map(str =>
parseInt(str))
const rawContents = result.slice(semi + 1)
let currentIndex = 0
const contents = []
for (const length of lengths) {
contents.push(rawContents.slice(currentIndex, currentIndex + length))
currentIndex += length
}
return contents.map(content =>
import(URL.createObjectURL(new Blob(
[content],
{ type: 'application/javascript' }
))))
}
importMultiple('a.js', 'b.js')
.then(promises => Promise.all(promises))
.then(console.log)
In case the snippet stops working (like a Content Security Policy change), here is the link to the repl: https://replit.com/#Programmerraj/dynamic-import-url#script.js.
Optimizations
What could make the example above slow is that it waits for the entire two (or more) files to get fetched, and then it loads them. Since the files are streamed file 1, file2, ...files, faster code would load file 1 as soon as it is available, and load the other files as they get downloaded.
I didn't implement this optimized stream stuff because I didn't setup a server which streams a response, but you could for maximum efficiency.

Related

Unzip a zip file with JavaScript [duplicate]

I want to display OpenOffice files, .odt and .odp at client side using a web browser.
These files are zipped files. Using Ajax, I can get these files from server but these are zipped files. I have to unzip them using JavaScript, I have tried using inflate.js, http://www.onicos.com/staff/iz/amuse/javascript/expert/inflate.txt, but without success.
How can I do this?
I wrote an unzipper in Javascript. It works.
It relies on Andy G.P. Na's binary file reader and some RFC1951 inflate logic from notmasteryet. I added the ZipFile class.
working example:
http://cheeso.members.winisp.net/Unzip-Example.htm (dead link)
The source:
http://cheeso.members.winisp.net/srcview.aspx?dir=js-unzip (dead link)
NB: the links are dead; I'll find a new host soon.
Included in the source is a ZipFile.htm demonstration page, and 3 distinct scripts, one for the zipfile class, one for the inflate class, and one for a binary file reader class. The demo also depends on jQuery and jQuery UI. If you just download the js-zip.zip file, all of the necessary source is there.
Here's what the application code looks like in Javascript:
// In my demo, this gets attached to a click event.
// it instantiates a ZipFile, and provides a callback that is
// invoked when the zip is read. This can take a few seconds on a
// large zip file, so it's asynchronous.
var readFile = function(){
$("#status").html("<br/>");
var url= $("#urlToLoad").val();
var doneReading = function(zip){
extractEntries(zip);
};
var zipFile = new ZipFile(url, doneReading);
};
// this function extracts the entries from an instantiated zip
function extractEntries(zip){
$('#report').accordion('destroy');
// clear
$("#report").html('');
var extractCb = function(id) {
// this callback is invoked with the entry name, and entry text
// in my demo, the text is just injected into an accordion panel.
return (function(entryName, entryText){
var content = entryText.replace(new RegExp( "\\n", "g" ), "<br/>");
$("#"+id).html(content);
$("#status").append("extract cb, entry(" + entryName + ") id(" + id + ")<br/>");
$('#report').accordion('destroy');
$('#report').accordion({collapsible:true, active:false});
});
}
// for each entry in the zip, extract it.
for (var i=0; i<zip.entries.length; i++) {
var entry = zip.entries[i];
var entryInfo = "<h4><a>" + entry.name + "</a></h4>\n<div>";
// contrive an id for the entry, make it unique
var randomId = "id-"+ Math.floor((Math.random() * 1000000000));
entryInfo += "<span class='inputDiv'><h4>Content:</h4><span id='" + randomId +
"'></span></span></div>\n";
// insert the info for one entry as the last child within the report div
$("#report").append(entryInfo);
// extract asynchronously
entry.extract(extractCb(randomId));
}
}
The demo works in a couple of steps: The readFile fn is triggered by a click, and instantiates a ZipFile object, which reads the zip file. There's an asynchronous callback for when the read completes (usually happens in less than a second for reasonably sized zips) - in this demo the callback is held in the doneReading local variable, which simply calls extractEntries, which
just blindly unzips all the content of the provided zip file. In a real app you would probably choose some of the entries to extract (allow the user to select, or choose one or more entries programmatically, etc).
The extractEntries fn iterates over all entries, and calls extract() on each one, passing a callback. Decompression of an entry takes time, maybe 1s or more for each entry in the zipfile, which means asynchrony is appropriate. The extract callback simply adds the extracted content to an jQuery accordion on the page. If the content is binary, then it gets formatted as such (not shown).
It works, but I think that the utility is somewhat limited.
For one thing: It's very slow. Takes ~4 seconds to unzip the 140k AppNote.txt file from PKWare. The same uncompress can be done in less than .5s in a .NET program. EDIT: The Javascript ZipFile unpacks considerably faster than this now, in IE9 and in Chrome. It is still slower than a compiled program, but it is plenty fast for normal browser usage.
For another: it does not do streaming. It basically slurps in the entire contents of the zipfile into memory. In a "real" programming environment you could read in only the metadata of a zip file (say, 64 bytes per entry) and then read and decompress the other data as desired. There's no way to do IO like that in javascript, as far as I know, therefore the only option is to read the entire zip into memory and do random access in it. This means it will place unreasonable demands on system memory for large zip files. Not so much a problem for a smaller zip file.
Also: It doesn't handle the "general case" zip file - there are lots of zip options that I didn't bother to implement in the unzipper - like ZIP encryption, WinZip encryption, zip64, UTF-8 encoded filenames, and so on. (EDIT - it handles UTF-8 encoded filenames now). The ZipFile class handles the basics, though. Some of these things would not be hard to implement. I have an AES encryption class in Javascript; that could be integrated to support encryption. Supporting Zip64 would probably useless for most users of Javascript, as it is intended to support >4gb zipfiles - don't need to extract those in a browser.
I also did not test the case for unzipping binary content. Right now it unzips text. If you have a zipped binary file, you'd need to edit the ZipFile class to handle it properly. I didn't figure out how to do that cleanly. It does binary files now, too.
EDIT - I updated the JS unzip library and demo. It now does binary files, in addition to text. I've made it more resilient and more general - you can now specify the encoding to use when reading text files. Also the demo is expanded - it shows unzipping an XLSX file in the browser, among other things.
So, while I think it is of limited utility and interest, it works. I guess it would work in Node.js.
I'm using zip.js and it seems to be quite useful. It's worth a look!
Check the Unzip demo, for example.
I found jszip quite useful. I've used so far only for reading, but they have create/edit capabilities as well.
Code wise it looks something like this
var new_zip = new JSZip();
new_zip.load(file);
new_zip.files["doc.xml"].asText() // this give you the text in the file
One thing I noticed is that it seems the file has to be in binary stream format (read using the .readAsArrayBuffer of FileReader(), otherwise I was getting errors saying I might have a corrupt zip file
Edit: Note from the 2.x to 3.0.0 upgrade guide:
The load() method and the constructor with data (new JSZip(data)) have
been replaced by loadAsync().
Thanks user2677034
If you need to support other formats as well or just need good performance, you can use this WebAssembly library
it's promised based, it uses WebWorkers for threading and API is actually simple ES module
How to use
Install with npm i libarchive.js and use it as a ES module.
The library consists of two parts: ES module and webworker bundle, ES module part is your interface to talk to library, use it like any other module. The webworker bundle lives in the libarchive.js/dist folder so you need to make sure that it is available in your public folder since it will not get bundled if you're using bundler (it's all bundled up already) and specify correct path to Archive.init() method.
import {Archive} from 'libarchive.js/main.js';
Archive.init({
workerUrl: 'libarchive.js/dist/worker-bundle.js'
});
document.getElementById('file').addEventListener('change', async (e) => {
const file = e.currentTarget.files[0];
const archive = await Archive.open(file);
let obj = await archive.extractFiles();
console.log(obj);
});
// outputs
{
".gitignore": {File},
"addon": {
"addon.py": {File},
"addon.xml": {File}
},
"README.md": {File}
}
I wrote "Binary Tools for JavaScript", an open source project that includes the ability to unzip, unrar and untar: https://github.com/codedread/bitjs
Used in my comic book reader: https://github.com/codedread/kthoom (also open source).
HTH!
If anyone's reading images or other binary files from a zip file hosted at a remote server, you can use following snippet to download and create zip object using the jszip library.
// this function just get the public url of zip file.
let url = await getStorageUrl(path)
console.log('public url is', url)
//get the zip file to client
axios.get(url, { responseType: 'arraybuffer' }).then((res) => {
console.log('zip download status ', res.status)
//load contents into jszip and create an object
jszip.loadAsync(new Blob([res.data], { type: 'application/zip' })).then((zip) => {
const zipObj = zip
$.each(zip.files, function (index, zipEntry) {
console.log('filename', zipEntry.name)
})
})
Now using the zipObj you can access the files and create a src url for it.
var fname = 'myImage.jpg'
zipObj.file(fname).async('blob').then((blob) => {
var blobUrl = URL.createObjectURL(blob)

Access a page's HTML [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
Is it possible to take a link and access its HTML code through that link? For example I would like to take a link from Amazon and put it within my own HTML code, use JavaScript to getElementsByClassName to get the price from that link and display it back into my HTML code.
It is possible. You could do a GET request to the Amazon page that will give you the html in the response from there you'll have a string now you'll need to format it, last time I used the node module jsdom to do that.
In more detail:
HTTP is a protocol that we use to request data from the server, I've wrote an explanatory node js script:
const https = require('https');
const JSD = require('jsdom');
const { JSDOM } = JSD;
const zlib = require('zlib');
// The http get request
https.get('https://www.amazon.com', (response) => {
html = '';
// we need this because amazon is tricky and encodes the response so it is smaller hence it is faster to send
let gunzip = zlib.createGunzip();
response.pipe(gunzip);
// we need this to get the full html page since it is too big to send in one amazon divides it to chunks
gunzip.on('data', (chunk) => {
html += chunk.toString();
});
// when the transmittion finished we can do wathever we want with it
gunzip.on('end', () => {
let amazon = new JSDOM(html);
console.log(amazon.window.document.querySelector('html').innerHTML);
});
});

File and Directory Entries API broken in Chrome?

I'm trying to use the File and Directory Entries API to create a file uploader tool that will allow me to drop an arbitrary combination of files and directories into a browser window, to be read and uploaded.
(I'm fully aware that similar functionality can be achieved by using an file input element with webkitdirectory enabled, but I'm testing a use case where the user isn't forced to put everything into a single folder)
Using the Drag and Drop API, I've managed to read the DataTransfer items and convert them to FileSystemEntry objects using DataTransferItem.webkitGetAsEntry.
From there, I am able to tell that if the entry is a FileSystemFileEntry or a FileSystemDirectoryEntry. My plan of course if to recursively walk the directory structure, if any, which I should be able to do using the FileSystemDirectoryReader method readEntries, like this:
handleDrop(event) {
event.preventDefault();
event.stopPropagation();
//assuming I dropped only one directory
const directory = event.dataTransfer.items[0];
const directoryEntry = directory.webkitGetAsEntry();
const directoryReader = directoryEntry.createReader();
directoryReader.readEntries(function(entires){
// callback: the "entries" param is an Array
// containing the directory entries
});
}
However, I'm running into the following issue: in Chrome, the readEntries method only returns 100 entries. Apparently, this is the expected behavior as the way to obtain subsequent files from the directory is to call readEntries again. However, I'm finding this impossible to do. A subsequent call to the method throws the error:
DOMException: An operation that depends on state cached in an interface object was made but the state had changed since it was read from disk.
Does anyone know a way around this? Is this API hopelessly broken for directories of 100+ files in Chrome? Is this API deprecated? (not that it was ever "precated"). In Firefox, readEntries returns the whole directory content at once, which apparently against the spec, but it is usable.
Please advice.
Of course, as soon as I had posted this question the answer hit me. What I was trying to do was akin to the following:
handleDrop(event) {
event.preventDefault();
event.stopPropagation();
//assuming I dropped only one directory
const directory = event.dataTransfer.items[0];
const directoryEntry = directory.webkitGetAsEntry();
const directoryReader = directoryEntry.createReader();
directoryReader.readEntries(function(entries){
// callback: the "entries" param is an Array
// containing the directory entries
}, );
directoryReader.readEntries(function(entries){
//call entries a second time
});
}
The problem with this is that readEntries is asynchronous, so I'm trying to call it while it's "busy" reading the first batch (I'm sure lower-level programmers will have a better term for that). A better way of achieving what I was trying to do:
handleDrop(event) {
event.preventDefault();
event.stopPropagation();
//assuming I dropped only one directory
const directory = event.dataTransfer.items[0];
const directoryEntry = directory.webkitGetAsEntry();
const directoryReader = directoryEntry.createReader();
function read(){
directoryReader.readEntries(function(entries){
if(entries.length > 0) {
//do something with the entries
read(); //read the next batch
} else {
//do whatever needs to be done after
//all files are read
}
});
}
read();
}
This way we ensure the FileSystemDirectoryReader is done with one batch before starting the next one.

Best practice for localization and globalization of strings and labels [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I'm a member of a team with more than 20 developers. Each developer works on a separate module (something near 10 modules). In each module we might have at least 50 CRUD forms, which means that we currently have near 500 add buttons, save buttons, edit buttons, etc.
However, because we want to globalized our application, we need to be able to translate texts in our application. For example, everywhere, the word add should become ajouter for French users.
What we've done till now, is that for each view in UI or Presentation Layer, we have a dictionary of key/value pairs of translations. Then while rendering the view, we translate required texts and strings using this dictionary. However, this approach, we've come to have something near 500 add in 500 dictionaries. This means that we've breached DRY principal.
On the other hand, if we centralize common strings, like putting add in one place, and ask developers to use it everywhere, we encounter the problem of not being sure if a string is already defined in the centralized dictionary or not.
One other options might be to have no translation dictionary and use online translation services like Google Translate, Bing Translator, etc.
Another problem that we've encountered is that some developers under the stress of delivering the project on-time can't remember the translation keys. For example, for the text of the add button, a developer has used add while another developer has used new, etc.
What is the best practice, or most well-known method for globalization and localization of string resources of an application?
As far as I know, there's a good library called localeplanet for Localization and Internationalization in JavaScript. Furthermore, I think it's native and has no dependencies to other libraries (e.g. jQuery)
Here's the website of library: http://www.localeplanet.com/
Also look at this article by Mozilla, you can find very good method and algorithms for client-side translation: http://blog.mozilla.org/webdev/2011/10/06/i18njs-internationalize-your-javascript-with-a-little-help-from-json-and-the-server/
The common part of all those articles/libraries is that they use a i18n class and a get method (in some ways also defining an smaller function name like _) for retrieving/converting the key to the value. In my explaining the key means that string you want to translate and the value means translated string.
Then, you just need a JSON document to store key's and value's.
For example:
var _ = document.webL10n.get;
alert(_('test'));
And here the JSON:
{ test: "blah blah" }
I believe using current popular libraries solutions is a good approach.
When you’re faced with a problem to solve (and frankly, who isn’t
these days?), the basic strategy usually taken by we computer people
is called “divide and conquer.” It goes like this:
Conceptualize the specific problem as a set of smaller sub-problems.
Solve each smaller problem.
Combine the results into a solution of the specific problem.
But “divide and conquer” is not the only possible strategy. We can also take a more generalist approach:
Conceptualize the specific problem as a special case of a more general problem.
Somehow solve the general problem.
Adapt the solution of the general problem to the specific problem.
- Eric Lippert
I believe many solutions already exist for this problem in server-side languages such as ASP.Net/C#.
I've outlined some of the major aspects of the problem
Issue: We need to load data only for the desired language
Solution: For this purpose we save data to a separate files for each language
ex. res.de.js, res.fr.js, res.en.js, res.js(for default language)
Issue: Resource files for each page should be separated so we only get the data we need
Solution: We can use some tools that already exist like
https://github.com/rgrove/lazyload
Issue: We need a key/value pair structure to save our data
Solution: I suggest a javascript object instead of string/string air.
We can benefit from the intellisense from an IDE
Issue: General members should be stored in a public file and all pages should access them
Solution: For this purpose I make a folder in the root of web application called Global_Resources and a folder to store global file for each sub folders we named it 'Local_Resources'
Issue: Each subsystems/subfolders/modules member should override the Global_Resources members on their scope
Solution: I considered a file for each
Application Structure
root/
Global_Resources/
default.js
default.fr.js
UserManagementSystem/
Local_Resources/
default.js
default.fr.js
createUser.js
Login.htm
CreateUser.htm
The corresponding code for the files:
Global_Resources/default.js
var res = {
Create : "Create",
Update : "Save Changes",
Delete : "Delete"
};
Global_Resources/default.fr.js
var res = {
Create : "créer",
Update : "Enregistrer les modifications",
Delete : "effacer"
};
The resource file for the desired language should be loaded on the page selected from Global_Resource - This should be the first file that is loaded on all the pages.
UserManagementSystem/Local_Resources/default.js
res.Name = "Name";
res.UserName = "UserName";
res.Password = "Password";
UserManagementSystem/Local_Resources/default.fr.js
res.Name = "nom";
res.UserName = "Nom d'utilisateur";
res.Password = "Mot de passe";
UserManagementSystem/Local_Resources/createUser.js
// Override res.Create on Global_Resources/default.js
res.Create = "Create User";
UserManagementSystem/Local_Resources/createUser.fr.js
// Override Global_Resources/default.fr.js
res.Create = "Créer un utilisateur";
manager.js file (this file should be load last)
res.lang = "fr";
var globalResourcePath = "Global_Resources";
var resourceFiles = [];
var currentFile = globalResourcePath + "\\default" + res.lang + ".js" ;
if(!IsFileExist(currentFile))
currentFile = globalResourcePath + "\\default.js" ;
if(!IsFileExist(currentFile)) throw new Exception("File Not Found");
resourceFiles.push(currentFile);
// Push parent folder on folder into folder
foreach(var folder in parent folder of current page)
{
currentFile = folder + "\\Local_Resource\\default." + res.lang + ".js";
if(!IsExist(currentFile))
currentFile = folder + "\\Local_Resource\\default.js";
if(!IsExist(currentFile)) throw new Exception("File Not Found");
resourceFiles.push(currentFile);
}
for(int i = 0; i < resourceFiles.length; i++) { Load.js(resourceFiles[i]); }
// Get current page name
var pageNameWithoutExtension = "SomePage";
currentFile = currentPageFolderPath + pageNameWithoutExtension + res.lang + ".js" ;
if(!IsExist(currentFile))
currentFile = currentPageFolderPath + pageNameWithoutExtension + ".js" ;
if(!IsExist(currentFile)) throw new Exception("File Not Found");
Hope it helps :)
jQuery.i18n is a lightweight jQuery plugin for enabling internationalization in your web pages. It allows you to package custom resource strings in ‘.properties’ files, just like in Java Resource Bundles. It loads and parses resource bundles (.properties) based on provided language or language reported by browser.
to know more about this take a look at the How to internationalize your pages using JQuery?

Unzipping files

I want to display OpenOffice files, .odt and .odp at client side using a web browser.
These files are zipped files. Using Ajax, I can get these files from server but these are zipped files. I have to unzip them using JavaScript, I have tried using inflate.js, http://www.onicos.com/staff/iz/amuse/javascript/expert/inflate.txt, but without success.
How can I do this?
I wrote an unzipper in Javascript. It works.
It relies on Andy G.P. Na's binary file reader and some RFC1951 inflate logic from notmasteryet. I added the ZipFile class.
working example:
http://cheeso.members.winisp.net/Unzip-Example.htm (dead link)
The source:
http://cheeso.members.winisp.net/srcview.aspx?dir=js-unzip (dead link)
NB: the links are dead; I'll find a new host soon.
Included in the source is a ZipFile.htm demonstration page, and 3 distinct scripts, one for the zipfile class, one for the inflate class, and one for a binary file reader class. The demo also depends on jQuery and jQuery UI. If you just download the js-zip.zip file, all of the necessary source is there.
Here's what the application code looks like in Javascript:
// In my demo, this gets attached to a click event.
// it instantiates a ZipFile, and provides a callback that is
// invoked when the zip is read. This can take a few seconds on a
// large zip file, so it's asynchronous.
var readFile = function(){
$("#status").html("<br/>");
var url= $("#urlToLoad").val();
var doneReading = function(zip){
extractEntries(zip);
};
var zipFile = new ZipFile(url, doneReading);
};
// this function extracts the entries from an instantiated zip
function extractEntries(zip){
$('#report').accordion('destroy');
// clear
$("#report").html('');
var extractCb = function(id) {
// this callback is invoked with the entry name, and entry text
// in my demo, the text is just injected into an accordion panel.
return (function(entryName, entryText){
var content = entryText.replace(new RegExp( "\\n", "g" ), "<br/>");
$("#"+id).html(content);
$("#status").append("extract cb, entry(" + entryName + ") id(" + id + ")<br/>");
$('#report').accordion('destroy');
$('#report').accordion({collapsible:true, active:false});
});
}
// for each entry in the zip, extract it.
for (var i=0; i<zip.entries.length; i++) {
var entry = zip.entries[i];
var entryInfo = "<h4><a>" + entry.name + "</a></h4>\n<div>";
// contrive an id for the entry, make it unique
var randomId = "id-"+ Math.floor((Math.random() * 1000000000));
entryInfo += "<span class='inputDiv'><h4>Content:</h4><span id='" + randomId +
"'></span></span></div>\n";
// insert the info for one entry as the last child within the report div
$("#report").append(entryInfo);
// extract asynchronously
entry.extract(extractCb(randomId));
}
}
The demo works in a couple of steps: The readFile fn is triggered by a click, and instantiates a ZipFile object, which reads the zip file. There's an asynchronous callback for when the read completes (usually happens in less than a second for reasonably sized zips) - in this demo the callback is held in the doneReading local variable, which simply calls extractEntries, which
just blindly unzips all the content of the provided zip file. In a real app you would probably choose some of the entries to extract (allow the user to select, or choose one or more entries programmatically, etc).
The extractEntries fn iterates over all entries, and calls extract() on each one, passing a callback. Decompression of an entry takes time, maybe 1s or more for each entry in the zipfile, which means asynchrony is appropriate. The extract callback simply adds the extracted content to an jQuery accordion on the page. If the content is binary, then it gets formatted as such (not shown).
It works, but I think that the utility is somewhat limited.
For one thing: It's very slow. Takes ~4 seconds to unzip the 140k AppNote.txt file from PKWare. The same uncompress can be done in less than .5s in a .NET program. EDIT: The Javascript ZipFile unpacks considerably faster than this now, in IE9 and in Chrome. It is still slower than a compiled program, but it is plenty fast for normal browser usage.
For another: it does not do streaming. It basically slurps in the entire contents of the zipfile into memory. In a "real" programming environment you could read in only the metadata of a zip file (say, 64 bytes per entry) and then read and decompress the other data as desired. There's no way to do IO like that in javascript, as far as I know, therefore the only option is to read the entire zip into memory and do random access in it. This means it will place unreasonable demands on system memory for large zip files. Not so much a problem for a smaller zip file.
Also: It doesn't handle the "general case" zip file - there are lots of zip options that I didn't bother to implement in the unzipper - like ZIP encryption, WinZip encryption, zip64, UTF-8 encoded filenames, and so on. (EDIT - it handles UTF-8 encoded filenames now). The ZipFile class handles the basics, though. Some of these things would not be hard to implement. I have an AES encryption class in Javascript; that could be integrated to support encryption. Supporting Zip64 would probably useless for most users of Javascript, as it is intended to support >4gb zipfiles - don't need to extract those in a browser.
I also did not test the case for unzipping binary content. Right now it unzips text. If you have a zipped binary file, you'd need to edit the ZipFile class to handle it properly. I didn't figure out how to do that cleanly. It does binary files now, too.
EDIT - I updated the JS unzip library and demo. It now does binary files, in addition to text. I've made it more resilient and more general - you can now specify the encoding to use when reading text files. Also the demo is expanded - it shows unzipping an XLSX file in the browser, among other things.
So, while I think it is of limited utility and interest, it works. I guess it would work in Node.js.
I'm using zip.js and it seems to be quite useful. It's worth a look!
Check the Unzip demo, for example.
I found jszip quite useful. I've used so far only for reading, but they have create/edit capabilities as well.
Code wise it looks something like this
var new_zip = new JSZip();
new_zip.load(file);
new_zip.files["doc.xml"].asText() // this give you the text in the file
One thing I noticed is that it seems the file has to be in binary stream format (read using the .readAsArrayBuffer of FileReader(), otherwise I was getting errors saying I might have a corrupt zip file
Edit: Note from the 2.x to 3.0.0 upgrade guide:
The load() method and the constructor with data (new JSZip(data)) have
been replaced by loadAsync().
Thanks user2677034
If you need to support other formats as well or just need good performance, you can use this WebAssembly library
it's promised based, it uses WebWorkers for threading and API is actually simple ES module
How to use
Install with npm i libarchive.js and use it as a ES module.
The library consists of two parts: ES module and webworker bundle, ES module part is your interface to talk to library, use it like any other module. The webworker bundle lives in the libarchive.js/dist folder so you need to make sure that it is available in your public folder since it will not get bundled if you're using bundler (it's all bundled up already) and specify correct path to Archive.init() method.
import {Archive} from 'libarchive.js/main.js';
Archive.init({
workerUrl: 'libarchive.js/dist/worker-bundle.js'
});
document.getElementById('file').addEventListener('change', async (e) => {
const file = e.currentTarget.files[0];
const archive = await Archive.open(file);
let obj = await archive.extractFiles();
console.log(obj);
});
// outputs
{
".gitignore": {File},
"addon": {
"addon.py": {File},
"addon.xml": {File}
},
"README.md": {File}
}
I wrote "Binary Tools for JavaScript", an open source project that includes the ability to unzip, unrar and untar: https://github.com/codedread/bitjs
Used in my comic book reader: https://github.com/codedread/kthoom (also open source).
HTH!
If anyone's reading images or other binary files from a zip file hosted at a remote server, you can use following snippet to download and create zip object using the jszip library.
// this function just get the public url of zip file.
let url = await getStorageUrl(path)
console.log('public url is', url)
//get the zip file to client
axios.get(url, { responseType: 'arraybuffer' }).then((res) => {
console.log('zip download status ', res.status)
//load contents into jszip and create an object
jszip.loadAsync(new Blob([res.data], { type: 'application/zip' })).then((zip) => {
const zipObj = zip
$.each(zip.files, function (index, zipEntry) {
console.log('filename', zipEntry.name)
})
})
Now using the zipObj you can access the files and create a src url for it.
var fname = 'myImage.jpg'
zipObj.file(fname).async('blob').then((blob) => {
var blobUrl = URL.createObjectURL(blob)

Categories

Resources