https://pieroxy.net/blog/pages/lz-string/guide.html
I have some Base64 data url strings which need to be compressed into shorter strings. I have downloaded the LZString library and tried to use Base64String.compress(base64) and Base64String.decompress(compressedBase64) as demonstrated in above official guide.
While the data urls are used for displaying images in offline page, the page is expected to be in compressed size, so each string has to be decompressed within the offline page. Here is a simplified code sample:
var compressed = Base64String.compress(dataUrl);
var script = document.createElement("script");
script.innerHTML = "var dataUrl = Base64String.decompress(" + compressed + ");";
offlineHtml.querySelector("body").appendChild(script);
However, when I download the page and open it, it shows various forms of SyntaxErrors occurred, including missing ) after argument list Invalid or unexpected token Unexpected identifiers. It seems that the string is compressed in a way that creates problematic syntax. Is there any solution, or any other suitable library for base64 string compression?
Instead of trying to concatenate to create a string containing valid Javascript syntax, save the compressed string somewhere else from which static Javascript can retrieve and parse it. You could use <script type="application/json">, or an invisible textarea, or a data attribute:
const compressed = Base64String.compress(dataUrl);
const script = document.createElement("script");
script.dataset.compressed = compressed;
script.textContent = `
var dataUrl = Base64String.decompress(document.currentScript.dataset.compressed);
`;
offlineHtml.querySelector("body").appendChild(script);
(or, preferably, rather than writing the Javascript inline like that, attach a src to the script tag instead, which contains the same content)
Related
I have a problem when it comes to filenames in Node.js where some incoming files have the pound sign # which causes problems when working with the file system and then using a URL spawned from the filename to reaccess it in an <img> tag.
var fileDir = "/foo/bar#s.png/";
var dirContents = fs.readdirSync(fileDir);
document.getElementById("id1").src = encodeURIComponent(path.join('cache', fileDir, dirContents[0]))
// %2Fcache%2Ffoo%2Fbar%23s.png%2Fbar%23s%2001.png :: Broken img
document.getElementById("id2").src = path.join('cache', fileDir, dirContents[1])
// /cache/foo/bar#s.png/bar#s 02.png :: Also broken img
I have control over the directory names, but the final file .pngs are up to the user who loaded the file. How do I go about handling these characters?
You would escape these characters:
document.getElementById("id1").src = path.join('cache',
...fileDir.split("/").map(encodeURIComponent),
encodeURIComponent(dirContents[0]));
However, you will need to make sure that your server properly decodes them again when receiving the image request, not all implementations agree on what to do with percent encoding in file paths. An easier an possibly better approach is to prevent these characters from occurring in file names in the first place.
I'm using CasperJS, but none of this has to do with the web scraping part.
I'm printing a string (containing french characters) to a csv. When I later open that csv file, all of the french characters don't display correctly.
var fs = require('fs');
var content = 'ë,è,é';
var path = 'target.csv';
fs.write(path, content, 'w');
This ends up being written as �
Is there any way, in the JS script, I can change the encoding of the CSV? I know about the notepad save-as trick but it needs to be automated. If not, is it possible to do through command line?
Excel (at least my version) needs the Byte Order Mark to properly display UTF-8 content from a CSV file. You can achieve this by prepending the content with the \uFEFF unicode character in JavaScript.
Here's the full script in PhantomJS for simplicity (CasperJS works in the same way since it runs in PhantomJS):
var fs = require('fs');
var content = 'ë,è,é';
var path = 'data.csv';
fs.write(path, "\uFEFF" + content, 'w');
phantom.exit();
Make sure that the script file itself is also UTF-8 encoded.
I have a large data in form of JSON object in the javascript. I have converted it into the string using JSON.stringify(). Now my use case is to provide this large string in a text file to the user. So for this i have written below code.
HTML code
<button id='text_feed' type="submit">Generate ION Feed</button>
Javascript code
var text = //huge string
$("#text_feed").click(function() {
_generateFeed(text);
});
var _generateFeed = function(text) {
//some code here
$("#textLink").attr("href",
"data:attachment/txt," + encodeURIComponent(text)) [0].click();
});
};
Problem: When the string length is small , i am able to download the data .
But when the string length goes higher (> 10^5) , my page crashes.
This occurred because "encodeUriComponet(text)" is not able to encode large
data.
I also tried window.open("data:attachment/txt," + encodeURIComponent(text));
But again my page got crashed because of the same reason that encodeURIComponet was unable to encode such a large string.
Another approach: I was also thinking of writing the data into a file using HTML5 File write API , but it has support only in Chrome web browser , but i need to make this work for atleast firefox and chrome both.
Use Case
I don't want to do multiple downloads by breaking the data, as i need to have data in a single file in the end.
And my target is to support string of aprroximately 10^6 length. Can anyone help me how to download/write this amount of data into a single file.
From the OP:
I solved it as below.
var _generateFeed = function(text) {
/*some code here*/
var url = URL.createObjectURL( new Blob( [text], {type:'text/plain'} ) );
$("#textLink").attr("href",url)[0].click();
};
Notes:
URL.createObjectURL() is compatible with modern browsers and IE10+, but it is an unstable, experimental technology.
Objects created using URL.createObjectURL() "must be released by calling URL.revokeObjectURL() when you no longer need them." - MDN
I am using JXA (javascript for automation) on my Mac to try to automate iTunes Connect screenshot uploads. Because of this, I want to automate grabbing each image and uploading it, but for iTunes Connect to allow this (using iTMSTransporter), I need to edit a XML file and add in each image's size in bits, as well as get the checksum (type="md5").
I know I could do this manually, but I want to automate it as this will save me a lot of time in the long run, with tons of different screenshots for each localization.
I've used JXA to grab the images, and get their dimensions, but can't figure out getting the size & checksum. Maybe someone can help? Or if not using JXA, maybe there is some sort of other script that JXA can run (like a shell script, which I have no experience with what so ever as of now), or maybe some script I could just run ahead of time and export the XML to a file manually. From there I could use JXA to parse that file.
Here is what I have so far for what it takes to get the image file:
desktopPath = finder.desktop.url();
desktopPath = desktopPath.substring(7, desktopPath.length);
var imagePath = Application('System Events').folders.byName(desktopPath + '/myImage.png');
imageEvents = Application("Image Events");
imageEvents.launch();
imageEvents.name();
img = imageEvents.open(Path(imagePath));
// now I don't know what to do with the image as the documentation is quite difficult for me to understand
I figured it out. I had to use shell scripts to access this info. Idk if there is another way, but this way worked...
// to get the size (newString is the path (as a string) to the file I am getting the size for
var theSize = app.doShellScript("stat -f%z " + newString.replace(" ", "\\ "));
// to get the MD5 checksum (newString is again the path (as a string) to the file I am getting the checksum for
var md5String = newString;
md5String = md5String.replace(" ", "\\ ");
var checksum = app.doShellScript("md5 " + md5String);
checksum = checksum.replace(/\/$/, "").split(' ').pop();
// I popped because I had to format the returned string so it's just the MD5 and not the file path as well. Maybe there is an easier way in shell script, but I'm a newbie to shell scripting
I have a Javascript program that extracts data from a text file (just a plain *.txt file) and then displays it in a table. It works as intended except for one issue: If I update the text file where the data lives in the update does not shows up on the table. The reason is that the file is being cached by the browser.
Is there a way to force Javascript to read from the latest version of the file and not from the cached version? Again, this is not a problem with the javascript file as doing Ctrl-5 and or Shift+Ctrl+R does not works and also updating the javascript file itself behaves as expected. it is only the file where the data is in that is the problem.
To read the text file I use the Webix toolkit which uses AJAX to read the file. The reading and parsing of the text file is done via Javascript and the Webix toolkit. There is very little html in my program.
//the name of the file is dependant of an user specified input
var theURL = rossObj.store_number.store_number + "macaddress.txt ";
webix.ajax(theURL,{
//response
error:function(text, xml, XmlHttpRequest){
webix.alert({ ok:"OK", type:"alert-warning", text:"could not open the following URL --> "+theURL, width:"400px"});
return "mac_address_error";
},
success:function(text, xml, XmlHttpRequest){
//parse the file;
}
});
Try adding a random number to the end of the url as a query string, the server will ignore but the browser will treat it as a new file
var theURL = rossObj.store_number.store_number +
"macaddress.txt?" +
Math.floor((Math.random() * 10000) + 1);