Develop Reference

How do you download an image which filename does not end with an extension?

If you right-click on any profile image on Github and open image in new tab you will see that it doesn't end with an extension
For example, here's the actual image of a user on github:
https://avatars0.githubusercontent.com/u/170270?s=60&v=4
Goal
I'm trying to add image saving functionality to my node.js app using request module:
// A proper image link (e.g. *.jpg)
let fileUrl_1 = "https://cdn.pixabay.com/photo/2016/06/20/03/15/pier-1467984_1280.jpg"
// Semi proper image link (e.g. *.jpeg?query)
let fileUrl_2 = "https://images.pexels.com/photos/371633/pexels-photo-371633.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=750&w=1260"
// Not a proper image link (e.g. filename[no extension]?query)
let fileUrl_3 = "https://avatars0.githubusercontent.com/u/170270?s=60&v=4"
let parsedPath = pathModule.parse(fileUrl_1)
let fileName = parsedPath.base
let destinationPath = `C:\\test\\${fileName}`
let request = require("request")
request
.get(fileUrl_1)
.on('error', function(err) {
console.log(err)
})
.pipe(fs.createWriteStream(destinationPath))
Problem
When I try to download images with normal paths like fileUrl_1 it works fine, but if I try to download images like fileUrl_2 or fileUrl_3 shown above I get an error:
ENOENT: no such file or directory, open 'C:\test\170270?s=60&v=4'
BUT, if you just right-click / save image as on a "problematic" image in any browser, you will get a save as dialog window and will be able to save this image as 170270.jpg
Question
How do you download any image with node.js like the save as dialog window does (retrive the image even if it doesn't end with a proper extension)?

It's entirely up to you to choose the destination filename, based on whatever you want. There is no problem in retrieving the image - your only problem is you are trying to save it with an invalid filename.
The server response may include a Content-Disposition header, which may include a recommended default filename.
The filename is always optional and must not be used blindly by the application: path information should be stripped, and conversion to the server file system rules should be done.
It should also include a Content-Type header from which you can derive a file extension, however this header may be incorrect.
Browsers will do MIME sniffing in some cases and will not necessarily follow the value of this header
Or you can "sniff" the first few bytes of the response body and check for a known magic number to indicate the file type.

pathModule.parse is designed to parse file paths, not URLs.
The examples you have where it fails to provide a valid filename are those with a ? in them.
Use a URL parser instead.
const url = require('url');
var fileName = url
.parse(
'https://images.pexels.com/photos/371633/pexels-photo-371633.jpeg?auto=compress&cs=tinysrgb&dpr=2&h=750&w=1260'
)
.pathname.match(/\/([^\/]*)$/)[1];
console.log(fileName);

Set file name and format if file or file location contains neither

I'd like to set the file name and format of a "non-typical file", for a lack of better words. The files are stored at locations such as the following...
let link = https://.../videoplayback?ipbits=0&initcwndb...D134213BA9465CB74DFD36CDE47BF.102638C4A9F3ACA357F79EE747DD5F49F1E0F0DE
When one downloads such files, even if they are completely different files, they always end up being saved with the same name and never a file extension.
I've tried url.download = link + '.mp4', but this has no effect whatsoever.
How can I do this?

According to the MDN for anchor tags with the download attribute:
Can be used with blob: URLs and data: URLs, to make it easy for users
to download content that is generated programmatically using
JavaScript (e.g. a picture created using an online drawing Web app).
If the HTTP header Content-Disposition: is present and gives a
different filename than this attribute, the HTTP header has priority
over this attribute.
If this attribute is present and Content-Disposition: is set to
inline, Firefox gives priority to Content-Disposition, like for the
filename case, while Chrome gives priority to the download attribute.
This attribute is only honored for links to resources with the
same-origin.
So if you are dynamically generating these links and they come from your own server - you can set the filename. A cross-origin request will not work!
You may be able use ajax to fetch the file as a blob and trick the browser into thinking that the data is not cross-origin. See this answer for one possible implementation.

One approach would be to perform a HEAD request before setting .download property of <a> element to determine the Content-Type of the requested resource, use a JSON string, JavaScript object or other key, value pair data storage format to reflect pairs of valid MIME types to valid file extensions corresponding to the MIME type property. Use .indexOf(), RegExp or other filtering method to determine if there is a match between the property name of the string or object and the value of the property, if true store the extension value and concatenate the extension to the suggested file name.
If the file is not served with CORS headers you can try using a proxy to make HEAD of GET request to get the Content-Type header before setting the .download attribute.
It should be noted that the .download attribute is only a suggestion to the user for the filename. The user can change the name of the file at any time, including deleting the file extension, for whatever reason they may or may not have. Or not download the resource at all.
const mimeTypeExtensions = {
"text/plain": ".txt",
"video/mp4": ".mp4",
/* valid MIME types and corresponding extensions */
}
const a = document.querySelector("a");
a.addEventListener("click", event => {
if (!a.download) {
event.preventDefault();
fetch(a.href, {method: "HEAD"})
.then(response => {
const mimeType = response.headers.get("content-type");
let fileExtension = "";
for (let [key, prop] of Object.entries(mimeTypeExtensions)) {
if (key.indexOf(mimeType) > -1) {
fileExtension = prop;
break;
};
}
a.download = `filename${fileExtension}`;
a.click();
})
.catch(err => console.error(err))
}
});
click

Here is an easy option if you don't mind adding a library. FileSaver.js handles these cases well while hiding the ugly details. I was going to go into how you could create Blob, write the contents of the download into it, and then create an anchor tag with an Object URL created from the Blob, but FileSaver handles that already and you could look at the source if you really wanted.
Just modify the following to add some checks and logic for determining filename/filetype and you should be able to tackle your use case.
<script src="cdn-to-FileSaver.js"></script>
<script>
var url = ...;
fetch(url).then((response) => {
var blob = response.blob();
var filename = 'song.mp4';
saveAs(blob, filename);
});
</script>

How to open a CSV file generated by POST

I am trying to implement these seemingly simple requirements but can't find a way :
Single Page App using Angular JS
REST(ish) back end
Back end resource exposed via POST request
Resource parameters passed as JSON in the request body
Resource produces a CSV file
When a user clicks a button, generate a request with the right JSON parameters in the body, send it, and allow user to download the response as a file (prompts the browser's "open / save as" dialog)
The problem is mainly, how to pass the JSON as request body? The most common technique seems to be the hidden HTML form to trigger the download, but an HTML form cannot send JSON data in the body. And I can't find any way to trigger a download dialog using an XMLHttpRequest...
Any ideas?
I specified Angular but any generic JS solution is very welcome too!

I finally found a solution that satisfies all my requirements, and works in IE11, FF and Chrome (and degrades kind of OK in Safari...).
The idea is to create a Blob object containing the data from the response, then force the browser to open it as a file. It is slightly different for IE (proprietary API) and Chrome/FF (using a link element).
Here is the implementation, as a small Angular service:
myApp.factory('Download', [function() {
return {
openAsFile : function(response){
// parse content type header
var contentTypeStr = response.headers('Content-Type');
var tokens = contentTypeStr.split('/');
var subtype = tokens[1].split(';')[0];
var contentType = {
type : tokens[0],
subtype : subtype
};
// parse content disposition header, attempt to get file name
var contentDispStr = response.headers('Content-Disposition');
var proposedFileName = contentDispStr ? contentDispStr.split('"')[1] : 'data.'+contentType.subtype;
// build blob containing response data
var blob = new Blob([response.data], {type : contentTypeStr});
if (typeof window.navigator.msSaveBlob !== 'undefined'){
// IE : use proprietary API
window.navigator.msSaveBlob(blob, proposedFileName);
}else{
var downloadUrl = URL.createObjectURL(blob);
// build and open link - use HTML5 a[download] attribute to specify filename
var a = document.createElement("a");
// safari doesn't support this yet
if (typeof a.download === 'undefined') {
window.open(downloadUrl);
}
var link = document.createElement('a');
link.href = downloadUrl;
link.download = proposedFileName;
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
}
}
}
}]);
The response argument expects a $http response object. Here is an example of use with a POST request:
$http.post(url, {property : 'value'}, {responseType: 'blob'}).then(function(response){
Download.openAsFile(response);
});
Note the responseType parameter. Without this, my CSV data was being read as text and stored in memory as UTF-8 (or 16), and subsequently the file was saved in the same encoding, causing Excel to not recognize special characters such as éè etc. Since my CSVs are intended to be opened by Excel, the server encodes them Windows 1252, I wanted to keep them that way. Setting the responseType parameter to blob achieves this.
Disclaimer: It should work with any file type. But I tested it only with CSV files ! Binary files might behave somehow differently !

Pass PDF document to user-agent via XHR, using modern (HTML5) methods, without compromising document encoding

I am trying to accomplish exactly what is described at Handle file download from ajax post , but with a dynamically-generated PDF file, which I'm generating with PHP v5.6.10 (using the third-party PDFLib extension, v9.0.5).
Yet, when the PDF file is downloaded, it is corrupted; no PDF-reading implementation that I've tried is able to read the file, and every observation points to the fact that the file content is being butchered somewhere between printing the PDF content to the response body and saving the file via the user-agent (web-browser) with JavaScript.
I happen to be using jQuery v2.1.4, but I'm not sure that it matters, ultimately.
Important Provisos
I should mention that, like the other asker (cited above), I have an HTML form that users fill-out and submit via the POST verb. The form submission is performed with JavaScript, because there are actually 5 forms displayed in a tabbed layout that are submitted simultaneously, and any validation errors must be sent back via AJAX and displayed without refreshing the entire page). I mention this to make clear the fact that this is a POST request, which may return either a) a JSON object (that contains validation error strings, primarily), or b) a string that represents a PDF document, which should be presented to the user-agent as a file download.
My Code
The JavaScript
$('#submit-button').click(function() {
$.ajax({
url: $(this).data('action'),
type: 'post',
data: $($(this).data('forms')).serialize(),
processData: false,
statusCode: {
500: function() {
alert('An internal server error occurred. Go pound sand.');
}
}
}).done(function(data, status, xhr) {
processResponse(data, status, xhr);
}).fail(function(jqXHR, textStatus) {
if (textStatus === 'timeout') {
alert('The request timed-out. Please try again.');
}
});
});
function processResponse(response, status, xhr)
{
if (response !== null && typeof response === 'object') {
//The server will return either a JSON string (if the input was invalid)
//or the PDF file. We land here in the former case.
}
else {
//This doesn't change the behavior.
xhr.responseType = 'blob';
//This doesn't change the behavior, either.
//xhr.overrideMimeType('text\/plain; charset=x-user-defined');
//The remainder of this function taken verbatim from:
//https://stackoverflow.com/a/23797348
// check for a filename
var filename = "";
var disposition = xhr.getResponseHeader('Content-Disposition');
if (disposition && disposition.indexOf('attachment') !== -1) {
var filenameRegex = /filename[^;=\n]*=((['"]).*?\2|[^;\n]*)/;
var matches = filenameRegex.exec(disposition);
if (matches != null && matches[1]) filename = matches[1].replace(/['"]/g, '');
}
var type = xhr.getResponseHeader('Content-Type');
//Is logged to console as "application/pdf".
console.log(type);
var blob = new Blob([response], { type: type });
if (typeof window.navigator.msSaveBlob !== 'undefined') {
// IE workaround for "HTML7007: One or more blob URLs were revoked by closing the blob for which they were created. These URLs will no longer resolve as the data backing the URL has been freed."
window.navigator.msSaveBlob(blob, filename);
} else {
var URL = window.URL || window.webkitURL;
var downloadUrl = URL.createObjectURL(blob);
//Is logged to console as URL() (it's an object, not a string).
console.log(URL);
//Is logged to console as "blob:https://example.com/108eb066-645c-4859-a4d2-6f7a42f4f369"
console.log(downloadUrl);
//Is logged to console as "pdftest.pdf".
console.log(filename);
if (filename) {
// use HTML5 a[download] attribute to specify filename
var a = document.createElement("a");
// safari doesn't support this yet
if (typeof a.download === 'undefined') {
window.location = downloadUrl;
} else {
a.href = downloadUrl;
a.download = filename;
document.body.appendChild(a);
a.click();
}
} else {
window.location = downloadUrl;
}
setTimeout(function () { URL.revokeObjectURL(downloadUrl); }, 100); // cleanup
}
}
}
The PHP
<?php
use File;
use \PDFLib;
class Pdf {
protected $p;
protected $bufferedContent;
public function __construct()
{
$this->p = new PDFlib();
$this->p->set_option('errorpolicy=return');
$this->p->set_option('textformat=utf8');
$this->p->set_option('escapesequence=true');
}
//...
public function sendToBrowser()
{
$this->bufferedContent = $this->p->get_buffer();
header_remove();
header('Content-Type: application/pdf');
header('Content-Length: ' . strlen($this->bufferedContent));
header('Content-Disposition: attachment; filename=pdftest.pdf');
$bytesWritten = File::put(realpath(__DIR__ . '/../../public/assets/pdfs') . '/' . uniqid() . '.pdf', $this->bufferedContent);
echo $this->bufferedContent;
exit;
}
//...
}
Notice that in the PHP method I am writing the PDF file to disk prior to sending it in the response body. I added this bit to determine whether the PDF file written to disk is corrupted, too, and it is not; it opens perfectly well in every reader I've tried.
Observations and Theories
What I find so strange about this is that I've tried the download in three different browsers (the most recent versions of Chrome, Firefox, and IE 11) and the PDF size is drastically different with each browser. Following are the file sizes from each:
Written to disk (not corrupted): 105KB
Chrome: 193KB
Firefox: 188KB
IE 11: 141KB
At this point, I am convinced that the problem relates to the encoding used within the PDF. I discovered a discrepancy when using WinMerge to compare the copy of the PDF that I dump directly to disk before returning the HTTP response with the copy that is handled via AJAX.
The first clue was this error message, which appears when I attempt to compare the two PDF documents:
I click OK to dismiss the error, and the comparison resumes.
The functional/correct PDF (at right, in WinMerge) is encoded using Windows-1252 (CP1252); I assume that that encoding happens within PDFLib (despite running on a GNU/Linux system). One can see from the PHP snippet, above, that I am calling $this->p->set_option('textformat=utf8'); explicitly, but that seems to set the encoding for input text that is included in the PDF document (and not the document encoding).
Ultimately, I am left wondering if there is any means by which to get this PDF to be displayed correctly after download.
Change the PDF Encoding, Instead?
I wonder if there is a "good reason" for which PDFLib is using Windows-1252 encoding to generate the PDF document. Is there any chance that this is as simple as changing the encoding on the PDFLib side to match what jQuery's AJAX implementation requires (UTF-8)?
I've consulted the PDFLib manual for more information, and there is a section dedicated to this subject: 4.2 Unicode-capable Language Bindings. This section has two subsections: 4.2.1 Language Bindings with native Unicode Strings (PHP is not among them) and 4.2.2 Language Bindings with UTF-8 Support (PHP falls into this category). But everything discussed herein seems to pertain to the actual strings that are inserted into the PDF body, and not to the overall document encoding.
Then there is 4.4 Single-Byte (8-Bit) Encodings, with the following note:
Note The information in this section is unlikely to be required in
Unicode workflows.
How does one employ a Unicode workflow in this context?
The manual is available at http://www.pdflib.com/fileadmin/pdflib/pdf/manuals/PDFlib-9.0.5-tutorial.pdf for anyone who feels it may be useful.
Approaches That I'd Prefer to Avoid
I really hesitate to get into the business of re-encoding the PDF in JavaScript, client-side, once it has been downloaded. If that is the only means by which to achieve this, I will go another direction.
Initially, my primary aim was to avoid an approach that could leave abandoned PDF files laying-around on the server, in some temporary directory (thereby necessitating a clean-up cron-job or similar), but that may be the only viable option.
If necessary, I will implement an interstitial step whereby I write the PDF file to disk (on the web-server), pass it to the client using some unsightly hidden-iframe hack, and then delete the file once the user-agent receives it. Of course, if the user-agent never finishes the download, the user closes the browser, etc., the file will be abandoned and I'll be left to clean it up by some other means (the idea of which I hate, on principle).
Any assistance with this is hugely appreciated.

You tried this on iframe?
I have this same problem, but i resolve by iframe.
ugly code but works for me.
solution with iframe

Creating download prompt using purely javascript

I have some text data (say var a = 'Hello World From Javascript';)in javascript variable in current window. I want to do the following
through javascript-
1. open a new window and write the text data to the window.
2. set the content type to text/plain.
3. set the content-disposition to attachment, so that download prompt comes.
4. user downloads the text data as a text file and saves it to his local disk.
is this all possible through javascript?
I know we can make ajax calls to server or redirect but in this case instead of following above steps. But in this case, these workarounds are not adaptable.

you can do that using JS & HTML5 features. Please find below a sample code.
var fileParts = ['Hello World From Javascript'];
// Create a blob object.
var bb = new Blob(fileParts,{type : 'text/plain'});
// Create a blob url for this.
var dnlnk = window.URL.createObjectURL(bb);
var currentLnk = $('#blobFl').attr('href');
// blobFl is the id of the anchor tag through which the download will be triggered.
$('#blobFl').attr('href',dnlnk);
$('#blobFl').attr('download','helloworld.txt');
// For some reason trigger from jquery dint work for me.
document.getElementById('blobFl').click();

Triggering a file download without any server request
Unfortunately this is not something you can do with normal browser capabilities. Something like flash or a browser-specific plugin will get you what you need, but security limitations within javascript will not let you download arbitrary data created within the browser.
Also the 'data' url is not supported across all browser/version combinations. I am not sure if your users are constrained on what browser they are using or not but that may limit what you can do with that solution.
Source: Triggering a file download without any server request

If you already have the file on the server (I make an ajax call to generate and save a PDF on the server) - you can do this
window.location.replace(fileUrl);

No, Content-Disposition is a response header, it has to come from the server. I think you could do it with Flash but I wouldn't recommend it.

Here's a clean, pure js version of #Rajagopalan Srinivasan's answer:
var fileParts = ["Hello World From Javascript"];
// The anchor tag to use.
const blobLink = document.getElementById("blobLink");
// Create a blob object.
var blob = new Blob(fileParts, { type: "text/plain" });
// Create a blob url for this.
var blobUrl = window.URL.createObjectURL(blob);
blobLink.setAttribute("href", blobUrl);
blobLink.setAttribute("download", "helloworld.txt");
blobLink.click();
<a id="blobLink">Download</a>