Develop Reference

Why is event.total in XMLHttpRequest always 0, unless Content-Type is PDF?

I tried various days to make a progress bar for my download, a task which I expected to be quite trivial.
Everything seemed to to be just right, but the event.total-value was always 0 (or undefined in the angular-version.
This is basically how the file got passed:
<?php // backend.php
if (isset($_GET["file"])) {
$file = "dlstuff/{$_GET["file"]}";
}
$stream = fopen($file, 'rb');
header('Content-Length: '. filesize($file));
header("Content-Type: text/plain", true);
fpassthru($stream);
And the header was exposed in .htaccess as stated here:
Header add Access-Control-Expose-Headers "Content-Length"
I obtained the file with Javascript like this:
The application was written in angular, but I could reproduce it with simple plain HTML:
const oReq = new XMLHttpRequest();
oReq.addEventListener("progress", oEvent => {
console.log("progress");
console.log({ loaded: oEvent.loaded, total: oEvent.total, status: oReq.status });
console.log(oReq.getAllResponseHeaders());
});
oReq.addEventListener("load", () => {
console.log("complete");
console.log({ status: oReq.status, downloaded: oReq.response.length });
console.log(oReq.getAllResponseHeaders());
});
oReq.open("GET", "http://localhost/backend.php?file=whatever.htm");
oReq.send();
I read a lot of stuff
XMLHttpRequest "total" returns 0 when fetching JSON
How can I access the Content-Length header from a cross domain Ajax request?
Content-length and other HTTP headers?
Content-Length header with HEAD requests?
But still - no event.total and the Content-Length was not visible in the browser-console.
progress
(index):11 {loaded: 65536, total: 0, status: 200}
(index):12 access-control-expose-headers: Content-Length
connection: Keep-Alive
content-encoding: gzip
content-type: text/plain;charset=UTF-8
date: Mon, 13 Sep 2021 15:09:22 GMT
keep-alive: timeout=5, max=99
server: Apache/2.4.38 (Debian)
transfer-encoding: chunked
vary: Accept-Encoding
I started to loose my mind overt this until I changed the Content-Type-header in the PHP-Script to application/pdf by accident. Suddenly it worked!
What it going on? The real application may have various file formats but I would like to treat them all as text from the Javascript side.
--
Edit 1:
`application/octet-stream``instead of Pdf works as well. But the question remains: why?

Cannot recognize image upload at Google Drive in js

i'm trying to upload image file at google drive, using oauth token & fetch url.
https://developers.google.com/drive/api/v3/manage-uploads
Perform a multipart upload, HTTP.
when i try to upload, fetch url response returns status 200, and in google drive, file is in there. But can't see(recognized no support img).
it's my header
method: post
Authorization: `Bearer ${token}`
Content-Type: `multipart/related; boundary=${boundaryString}`
Content-Length: ${body.Length}
and it's my body
--`${boundaryString}`
Content-Type: application/json; charset=UTF-8
{"name":"myimage.png","description":"Upload image","mimeType":"image/png"}
--`${boundaryString}`
Content-Type: image/png; Content-Transfer-Encoding: base64
data:image/png;base64,iVBO......TkSuQmCC
--`${boundaryString}`--
response :
status: 200 url: "https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart"
body: {
id: "~~~~"
kind: "drive#file"
mimeType: "image/png"
name: "myimage.png"
when i go to drive, it exist. it's details correct(name, description, mimeType),
but can't recognize like another images.(file format is not supported.)
when i check <img src ="data:image/png;base64,iVBO......TkSuQmC" /> it works.
could tell me what's the problem?

How about this modification?
Modification points:
Please remove Content-Type: image/png; from the data part.
Please remove the header of base64 data.
When above points are reflected to your request body, it becomes as follows.
Modified request body:
--`${boundaryString}`
Content-Type: application/json; charset=UTF-8
{"name":"myimage.png","description":"Upload image","mimeType":"image/png"}
--`${boundaryString}`
Content-Transfer-Encoding: base64
iVBO......TkSuQmCC
--`${boundaryString}`--
Note:
In this case, the line breaks are important. Please be careful this.
In this modification, it supposes that your access token can be used for uploading the file to Google Drive.
Although I'm not sure about your actual script, if you use Javascript, how about the following modified script?
var data = `--${boundaryString}
Content-Type: application/json; charset=UTF-8
{"name":"myimage.png","description":"Upload image","mimeType":"image/png"}
--${boundaryString}
Content-Transfer-Encoding: base64
iVBO......TkSuQmCC
--${boundaryString}--`;

How is an image sent when submitting a form? [duplicate]

When I submit a simple form like this with a file attached:
<form enctype="multipart/form-data" action="http://localhost:3000/upload?upload_progress_id=12344" method="POST">
<input type="hidden" name="MAX_FILE_SIZE" value="100000" />
Choose a file to upload: <input name="uploadedfile" type="file" /><br />
<input type="submit" value="Upload File" />
</form>
How does it send the file internally? Is the file sent as part of the HTTP body as data? In the headers of this request, I don't see anything related to the name of the file.
I just would like the know the internal workings of the HTTP when sending a file.

Let's take a look at what happens when you select a file and submit your form (I've truncated the headers for brevity):
POST /upload?upload_progress_id=12344 HTTP/1.1
Host: localhost:3000
Content-Length: 1325
Origin: http://localhost:3000
... other headers ...
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryePkpFF7tjBAqx29L
------WebKitFormBoundaryePkpFF7tjBAqx29L
Content-Disposition: form-data; name="MAX_FILE_SIZE"
100000
------WebKitFormBoundaryePkpFF7tjBAqx29L
Content-Disposition: form-data; name="uploadedfile"; filename="hello.o"
Content-Type: application/x-object
... contents of file goes here ...
------WebKitFormBoundaryePkpFF7tjBAqx29L--
NOTE: each boundary string must be prefixed with an extra --, just like in the end of the last boundary string. The example above already includes this, but it can be easy to miss. See comment by #Andreas below.
Instead of URL encoding the form parameters, the form parameters (including the file data) are sent as sections in a multipart document in the body of the request.
In the example above, you can see the input MAX_FILE_SIZE with the value set in the form, as well as a section containing the file data. The file name is part of the Content-Disposition header.
The full details are here.

How does it send the file internally?
The format is called multipart/form-data, as asked at: What does enctype='multipart/form-data' mean?
I'm going to:
add some more HTML5 references
explain why he is right with a form submit example
HTML5 references
There are three possibilities for enctype:
x-www-urlencoded
multipart/form-data (spec points to RFC2388)
text-plain. This is "not reliably interpretable by computer", so it should never be used in production, and we will not look further into it.
How to generate the examples
Once you see an example of each method, it becomes obvious how they work, and when you should use each one.
You can produce examples using:
nc -l or an ECHO server: HTTP test server accepting GET/POST requests
a user agent like a browser or cURL
Save the form to a minimal .html file:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<title>upload</title>
</head>
<body>
<form action="http://localhost:8000" method="post" enctype="multipart/form-data">
<p><input type="text" name="text1" value="text default">
<p><input type="text" name="text2" value="aωb">
<p><input type="file" name="file1">
<p><input type="file" name="file2">
<p><input type="file" name="file3">
<p><button type="submit">Submit</button>
</form>
</body>
</html>
We set the default text value to aωb, which means aωb because ω is U+03C9, which are the bytes 61 CF 89 62 in UTF-8.
Create files to upload:
echo 'Content of a.txt.' > a.txt
echo '<!DOCTYPE html><title>Content of a.html.</title>' > a.html
# Binary file containing 4 bytes: 'a', 1, 2 and 'b'.
printf 'a\xCF\x89b' > binary
Run our little echo server:
while true; do printf '' | nc -l 8000 localhost; done
Open the HTML on your browser, select the files and click on submit and check the terminal.
nc prints the request received.
Tested on: Ubuntu 14.04.3, nc BSD 1.105, Firefox 40.
multipart/form-data
Firefox sent:
POST / HTTP/1.1
[[ Less interesting headers ... ]]
Content-Type: multipart/form-data; boundary=---------------------------735323031399963166993862150
Content-Length: 834
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="text1"
text default
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="text2"
aωb
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="file1"; filename="a.txt"
Content-Type: text/plain
Content of a.txt.
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="file2"; filename="a.html"
Content-Type: text/html
<!DOCTYPE html><title>Content of a.html.</title>
-----------------------------735323031399963166993862150
Content-Disposition: form-data; name="file3"; filename="binary"
Content-Type: application/octet-stream
aωb
-----------------------------735323031399963166993862150--
For the binary file and text field, the bytes 61 CF 89 62 (aωb in UTF-8) are sent literally. You could verify that with nc -l localhost 8000 | hd, which says that the bytes:
61 CF 89 62
were sent (61 == 'a' and 62 == 'b').
Therefore it is clear that:
Content-Type: multipart/form-data; boundary=---------------------------735323031399963166993862150 sets the content type to multipart/form-data and says that the fields are separated by the given boundary string.
But note that the:
boundary=---------------------------735323031399963166993862150
has two less dadhes -- than the actual barrier
-----------------------------735323031399963166993862150
This is because the standard requires the boundary to start with two dashes --. The other dashes appear to be just how Firefox chose to implement the arbitrary boundary. RFC 7578 clearly mentions that those two leading dashes -- are required:
4.1. "Boundary" Parameter of multipart/form-data
As with other multipart types, the parts are delimited with a
boundary delimiter, constructed using CRLF, "--", and the value of
the "boundary" parameter.
every field gets some sub headers before its data: Content-Disposition: form-data;, the field name, the filename, followed by the data.
The server reads the data until the next boundary string. The browser must choose a boundary that will not appear in any of the fields, so this is why the boundary may vary between requests.
Because we have the unique boundary, no encoding of the data is necessary: binary data is sent as is.
TODO: what is the optimal boundary size (log(N) I bet), and name / running time of the algorithm that finds it? Asked at: https://cs.stackexchange.com/questions/39687/find-the-shortest-sequence-that-is-not-a-sub-sequence-of-a-set-of-sequences
Content-Type is automatically determined by the browser.
How it is determined exactly was asked at: How is mime type of an uploaded file determined by browser?
application/x-www-form-urlencoded
Now change the enctype to application/x-www-form-urlencoded, reload the browser, and resubmit.
Firefox sent:
POST / HTTP/1.1
[[ Less interesting headers ... ]]
Content-Type: application/x-www-form-urlencoded
Content-Length: 51
text1=text+default&text2=a%CF%89b&file1=a.txt&file2=a.html&file3=binary
Clearly the file data was not sent, only the basenames. So this cannot be used for files.
As for the text field, we see that usual printable characters like a and b were sent in one byte, while non-printable ones like 0xCF and 0x89 took up 3 bytes each: %CF%89!
Comparison
File uploads often contain lots of non-printable characters (e.g. images), while text forms almost never do.
From the examples we have seen that:
multipart/form-data: adds a few bytes of boundary overhead to the message, and must spend some time calculating it, but sends each byte in one byte.
application/x-www-form-urlencoded: has a single byte boundary per field (&), but adds a linear overhead factor of 3x for every non-printable character.
Therefore, even if we could send files with application/x-www-form-urlencoded, we wouldn't want to, because it is so inefficient.
But for printable characters found in text fields, it does not matter and generates less overhead, so we just use it.

Send file as binary content (upload without form or FormData)
In the given answers/examples the file is (most likely) uploaded with a HTML form or using the FormData API. The file is only a part of the data sent in the request, hence the multipart/form-data Content-Type header.
If you want to send the file as the only content then you can directly add it as the request body and you set the Content-Type header to the MIME type of the file you are sending. The file name can be added in the Content-Disposition header. You can upload like this:
var xmlHttpRequest = new XMLHttpRequest();
var file = ...file handle...
var fileName = ...file name...
var target = ...target...
var mimeType = ...mime type...
xmlHttpRequest.open('POST', target, true);
xmlHttpRequest.setRequestHeader('Content-Type', mimeType);
xmlHttpRequest.setRequestHeader('Content-Disposition', 'attachment; filename="' + fileName + '"');
xmlHttpRequest.send(file);
If you don't (want to) use forms and you are only interested in uploading one single file this is the easiest way to include your file in the request.
Update:
In all modern browsers you can these days also use the fetch API for (binary) upload. The same as mentioned in the example above would then look like this:
const promise = fetch(target, {
method: 'POST',
body: file,
headers: {
'Content-Type': mimeType,
'Content-Disposition', `attachment; filename="${fileName}"`,
},
});
promise.then(
(response) => { /*...do something with response*/ },
(error) => { /*...handle error*/ },
);

I have this sample Java Code:
import java.io.*;
import java.net.*;
import java.nio.charset.StandardCharsets;
public class TestClass {
public static void main(String[] args) throws IOException {
ServerSocket socket = new ServerSocket(8081);
Socket accept = socket.accept();
InputStream inputStream = accept.getInputStream();
InputStreamReader inputStreamReader = new InputStreamReader(inputStream, StandardCharsets.UTF_8);
char readChar;
while ((readChar = (char) inputStreamReader.read()) != -1) {
System.out.print(readChar);
}
inputStream.close();
accept.close();
System.exit(1);
}
}
and I have this test.html file:
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>File Upload!</title>
</head>
<body>
<form method="post" action="http://localhost:8081" enctype="multipart/form-data">
<input type="file" name="file" id="file">
<input type="submit">
</form>
</body>
</html>
and finally the file I will be using for testing purposes, named a.dat has the following content:
0x39 0x69 0x65
if you interpret the bytes above as ASCII or UTF-8 characters, they will actually will be representing:
9ie
So let 's run our Java Code, open up test.html in our favorite browser, upload a.dat and submit the form and see what our server receives:
POST / HTTP/1.1
Host: localhost:8081
Connection: keep-alive
Content-Length: 196
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Origin: null
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary06f6g54NVbSieT6y
DNT: 1
Accept-Encoding: gzip, deflate
Accept-Language: en,en-US;q=0.8,tr;q=0.6
Cookie: JSESSIONID=27D0A0637A0449CF65B3CB20F40048AF
------WebKitFormBoundary06f6g54NVbSieT6y
Content-Disposition: form-data; name="file"; filename="a.dat"
Content-Type: application/octet-stream
9ie
------WebKitFormBoundary06f6g54NVbSieT6y--
Well I am not surprised to see the characters 9ie because we told Java to print them treating them as UTF-8 characters. You may as well choose to read them as raw bytes..
Cookie: JSESSIONID=27D0A0637A0449CF65B3CB20F40048AF
is actually the last HTTP Header here. After that comes the HTTP Body, where meta and contents of the file we uploaded actually can be seen.

An HTTP message may have a body of data sent after the header lines. In a response, this is where the requested resource is returned to the client (the most common use of the message body), or perhaps explanatory text if there's an error. In a request, this is where user-entered data or uploaded files are sent to the server.
http://www.tutorialspoint.com/http/http_messages.htm

Composing multipart/form-data with a different Content-Type on each parts with Javascript (or Angular)

Wrong question asked, see my update below
I need to integrate my AngularJS Project with an existing RESTful API. These API consume POST request which upload a file, and also submit the form data in a request. Unfortunately, one of the form input requires to be in Content-Type: Application/json.
After search on the web, I could only POST with Content-Type: multipart/form-data in which each of the parts does not have a specific MIME.
How can I compose my multipart/form-data with a different MIME for each parts in Javascript?
POST /api/v1/inventory
Host: localhost:8000
Origin: http://localhost:9000
Content-Type: multipart/form-data; boundary=------border
------border
Content-Disposition: form-data; name="owner"
john doe
------border
Content-Disposition: form-data; name="image"; filename="mybook.png"
Content-Type: image/png
------border
Content-Disposition: form-data; name="items"
Content-Type: application/json
{"name": "Book", "quantity": "12"}
------border--
Relevant References:
https://developer.mozilla.org/en-US/docs/Web/Guide/Using_FormData_Objects
REST - HTTP Post Multipart with JSON
http://code.activestate.com/recipes/578846-composing-a-postable-http-request-with-multipartfo/
application/x-www-form-urlencoded or multipart/form-data?
https://stackoverflow.com/a/9082243/764592
Update
Apologize for asking a wrong question. The original problem is that, I can see the server calling the logic something like,
func POST(req):
owner = req.owner // This is string
image = req.image // This is file object
itemQuantity = req.items.quantity // Items is an object with attribute quantity
itemName = req.items.name // Items is an object with attribute name
I have also managed to figure out how to submit such a post request. I will post my answer below.
Once again sorry for asking a wrong question.

According to the documentation of FormData, you can append a field with a specific content type by using the Blob constructor:
var formData = new FormData();
formData.append('items', new Blob([JSON.stringify({
name: "Book",
quantity: "12"
})], {
type: "application/json"
}));
After careful observation, it turns out that it will send the part as follows:
Content-Disposition: form-data; name="items"; filename="blob"
Content-Type: text/json
The only alternative, safe from building the whole request yourself is to pass a string value:
formData.append('items', '{"name": "Book", "quantity": "12"}');
This, unfortunately, doesn't set the Content-Type header.

Mistake #1: I mistakenly assume that the items has to be a json, so that we can call its attribute.
Solution: To submit a multipart request that contain a file and an object like format is very simple.
form = new FormData();
form.append('items[name]', 'Book');
form.append('items[quantity]', 12);
form.append('image', imageFile);
form.append('owner', 'John Doe');
So thus the request header and body will looks something like this
POST /api/v1/inventory
Host: localhost:8000
Origin: http://localhost:9000
Content-Type: multipart/form-data; boundary=------border
------border
Content-Disposition: form-data; name="owner"
john doe
------border
Content-Disposition: form-data; name="image"; filename="mybook.png"
Content-Type: image/png
------border
Content-Disposition: form-data; name="items[name]"
Book
------border
Content-Disposition: form-data; name="items[quantity]"
12
------border--

Nothing would get this to work, until I set the Content-Type header to undefined. In my case I am posting a file and some json.
public uploadFile(code: string, file):angular.IHttpPromise<any>{
var data = `{"query":"mutation FIRMSCORECARD_CALCULATE($code:String!){ FirmScorecardMutation{ BatchCalculate(Code:$code) }}","variables":{"code":"${code}"},"operationName":"FIRMSCORECARD_CALCULATE"}`;
var formData = new FormData();
formData.append('operations', data);
formData.append('file', file, file.name);
let config = {
headers: {
'Accept': 'application/json',
'Content-Type': undefined
}
};
let response = this.$http.post(this.graphqlUrl, formData, config);
return response;
}

How to get final destination URL from AJAX?

When I do an XMLHttpRequest, I always get redirected automatically to the URL (presumably by the headers of the response). For example, if I query "http://www.stackoverflow.com" I will be redirected to "http://stackoverflow.com".
How can I get that final URL? (http://stackoverflow.com/ in the example)
I checked in the response headers but I cannot seem to find it. (I just used the GET/POST method not HEAD).

Look for a location header in the response.
In the example you gave, accessing www.stackoverflow.com and being redirected to stackoverflow.com here is most definatly a Location header being used.
[trcjr#rigel ~]$ curl -I http://www.stackoverflow.com
HTTP/1.1 301 Moved Permanently
Content-Length: 148
Content-Type: text/html; charset=UTF-8
Location: http://stackoverflow.com/
Date: Sat, 05 Feb 2011 21:47:17 GMT
[trcjr#rigel ~]$

Using jQuery (this is for current page url) :
$(document).ready(function () {
var href = window.location.href.toString();
});
EDIT : For, response page final url, pass jqXHR to ajaxComplete & then read the header.