I am using LZW algorithm for stringified JSON object to reduce size. Size reduction is fantastic, but I am having problems transmitting compressed string via AJAX because of URL malformed error. If I encode it with base64, the size goes up enormously.
What are the practices to send LZW compressed data to server ?
Related
I'm trying to implement NgOptimizedImage in my angular app.
<img [ngSrc]="imageUrl" width="196" height="196">
imageUrl is url to my server. The response is buffer. I used to convert the buffer to string in component.ts file and then using [src]="convertedBuffer" show the image. But NgOptimizedImage requires src, so it loads from the server when the image can be seen.
Do I have to convert the buffer to string on server, or is there a way to convert it to string on client side?
There is no way to do that. HTTP doesn't have native support for sending base64-encoded data, so the browser won't be able to decipher your image once it is requested.
Base64 is only supported in data URIs. It seems that you have to convert your server to send non-base64-encoded images or go back to your old way of displaying images.
The problem is on server side.
You have to send buffer and set "Content-Type": "image/TYPE" header.
If you're using express and nodejs here's how you can do it:
res.contentType('image/jpeg'); // set Content-Type header
res.send(buffer); // send buffer. make sure it is buffer (buffer instanceof Buffer)
I want to send a BLOB to an APEX RESTful service. The payload has to be a JSON (not form-data, I've had a lor of problems with that in the server side).
The BLOB is an image that I need to upload to the DB, my problem is that I don't know how to send the data in the payload. Should it be a string that represents the array of bytes? Should it be the array itself? (I managed to extract a binary string with the readASBinaryString method of javascipt's FileReader).
Thanks for any help.
Your BLOB data could be encoded to Base64 and then transported in JSON document as a string. You can use window.btoa() to convert binary data to Base64 string.
For example:
{
"data": "SGVsbG8gd29ybGQ="
}
I am creating a web API with Node.js and Express and intend to use a single response to return an image and JSON data simultaneously. I saw a solution to specify headers: https://stackoverflow.com/a/18864738/1703565 but I do not want to perform two get requests. How do I accomplish this?
You could encode the image as a base64 string (http://nodejs.org/api/buffer.html#buffer_buf_tostring_encoding_start_end), and return this as part of your JSON.
Cookies
You could return the image as a normal image response body and then set a session cookie with the same request. The session cookie would contain the JSON data. Once the image loads you can pull the JSON data from the cookie via javascript.
This would work in all browsers. Only limitation would be the size of the JSON you could return. Max size looks to be 4093 bytes.
Image Encoding
If your JSON is too big to fit in a cookie then you could encode the image as a base64 object and return the image data in the JSON response.
In this case you would have to reconstruct the image on a canvas or use the image data url format to dynamically create an image object as the url in the comment from Kevin does.
One downside to image encoding would be the increased size of the response. Roughly, the size of the image after encoding in base64 would be 37% larger than the original image.
Ultimately it depends on your specific needs as to which method would best fit your requirements.
the client will be sending my server a change log, containing a list of commands and parameters, JSON or not is TBD.
This payload can be a 3 or 4K not likely to be more.
What is the standard approach to deal with requirement?
Client should send a json, containing all of the changes, as part of the request body?
Any recommendations? Lessons learned?
Just POST the data. 3-4 KB is nothing unless you're dealing with feature-phone WAP browsers in the middle of rural India, performance issues of the "OMG, I'm Google and care about every byte ever because of my zillion-user userbase" type, or something like that.
If you're really worried about payload size, you can gzip-base64 encode it before sending - but only do this if a) you really care about this (which is unlikely) and b) your payload is large enough that this saves you bandwidth. (gzip-base64'ing small payloads often increases their size, since there isn't enough data to get enough compression benefit to offset the 33% size increase from base64 encoding.)
You can use a normal JSON post to send across 3/4K of data.
You should pay more attention to what you do with the data received on the server side, whether you buffer up all data before you start processing them (store in db or elsewhere), or process them in chunks. If you are simply dumping the data into files on server, you should create a Writable stream and pump chunks of data received into the stream.
How are you going to process the received data on the server? But then, 3/4K is not really worrying amount of data.
You can set the maximun upload size with
app.use(express.limit('5mb'));
if that's an issue?
But, there shouldn't really be any limitations on this as default, except the max buffer size (which I believe is 1GB).
It also sounds like this is something that you can just post to the server with a regular POST request, in other words you use a form with a file input and just upload the file the regular way, as 4kb isn't really a big file.
I am posting a large amount of data. I need this to be as small as possible for performance reasons.
My data starts out as JS objects. I then stringify it using json. I then send it in a post.
The thing is that I have a lot of objects:lists [] and dict {}, as well as short texts, which are placed in quotes "" by json.
These are then uri encoded before being posted. I do not do this; the browser does it. I can see the result when I look in the request body.
So, every [, {, and "" is now uri encoded, meaning that my string becomes much longer. In fact, if I compare
alert( JSON_local.stringify(myStuff).length);
alert(encodeURI(JSON_local.stringify(myStuff).length);
the uri encoded string is 50% larger. That's a lot bigger when the string starts out big.
Am I missing something here? json is standard, but it seems to have a negative side effect for me. is there an alternative to using json? Or am I doing something wrong here? Data alsways has to be send as uri encoded, no?
Data always has to be send as uri encoded, no?
Not true. This depends on the content type you're sending it.
If you use x-www-form-urlencoded content-type when sending it, you need to encode the data. If you use multipart/form-data, for example, you do not need to. This has been discussed in more length in here. For considerable amount of data, I don't see any real reason to use x-www-form-urlencoded.
Of course, there is more to it than just changing the content type, you need to supply the mime boundaries then. It does sound to me however that that'd be more efficient for you. From http://www.w3.org/TR/html401/interact/forms.html#h-17.13.4:
The content type "application/x-www-form-urlencoded" is inefficient
for sending large quantities of binary data or text containing
non-ASCII characters. The content type "multipart/form-data" should be
used for submitting forms that contain files, non-ASCII data, and
binary data.