Transferring strings containing emotions to server - javascript

On my website whenever a user enters a mobile emoji like šŸ˜€ into an input field it will be saved as ?? in my database.
Those emojis are encoded in utf8mb4, so I already updated my database collation to utf8mb4_general_ci.
While the emoticons can be saved successfully now when transfering the message containing a emoji from a client to my server, it still get's somewhere changed into ?? and I am now trying to figure out where and how to solve it.
Sending the message to my server happens in this ajax call:
function updateStatus() {
var status = $("#status").val();
jsRoutes.controllers.Userdata.updateStatus( status ).ajax({
success : function(data) {
$("#profil").cftoaster({content: data});
},
error : function(err) {
$("#profil").cftoaster({content: err.responseText});
}
});
}
On serverside I am using java based Play Framework 2.4.4.
This is the beginning of the method which is called in the ajax call:
public static Result updateStatus(String status) {
String receivedName = Application.getSessionUser();
Logger.debug("RecvStatus: " + status);
...
}
The Logger output already is ?? for an emoticon.
The route looks like this:
PUT /status/ controllers.Userdata.updateStatus(status: String)
EDIT:
To make sure the transfer from client to server is alright I am now transferring the actual unicode values, I change my server function like this
Logger.debug("RecvStatus: " + status);
status = status.replace("\\","");
String[] arr = status.split("u");
status = "";
for(int i = 1; i < arr.length; i++){
int hexVal = Integer.parseInt(arr[i], 16);
status += (char)hexVal;
}
Logger.debug("RecvStatus: " + status);
and get the following output:
[debug] application - RecvStatus: \ud83d\ude01
[debug] application - RecvStatus: ?
which means the problem is probably with java

So for anyone having a similiar problem here my workaround.
First I tried to convert the String in java to base64 and store it this way. Unfortunately after decoding the base64 the unicode information still got lost and ?? was shown instead of emoticons.
What I than did was converting the received String into unicode and than into base64, and when I load the Strings from the database I first decode base64 and then convert the unicode information into an actual String. This way the emoticons are stored and afterwards shown correctly.

Related

Swift 5 - How to convert LONGBLOB/Buffer into Data

I am currently working on a project for school.
I have written an API using Express connected to a mysql database. And now I am writing the iOS app.
My problem is that I need to save profile pictures. So I saved the png data of the picture into a **LONGBLOB** into db and I want to recreate the image into a **UIImage**.
To do that I am trying to convert the buffer into ```Data```
So, the API is returning a buffer created that way:
let buffer = Buffer.from(ppData.data, 'binary').toString('base64');
And on the iOS side I tried:
guard let data = dict["data"] as? Data else {return nil}
Where dict["data"] is the buffer returned by the API.
But it always enter into the "else" part.
What am i doing wrong
Edit:
As what it was said in comments, I decoded the Base64 encoded string. Now the data are decoded but creating a UIImage from it, fails, without any details. What I tried is:
let image = UIImage(from: base64DecodedData)
For example:
guard let strData = dict["data"] as? String else {
return nil
}
guard let data = Data(base64Encoded: strData, options: .ignoreUnknownCharacters) else {
return nil
}
guard let picture = UIImage(data: data) else {
return nil
}
Thanks.
The mistake was not in the swift code part but in my API and database structure. After reading some MySQL and Node.js documentaion, I switched from LONGBLOB (which is totally oversized) to MEDIUMTEXT.
Also, in the API I was trying to create a buffer from binary data but not from a base64 string encoded data, so I removed this line:
let buffer = Buffer.from(ppData.data, 'binary').toString('base64');

Convert TCP Data in with pipe to JSON

Sorry if the issue is not particularly clear (I am still a novice). I have a simple setup to get data from a mock feed and then convert the data to JSON. I can retrieve the data and display it but converting it to JSON has proved a bit tricky.
var completeData = '';
let client = net.createConnection({ port: 8282 }, () => {
client.write('Test Worked!\r\n');
});
client.on('data', (data) => {
// change buffer to string
let readData = data.toString();
// Merge response data as one string
completeData += readData += '\n';
// End client after server's final response
client.end();
});
This is the sample output for one packet below (Pipe delimited with some data escaped):
|2054|create|event|1497359166352|ee4d2439-e1c5-4cb7-98ad-9879b2fd84c2|Football|Sky Bet League Two|\|Accrington\| vs \|Cambridge\||1497359216693|0|1|
I would like the pipes to represent keys/values in an object. The issue is that some of the values are escaped (e.g '\|' ). That kind of makes using the split function in Javascript difficult.
My question is there a way to get pipe delimited data from TCP packets and then convert them to a JSON object?
Any help will be highly appreciated. Thank you.
const { StringDecoder } = require("string_decoder");
let header = "";
const decoder = new StringDecoder("utf8");
let str = decoder.write(chunk); // chunk is your data stream
header += str;
header.replace(/s/, "");
var obj = JSON.parse(header);
console.log(obj);
While receiving data through tcp ip net npm package, we face trouble in parsing the continous stream of data. string_decoder is a npm package which can parse it into a json readable format. Also since the packets being received has a variable length so one can basically put a check on header.length to fetch only a complete set of object.

SHA1 varies in java and javascript for same input

Facing an issue while creating SHA1 from javascript and java. The problem is both are different. It is used for validating the client request to web server. That means client send a based64 encoded security key to server and server regenerate the same key and equate both are same. Please find below code for generating secret keys in client and server.
Server
MessageDigest mDigest = null;
try {
mDigest = MessageDigest.getInstance("SHA1");
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
}
String input = value1 + value1 + server_key;
byte[] result = mDigest.digest(input.getBytes());
String secret = Base64.encodeToString(result, Base64.NO_WRAP);
...
//comparison logic goes here
...
Client (java script)
var input = value1 + value2 + server_key;
//http://code.google.com/p/crypto-js/
var hash = CryptoJS.SHA1(input);
var encoded = base64Encode(hash.toString());
//WEB SERVICE INVOCATION FROM JAVASCRIPT GIES HERE.
The values value1, value1, server_key will be available in both client and server. The issue we are facing is, the SHA1 generated in both client and server is not matching. I understand the issue is in java its using getBytes() and in javascript using string value for generating SHA1. The CryptoJS.SHA1 does not support bytearray as parameter. We cannot change the server code as it is used by many client applications. Any help will be much appreciated.
In Java ->
byte[] result = mDigest.digest(input.getBytes());
and in JavaScript ->
var hash = CryptoJS.SHA1(input);.
I belief this is the problem. In java the parameter is a bytearray and output is also a bytearray. But in javascript the parameter is var (string) and return is also var (string). I 've also compared the output of CryptoJS.SHA1 with some online SHA1 generating tools. The comparison is true. I am not an expert in this area. If you can explain more, it will be more helpful.
I managed it to do in another way. My application is a cordova based application. So generated the sha1 and encoded it from java and objC and invoked it using cordova plugins.

I have an external json file ,say abc.json. in that it contain { "remember me": "AcuƩrdate " }

$.get("resources-locale_fr.json", function(data) {
for (var key in data) {
if (key == data1) {
console.log("sijilal" + data[key]);
$scope.englishText = data[key];
$scope.$digest();
}
}
});
example if i enter ["remember me"] i will get "AcuƩrdate ".
My problem is i need to edit this " "AcuƩrdate "" to "AcuƩrdate123 " and need to save it to my external json abc.json.am using localhost.answers in angularjs is much appreciated.
Here data1 is the key name i enetered and i will get its corresponding value.
It will retrieve the correct word (value) for the key i entered,but problem is i need to edit the value name and need to save that to external json file.
Is there any way to save value to corresponding pair in external json
To save the data you need to do two things:
On the client you need to send the changed data back to the server. This will typically be done with a POST request.
For example:
$.post("resources-locale_fr.fcgi", { "remember me", updated_variable });
On the server you will need some server side code, written in the language of your choice, that accepts the POST request and writes the data to the file (usually after performing some authentication/authorization check).
For this sort of application you would usually replace the static JSON file with a database and then generate JSON on demand with a server side program.

Gmail API - Parse message content (Base64 decoding?) with Javascript

I'm trying to use the Gmail API to get a user's email, grab the message subject and body, and then display it on a webpage. I'll be doing other stuff with it, but this is the part that I am having difficulty with. I am using Angular.js.
Here is my API call:
function makeApiCall() {
gapi.client.load('gmail', 'v1', function() {
var request = gapi.client.gmail.users.messages.list({
labelIds: ['INBOX']
});
request.execute(function(resp) {
var content = document.getElementById("message-list");
angular.forEach(resp, function(message) {
var email = gapi.client.gmail.users.messages.get({'id': message.id});
// var raw = email.payload.parts;
// console.log(raw);
content.innerHTML += JSON.stringify(email) + "<br>";
})
});
});
}
So gapi.client.gmail.users.messages.list returns an array of my messages, with their ID numbers. That is working.
The call to gapi.client.gmail.users.messages.get({<specific message ID>}) outputs this - {"B":{"method":"gmail.users.messages.get","rpcParams":{},"transport":{"name":"googleapis"}}}.
Not sure what that is, but trying to get the message payload (email.payload.parts), results in undefined. So, how can I get the message content?
Also, I would assume that if I can get the message contents, I would then have to Base64 decode the contents to get some English out of it. Any suggestions for that would be of great help also. I've found this: https://github.com/kvz/phpjs, but since I'm not sure how to go about getting the message contents so that I can try and decode them, so not sure if that php.js is of an help in that regard.
Regarding the Base64 decoding, you can use
atob(dataToDecode)
For Gmail, you'll also want to replace some characters:
atob( dataToDecode.replace(/-/g, '+').replace(/_/g, '/') );
The above function is available to you in JavaScript (see ref). I use it myself to decode the Gmail messages. No need to install extra stuff. As an interesting tangent, if you want to encode your message to Base64, use btoa.
Now, for accessing your message payload, you can write a function:
var extractField = function(json, fieldName) {
return json.payload.headers.filter(function(header) {
return header.name === fieldName;
})[0].value;
};
var date = extractField(response, "Date");
var subject = extractField(response, "Subject");
referenced from my previous SO Question and
var part = message.parts.filter(function(part) {
return part.mimeType == 'text/html';
});
var html = atob(part.body.data.replace(/-/g, '+').replace(/_/g, '/'));
Depending on what your emails look like (single text/plain part? multipart with text/html? attachments, etc?) you may or may not have any "parts" in your email.payload and instead you'll have what you're looking for in "email.payload.body.data" (for single-part messages). This is all assuming you're doing a message.get with the default format ("full"). If you instead want to get the entire email in the message.raw field and deal with it in email libraries for your language you can call message.get(format=raw).
For more info check out the "body" and "parts[]" field documentation for "Message" at https://developers.google.com/gmail/api/v1/reference/users/messages
Ah! I figured it out. parts is an array, so I should have been calling it like: gapi.client.gmail.users.messages.get({'id': <message ID>}).payload.parts[0].body.data
Now my problem is decoding the emails, which is proving successful in plain text emails, but failing in emails from non-personal locations (businesses, social media update emails, etc.). But I'll make a new question to get answers for that.
You need to search where the body for a given mime type is, I have written a recursive function for that:
function searchBodyRec(payload, mimeType){
if (payload.body && payload.body.size && payload.mimeType === mimeType) {
return payload.body.data;
} else if (payload.parts && payload.parts.length) {
return payload.parts.flatMap(function(part){
return searchBodyRec(part, mimeType);
}).filter(function(body){
return body;
});
}
}
So now you can call
var encodedBody = searchBodyRec(this.message.payload, 'text/plain');
See the flatMap method up there? Classic FP method missing in js, here is how to add it (or you can use lodash.js, or underscore.js if you don't want to mess with the native objects)
Array.prototype.flatMap = function(lambda) {
return Array.prototype.concat.apply([], this.map(lambda));
};

Categories

Resources