I was developing python program with Eel module.
But there is problem. It's that return null from eel.getImgSrc(path) function to get image's byte data.
Look at this code.
-----web/main.js------
async function run(path) {
let n = await eel.getImgSrc(path)() //path is correct. but n is null.
console.log(n);
}
-----app.py------
#eel.expose
def getImgSrc(path):
f = open(path, 'rb')
return f.read()
In Eel, the Python and JavaScript layers communicate by passing JSON back-and-forth. Internally, the Python side is passing your Python return value through a function that's defined like this: jsn.dumps(obj, default=lambda o: None) which results in the string null in your case. You can confirm this behavior by updating your code for debugging purposes only to something like this:
#eel.expose
def getImgSrc(path):
f = open(path, 'rb')
data = f.read()
print(json.dumps(data, default=lambda o: None))
return data
You'll see it prints out null. That's why you're seeing null on the JavaScript side.
I'm not a developer on the Eel project, so I can't explain the rationale of why this happens (I just see it in the source code). But now that you know it's happening, you can solve the problem by finding a way to encode your data so that it can be passed via JSON. (And you can use the sample code I posted above to check that it gets encoded properly.)
A search on the web for how to encode binary data into JSON with Python and decode with JavaScript will yield many results... so pick one that suits your particular needs. One way that's common (and is discussed in an issue on the Eel project website), is to base64 encode the data on the Python side and then decode it on the JavaScript side. It's not ideal, but is a workable solution that I've used in the past. Your code would be something like this...
-----web/main.js------
async function run(path) {
let n = await eel.getImgSrc(path)() // should no longer be null
console.log(atob(n));
}
-----app.py------
#eel.expose
def getImgSrc(path):
f = open(path, 'rb')
data = f.read()
data = base64.b64encode(data).decode("utf-8")
return data
Additionally, HTML image tags can accept base64 encoded data, so you could do something like:
async function run(path) {
let n = await eel.getImgSrc(path)() // should no longer be null
let imageEl = document.getElementById('image');
imageEl.src = "data:image/jpeg;base64," + n;
// any other processing...
}
You will probably need to tweak these examples to fit your exact needs.
Related
I am writing DNS-over-HTTPS server which should resolve custom names, not just proxy them to some other DoH server, like Google's. I am having trouble properly decoding the body of the request.
For example, I get body of request, that is in binary format, specifically in javascript in Uint8 ArrayBuffer type. I am using the following code to get base64 format of the array:
function _arrayBufferToBase64(buffer) {
var binary = '';
var bytes = new Uint8Array(buffer);
var len = bytes.byteLength;
for (var i = 0; i < len; i++) {
binary += String.fromCharCode(bytes[i]);
}
return btoa(binary);
}
And I get something like this as a result:
AAABAAABAAAAAAABCmFwbngtbWF0Y2gGZG90b21pA2NvbQAAAQABAAApEAAAAAAAAE4ADABKAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
Now, per RCF8484 standard this should be decoded as base64url, but when I decode it as such, I get the following:
apnx-matchdotomicom)N J
I also used this "tutorial" as the reference, but they also decode similarly formatted blob and I get similar nonsense like previously.
There is very little to no information about something like this on the internet and if it is of any help DoH standard uses application/dns-message media type for the body.
If anyone has some insight on what I am doing wrong or how I could edit the question to make it more clear, please help me, cheers :)
As stated in the RFC:
Definition of the "application/dns-message" Media Type
The data payload for the "application/dns-message" media type is a
single message of the DNS on-the-wire format defined in Section 4.2.1
of [RFC1035], which in turn refers to the full wire format defined in
Section 4.1 of that RFC.
So what you get is exactly what is sent on the wire in the normal DNS over 53 case.
I would recommend you use a DNS library that should have a from_wire or similar method to which you can feed this content and get back some structured data.
Showing an example in Python with the content you gave:
In [1]: import base64
In [3]: import dns.message
In [5]: payload = 'AAABAAABAAAAAAABCmFwbngtbWF0Y2gGZG90b21pA2NvbQAAAQABAAApEAAAAAAAAE4ADABKAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
In [7]: raw = base64.b64decode(payload)
In [9]: msg = dns.message.from_wire(raw)
In [10]: print msg
id 0
opcode QUERY
rcode NOERROR
flags RD
edns 0
payload 4096
option Generic 12
;QUESTION
apnx-match.dotomi.com. IN A
;ANSWER
;AUTHORITY
;ADDITIONAL
So your message is a DNS query for the A record type on name apnx-match.dotomi.com.
Also about:
I am writing DNS-over-HTTPS server which should resolve custom names,
If you don't do that to learn (which is a fine goal), note that there are already various open source nameservers software that do DOH so you don't need to reinvent it. For example: https://blog.nlnetlabs.nl/dns-over-https-in-unbound/
I am currently working on a project for school.
I have written an API using Express connected to a mysql database. And now I am writing the iOS app.
My problem is that I need to save profile pictures. So I saved the png data of the picture into a **LONGBLOB** into db and I want to recreate the image into a **UIImage**.
To do that I am trying to convert the buffer into ```Data```
So, the API is returning a buffer created that way:
let buffer = Buffer.from(ppData.data, 'binary').toString('base64');
And on the iOS side I tried:
guard let data = dict["data"] as? Data else {return nil}
Where dict["data"] is the buffer returned by the API.
But it always enter into the "else" part.
What am i doing wrong
Edit:
As what it was said in comments, I decoded the Base64 encoded string. Now the data are decoded but creating a UIImage from it, fails, without any details. What I tried is:
let image = UIImage(from: base64DecodedData)
For example:
guard let strData = dict["data"] as? String else {
return nil
}
guard let data = Data(base64Encoded: strData, options: .ignoreUnknownCharacters) else {
return nil
}
guard let picture = UIImage(data: data) else {
return nil
}
Thanks.
The mistake was not in the swift code part but in my API and database structure. After reading some MySQL and Node.js documentaion, I switched from LONGBLOB (which is totally oversized) to MEDIUMTEXT.
Also, in the API I was trying to create a buffer from binary data but not from a base64 string encoded data, so I removed this line:
let buffer = Buffer.from(ppData.data, 'binary').toString('base64');
I am trying to run what I expect is a very common use case:
I need to download a gzip file (of complex JSON datasets) from Amazon S3, and decompress(gunzip) it in Javascript. I have everything working correctly except the final 'inflate' step.
I am using Amazon Gateway, and have confirmed that the Gateway is properly transferring the compressed file (used Curl and 7-zip to verify the resulting data is coming out of the API). Unfortunately, when I try to inflate the data in Javascript with Pako, I am getting errors.
Here is my code (note: response.data is the binary data transferred from AWS):
apigClient.dataGet(params, {}, {})
.then( (response) => {
console.log(response); //shows response including header and data
const result = pako.inflate(new Uint8Array(response.data), { to: 'string' });
// ERROR HERE: 'buffer error'
}).catch ( (itemGetError) => {
console.log(itemGetError);
});
Also tried a version to do it splitting the binary data input into an array by adding the following before the inflate:
const charData = response.data.split('').map(function(x){return x.charCodeAt(0); });
const binData = new Uint8Array(charData);
const result = pako.inflate(binData, { to: 'string' });
//ERROR: incorrect header check
I suspect I have some sort of issue with the encoding of the data and I am not getting it into the proper format for Uint8Array to be meaningful.
Can anyone point me in the right direction to get this working?
For clarity:
As the code above is listed, I get a buffer error. If I drop the Uint8Array, and just try to process 'result.data' I get the error: 'incorrect header check', which is what makes me suspect that it is the encoding/format of my data which is the issue.
The original file was compressed in Java using GZIPOutputStream with
UTF-8 and then stored as a static file (i.e. randomname.gz).
The file is transferred through the AWS Gateway as binary, so it is
exactly the same coming out as the original file, so 'curl --output
filename.gz {URLtoS3Gateway}' === downloaded file from S3.
I had the same basic issue when I used the gateway to encode the binary data as 'base64', but did not try a whole lot around that effort, as it seems easier to work with the "real" binary data than to add the base64 encode/decode in the middle. If that is a needed step, I can add it back in.
I have also tried some of the example processing found halfway through this issue: https://github.com/nodeca/pako/issues/15, but that didn't help (I might be misunderstanding the binary format v. array v base64).
I was able to figure out my own problem. It was related to the format of the data being read in by Javascript (either Javascript itself or the Angular HttpClient implementation). I was reading in a "binary" format, but it was not the same as that recognized/used by pako. When I read the data in as base64, and then converted to binary with 'atob', I was able to get it working. Here is what I actually have implemented (starting at fetching from the S3 file storage).
1) Build AWS API Gateway that will read a previously stored *.gz file from S3.
Create a standard "get" API request to S3 that supports binary.
(http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings-configure-with-console.html)
Make sure the Gateway will recognize the input type by setting 'Binary types' (application/gzip worked for me, but others like application/binary-octet and image/png should work for other types of files besides *.gz). NOTE: that setting is under the main API selections list on the left of the API config screen.
Set the 'Content Handling' to "Convert to text(if needed)" by selecting the API Method/{GET} -> Integration Request Box and updating the 'Content Handling' item. (NOTE: the example in the link above recommends "passthrough". DON'T use that as it will pass the unreadable binary format.) This is the step that actually converts from binary to base64.
At this point you should be able to download a base64 verion of your binary file via the URL (test in browser or with Curl).
2) I then had the API Gateway generate the SDK and used the respective apiGClient.{get} call.
3) Within the call, translate the base64->binary->Uint8 and then decompress/inflate it. My code for that:
apigClient.myDataGet(params, {}, {})
.then( (response) => {
// HttpClient result is in response.data
// convert the incoming base64 -> binary
const strData = atob(response.data);
// split it into an array rather than a "string"
const charData = strData.split('').map(function(x){return x.charCodeAt(0); });
// convert to binary
const binData = new Uint8Array(charData);
// inflate
const result = pako.inflate(binData, { to: 'string' });
console.log(result);
}).catch ( (itemGetError) => {
console.log(itemGetError);
});
}
I'm trying to create hash for calling thesubdb api they've given the python implementation of their hash generation method.
My project needs me to create hash on the go on browser and then pass it to my php backend and then call subdbapi using php.
But for that first of all I need to create hash of video file on my browser using javascript.
I'm using the new html 5 file api to read local file on users computer and create hash for same.
var md5 = CryptoJS.MD5(binary).toString();
console.log(md5);
above is my javascript code.
This does not match the output of the python code that they've given which is :
def get_hash(name):
readsize = 64 * 1024
with open(name, 'rb') as f:
size = os.path.getsize(name)
data = f.read(readsize)
f.seek(-readsize, os.SEEK_END)
data += f.read(readsize)
return hashlib.md5(data).hexdigest()
i'm sure i'm missing something fundamental in my javascript code.
thanks in advance.
I'm using Python websockets to return JSON objects (I think) however I'm not sure how to deal with the response. I have an example using JavaScript which uses parseJSON, a snippet is bellow:
socket = io.connect("__socket address__");
socket.on("connect", function() {socket.emit("subscribe_areas", areas)});
var d = jQuery.parseJSON(c);
console.log(d);
d.type == "INTERPOSE";
which returns data I can access eg d.type == "INTERPOSE";:
Object {area: "BE", type: "MOVE", to: "0109", from: "0107", value: "2L65"}
Object {area: "BE", type: "MOVE", to: "0113", from: "0109", value: "2L65"}
where as my Python I recieve:
(u'{"area":"EH","type":"INTERPOSE","to":"B243","value":""}',)
(u'{"area":"EH","type":"INTERPOSE","to":"0337","value":""}',)
(u'{"area":"EH","type":"INTERPOSE","to":"0085","value":""}',)
Why does the python return JSON wrapped in brackets and commars? My Python is bellow:
from socketIO_client import SocketIO
import logging
import json
def on_connect():
print('CONNECTED')
areas = ['EH']
socketIO.emit('subscribe_areas', areas)
def on_message(*answer):
print(answer)
socketIO = SocketIO('_address_', _port_)
socketIO.on('connect', on_connect)
socketIO.on('message', on_message)
socketIO.wait()
I've tried decoding using json.load, json.loads, json.decode but recieve all sorts of errors. What am I missing and why is the return wrapped in brackets and such?
You told Python to expect 0 or more arguments:
def on_message(*answer):
print(answer)
Python then gives you a tuple of all arguments passed in; your print statements show you got one argument, so you got a tuple with just one element in it. That one element is a JSON string that you'd decode with json.loads().
Take the first element and decode that:
def on_message(*answer):
print(json.loads(answer[0]))
or don't use * and require that one argument is passed in, always and load from that one argument
def on_message(answer):
print(json.loads(answer))