Develop Reference

Bug when converting image buffer back to base64

I am getting a strange bug where, when I convert a base64 string to an image buffer (Buffer.from(image, "base64")) and back to base64 (.toString("base64")) the resulting base64 string is missing its formatting (dataimage/pngbase64 instead of data:image/png;base64) as well as missing g== at the end. This results in the image being "corrupt" and not rendering when I put it in an <img /> in the frontend. The current workaround I'm using is the following:
image.toString("base64").replace("dataimage/pngbase64", "data:image/png;base64,") + "g=="
but this is far from an optimal solution and I would like to not use this sort of workaround.
This is where I bufferize the image (image is base64) and store it in the database
t.field("createModel", {
type: $name,
args: { input: nonNull(arg({ type: createModelInput.name })) },
resolve: async (_, args) => {
const { image, name, manufacturerId, identifiers } = args.input;
console.log(image) // correct base64 image from frontend
const buffedImage = Buffer.from(image, "base64");
console.log(buffedImage.toString("base64")) // not the same as image above: missing formatting & g== at the end
return await prisma.model.create({
data: {
image: buffedImage,
name,
identifiers,
manufacturer: {
connect: {
id: manufacturerId,
},
},
},
});
},
});
Please tell me any further information is needed.

The Base64 part of the string doesn't start until after the ,; the data: part is a scheme, the image/png part is a content type, and the base64, part is an indicator that what follows it is Base64 encoded text. So you're asking to convert non-Base64 data when you try to use that entire string as Base64.
You have to remove that prefix first, because it's not part of the Base64 data. It's just part of the data: URI.

How get image from FormData to Rails API and saves in ActiveStorage

I am trying to send a FormData object from my React Native app to my Rails API.
I am using react-native-image-crop-picker to pick the image from gallery and store the image object in my state:
ImagePicker.openPicker({
multiple: true,
cropping: true,
includeBase64: true,
compressImageMaxHeight: 1080,
compressImageMaxWidth: 1080,
})
.then(selectedImages => {
selectedImages.map(image => {
let prefix;
let ext;
[prefix, ext] = image.filename.split(".");
ext = ext.toLowerCase() === "heic" ? "jpg" : ext;
const upload = {
uri: image.path,
type: image.mime,
name: `${prefix}.${ext}`,
};
this.setState(prevState => ({
event: {
...prevState.event,
images: [...prevState.event.images, upload],
},
}));
});
})
.catch(error => {
});
Then i create a FormData object to send yo my Rails API:
const data = new FormData();
data.append("name", event.name);
data.append("description", event.description);
data.append("date", event.date);
data.append("time", event.time);
data.append("images", event.images[0]);
My api successfully receives que request:
Parameters: {"event"=>{"_parts"=>[["event[name]", ""], ["event[description]", ""], ["event[date]", ""], ["event[time]", ""], ["event[images]", {"uri"=>"/private/var/mobile/Containers/Data/Application/6226B812-CDEC-4994-A864-0A91EE8C44B3/tmp/react-native-image-crop-picker/BFC043EC-D33F-4E07-BBEA-634CE5DE8A3F.jpg", "type"=>"image/jpeg", "name"=>"IMG_7142.jpg"}]]}}
Now, how can i recover this image and saves directly in my Rails ActiveStorage?
I am trying to directly attach the image object: {"uri"=>"/private/var/mobile/Containers/Data/Application/6226B812-CDEC-4994-A864-0A91EE8C44B3/tmp/react-native-image-crop-picker/BFC043EC-D33F-4E07-BBEA-634CE5DE8A3F.jpg", "type"=>"image/jpeg", "name"=>"IMG_7142.jpg"} to my model, but i am getting the current exception: ActiveRecord::RecordNotSaved (Failed to save the new associated avatar_attachment.):

I see that you have multiple images to attach.
If you've already gone through the documentation, I believe you must have already setup active storage by now.
In your model do you have has_many_attached :images? note that images could've been something else or just about any name.
I was able to retrieve the image name by doing the following (but I doubt if it will work for your purpose):
hash = {"uri"=>"/private/var/mobile/Containers/Data/Application/6226B812-CDEC-4994-A864-0A91EE8C44B3/tmp/react-native-image-crop-picker/BFC043EC-D33F-4E07-BBEA-634CE5DE8A3F.jpg", "type"=>"image/jpeg", "name"=>"IMG_7142.jpg"}
hash['uri'].split('/')[-1]
=> "BFC043EC-D33F-4E07-BBEA-634CE5DE8A3F.jpg"
If the above route is how you want to go and you are sure that it will work this way, then you need to do something like:
#your_model.images.attach(io: File.open("#{hash['uri']}"), filename: "#{hash['uri'].split('/')[-1]}")
You can find here a complete tutorial on how this should actually work here - https://medium.com/#ashley_foster/react-native-image-uploads-with-activestorage-ec4898317a82
Give it a try and if you get stuck along the way, you can drop a comment or edit/update your question.

Appending FormData nested object along with a file

I need to do a PATCH to the server which expects a very specific body format. Something like this:
{
file: MY_FILE_OBJECT
name: 'Name',
nestedObject: {
nestednestedObject1: {
name: 'Some other name'
},
nestednestedObject2: {
name: 'Yet another name'
},
}
}
Because I need to be able to pass in a file, I have to make this into a FormData object. So, for the main fields it is nice and easy to form the body of the request
const data = new FormData()
data.append('file', file, file.name)
data.append('name', 'Name')
However, how can I do the nested translation object? I have seen in another thread the suggestion of me doing a blob. Which should be something like this:
const fullPatchObject = {
nestedObject: {
nestednestedObject1: { name: 'Some other name' }
nestednestedObject2: { name: 'Yet another name' }
}
}
const blob = new Blob([JSON.stringify(fullPatchObject)], { type: 'application/json' })
data.append('nestedObject', blob)
This however does not work. I am using multer in the server to handle the file, and I get the following error object:
name: 'MulterError',
message: 'Unexpected field',
code: 'LIMIT_UNEXPECTED_FILE',
field: 'nestedObject'
Keep in mind that if I do not use the nested object, it all works as expected, for both the file and the fields I am attempting to PATCH. The issue seems to be that multer thinks my nestedObject is perhaps also a file? The thing is... Well... It isn't, I just want it to be passed in as field like the name field.
Am I missing something? What is a way I can send a nested object using the FormData?

Try using array-style names:
data.append("nestedObject[nestedNestedObject1][name]", "Some other name");
data.append("nestedObject[nestedNestedObject2][name]", "Yet another name");

set processor in $http PUT request not saving proper data

Relatively new to javascript and angular. I'm using an elasticsearch instance to ingest/index different files. The webapp should have the capability to allow the user to upload multiple files, which will then be processed using the ingest processor and indexed into ES. I'm using angular-base64-upload to extract the base64 data. The following is what I've tried so far.
The html:
<div ng-switch-when="upload">
...
<input type="file" ng-model="obj.files" name="files"
base-sixty-four-input multiple>
...
<div ng-show="obj.files.length != 0">
<button class="btn btn-primary" type="submit" ng-click="ingest()">
Index All {{obj.files.length}} Files
</button> <br><br>
</div>
</div>
The javascript ingest() function code in my controller:
//Ingesting multiple files
$scope.obj = {files: []};
$scope.ingest = function () {
$scope.obj.files.forEach(function (file){
var fname = file.filename.replace(/\.[^/.]+$/, "");
var fdata = file.base64;
//Creating the pipeline
console.log('filename is: ' + fname);
$http({
url: 'http://192.168.100.30:9200/_ingest/pipeline/attachment',
method: "PUT",
data: {
"description" : "Indexing files",
"processors" : [
{
"set" : {
"field" : "filename",
"value" : fname
},
"attachment" : {
"field" : "data"
}
}
]
}
})
.then(function(allowed){
//Indexing the document
$http({
url: 'http://192.168.100.30:9200/my_index4/my_type/'+fname+'?pipeline=attachment', //unique ID for every document, ensures that there are no overlaps
method: "PUT",
data: {
"data": fdata
}
})
})
})
}
The console logging is just used for debug purposes.
The issue I'm running into is while elastic stores the file under the proper _id, which in my case is the filename, it does not store the proper field: filename. For instance, if I upload 2 files called hello.txt and world.txt, ES will store both files with hello and world as their respective _ids, but the filename field is often swapped or just generally, incorrect. I've ran the code multiple times to see if there was a pattern, and I can't really seem to find one.
The console.logs show that fname is the proper filename before the first http, after it, and after the second http, which is why I'm confused as to why the set processor is not storing it properly.
I may not have clearly explained the issue very well, as it's kind of convoluted. Let me know if anything needs extra explanation. Thanks!

How to create a file object with a path in NodeJS?

I want to know if it is possible to create a file object (name, size, data, ...) in NodeJS with the path of existing file ? I know that it is possible in client side but I see nothing for NodeJS.
In others words, I want the same function works in NodeJS :
function srcToFile(src, fileName, mimeType){
return (fetch(src)
.then(function(res){return res.arrayBuffer();})
.then(function(buf){return new File([buf], fileName, {type:mimeType});})
);
}
srcToFile('/images/logo.png', 'logo.png', 'image/png')
.then(function(file){
console.log(file);
});
And ouput will be like :
File {name: "logo.png", lastModified: 1491465000541, lastModifiedDate: Thu Apr 06 2017 09:50:00 GMT+0200 (Paris, Madrid (heure d’été)), webkitRelativePath: "", size: 49029, type:"image/png"…}

For those that are looking for a solution to this problem, I created an npm package to make it easier to retrieve files using Node's file system and convert them to JS File objects:
https://www.npmjs.com/package/get-file-object-from-local-path
This solves the lack of interoperability between Node's fs file system (which the browser doesn't have access to), and the browser's File object type, which Node cannot create.
3 steps are required:
Get the file data in the Node instance and construct a LocalFileData object from it
Send the created LocalFileData object to the client
Convert the LocalFileData object to a File object in the browser.
// Within node.js
const fileData = new LocalFileData('path/to/file.txt')
// Within browser code
const file = constructFileFromLocalFileData(fileData)

So, I search with File Systems and others possibilities and nothing.
I decide to create my own File object with JSON.
var imagePath = path.join('/images/logo.png', 'logo.png');
if (fs.statSync(imagePath)) {
var bitmap = fs.readFileSync(imagePath);
var bufferImage = new Buffer(bitmap);
Magic = mmm.Magic;
var magic = new Magic(mmm.MAGIC_MIME_TYPE);
magic.detectFile(imagePath, function(err, result) {
if (err) throw err;
datas = [{"buffer": bufferImage, "mimetype": result, "originalname": path.basename(imagePath)}];
var JsonDatas= JSON.parse(JSON.stringify(datas));
log.notice(JsonDatas);
});
}
The output :
{
buffer:
{
type: 'Buffer',
data:
[
255,
216,
255
... 24908 more items,
[length]: 25008
]
},
mimetype: 'image/png',
originalname: 'logo.png'
}
I think is probably not the better solution, but it give me what I want. If you have a better solution, you are welcome.

You can use arrayBuffer (thats what i did to make a downloadable pdf) or createReadStream / createWriteStream under fs(FileSystem objects)