i trying host media/image files on cloudflare r2 bucket. I tried lots of stuff but I still can't reach bucket from outsite with nodeJS.
How can i fix this code?
My Final Code:
fetch('https://USER_ID.r2.cloudflarestorage.com/PROJECT_NAME/src/thumbs/BLAHBLAH', {
method: 'GET',
headers: {
'Content-Type': 'image/jpeg',
'Authorization': 'SECRET_KEY_FOR_API',
'X-Amz-Ac': 'private',
'X-Amz-Algorithm': 'AWS4-HMAC-SHA256',
'X-Amz-Date': (new Date().toISOString().split(':').join('').split('.')[0] + 'Z').split('-').join(''),
'X-Amz-Expires': '86400',
'x-amz-content-sha256': 'UNSIGNED-PAYLOAD',
}}).then(res => res.text()).then(buffer => {
console.log(buffer);
}).catch(err => {
console.log(err);
});
You'll need to implement proper SigV4 signing for your request.
It'll be easiest to use one of the AWS SDKs, like aws-sdk-js - take a look at the example documentation for R2: https://developers.cloudflare.com/r2/examples/aws-sdk-js-v3/
Related
I have a working Apollo-Graphql Node.js server running with express middleware. Queries and Mutations, including file upload mutation, work fine from connecting front end clients and when called from functions run in Node and passed via axios requests -- except for the file upload mutations.
I've tested using the same query and file paths in Firecamp, and tried variations of passing and checking the file path / doing my best to confirm that the directory structure is getting parsed accurately. This is the error code returned with the axios response (I broke up the response for config.data from console output):
response: {
status: 400,
statusText: 'Bad Request',
headers: {
'x-powered-by': 'Express',
'access-control-allow-origin': '*',
'content-type': 'application/json; charset=utf-8',
'content-length': '1424',
etag: 'W/"590-jNRKeEwYD1b3Cxa/bjf3qp7npHg"',
date: 'Fri, 11 Dec 2020 22:36:12 GMT',
connection: 'close'
},
config: {
url: 'http://localhost:4002/graphql',
method: 'post',
data: '{
"query":"mutation singleUpload($file: Upload!) {
singleUpload(file: $file) {
filename
mimetype
encoding
}
}",
"variables":{"file":"../bulkImports/testPenThumbnail.png"}
}',`
The query definition and function call to axios:
const UPLOAD_FILE = gql`
mutation singleUpload($file: Upload!) {
singleUpload(file: $file) {
filename
mimetype
encoding
}
}
`
export function uploadFile(endpoint) {
const file = '../bulkImports/testPenThumbnail.png';
axios.post(endpoint, {
query: print(UPLOAD_FILE),
variables: { file } })
.then(res => console.dir(res.data))
.catch(err => console.error(err));
}
And the resolver for singleUpload
singleUpload(parent, args) {
return args.file.then(file => {
const { createReadStream, filename, mimetype, encoding } = file;
const stream = createReadStream();
const pathName = join(__dirname, `../../testUploads/${filename}`);
stream.pipe(createWriteStream(pathName));
return {
filename,
mimetype,
encoding,
};
});
}
From other errors/debugging along the way, my best guess is that the upload mutation is only seeing the file path as an ordinary String and not parsing it as an Upload scalar -- and that I should be looking at using the fs module to send more in the way of file object data/stream? I've tried a few things using fs methods, but node/back-end is still pretty new to me and I'm not really sure if I'm even barking up the right tree for how the Upload scalar is constructed.
Of course I'm happy to post any more config or error details that would help -- and thanks in advance to everyone who can help me make sense of this or improve the code below!
(oh, and the intended use-case for calling this from a server will be for bulk uploading records to populate a new db collection; besides just trying to learn more about back-end/node/axios/graphql basics...)
The API I am trying to consume is from a third party and it blocks Cross-Origin, so I can not consume it with jquery, nor javascript ... So I had to create a script in nodejs to consume this API ...
I would like to know how do I consume this data that nodejs got from the API to my frontend, using javascript?
Remembering that this nodejs is in a separate file from my frontend and running from another server.
var request = require("../../node_modules/request");
var options = { method: 'GET',
url: 'https://....apimanagement.us2.hana.ondemand.com/bot/v1/...',
qs: { Shop: '..'', PeriodoAte: '...' },
headers:
{ 'postman-token': '822e513f-da5e-4a0b-b403-1dd8fa46e86f',
'cache-control': 'no-cache',
authorization: 'Basic .........',
apikey: '....',
'content-type': 'application/json' },
json: true };
request(options, function (error, response, body) {
console.log('error:', error); // Print the error if one occurred
console.log('statusCode:', response && response.statusCode); // Print the response status code if a response was received
console.log('body:', body); // Print the HTML for the Google homepage.
});
One path would be:
pick a node framework (Express, Hapi, ...) lets pick Hapi for this example
read the getting started guide https://hapijs.com/tutorials/getting-started
learn about promises or go through callback hell
end up with something like this for testing
'use strict';
const Hapi = require('hapi');
const rp = require('request-promise');
const server = Hapi.server({
port: 3000,
host: 'localhost'
});
server.route({
method: 'GET',
path: '/',
handler: (request, h) => {
const options = {
method: 'GET',
uri: 'https://./..',
qs: {Shop: '..', PeriodoAte: '...'},
headers: {
'postman-token': '822e513f-da5e-4a0b-b403-1dd8fa46e86f',
'cache-control': 'no-cache',
authorization: 'Basic .........',
apikey: '...',
'content-type': 'application/json'
},
json: true
};
return rp(options).catch(e => {
console.log(`api call failed ${e}`);
return 'fail';
})
}
});
const init = async () => {
await server.start();
console.log(`Server running at: ${server.info.uri}`);
};
process.on('unhandledRejection', (err) => {
console.log(err);
process.exit(1);
});
init();
now you can start it with node and visit 'localhost:3000/'
read more, try more
read about node production mode and hosting
Something like that, hope it helps a bit
EDIT: //
to consume on client just use for example jquery to fetch the route served with the code above
Here is some client example as requested in the comments:
If you are serving your html from another location, not the hapi api, you will need to enable cors in the HapiJS route.
server.route({
method: 'GET',
path: '/',
config: {
cors: {
origin: ['*'],
additionalHeaders: ['cache-control', 'x-requested-with']
}
},
Then one way to consume this endpoint would be jquery
<!doctype html>
<html>
<head>
<script src="https://code.jquery.com/jquery-3.3.1.min.js"></script>
</head>
<body>
<input type="button" id="Button" value="Fetch"/>
<div id='main'></div>
<script type="text/javascript">
$(document).ready(function () {
$('#Button').click(() => {
$.ajax({
url: "http://localhost:3000/info", success: (result) => {
$("#main").text(result);
}
});
});
});
</script>
</body>
</html>
You need a web server listening for requests in order to communicate your frontend with your backend (Node.js). You could use Express if you want something simple. You could also use a JavaScript framework, there are many (Sails.js would be a good option to get started fast).
If you can't install Node.js in your server, another option would be using AWS Lambda to quickly create an API that you can consume from your frontend through an HTTP request.
I am trying to upload a file from a react front end to a C# backend. I am using drop zone to get the file and then I call an api helper to post the file but I am getting different errors when I try different things. I am unsure exactly what the headers should be and exactly what I should send but I get two distinct errors. If I do not set the content-type I get 415 (Unsupported Media Type) error. If I do specify content type as multipart/form-data I get a 500 internal server error. I get the same error when the content-type is application/json. The url is being past in and I am certain it is correct. I am unsure if the file should be appended as file[0][0] as I have done or as file[0] as it is an array but I believe it should be the first. Any suggestions welcome :) Here is my api post helper code:
export const uploadAdminFile = (file, path, method = 'POST', resource =
config.defaultResource) => {
const url = createUrl(resource, path);
const data = new FormData();
data.append('file', file[0][0]);
data.append('filename', file[0][0].name);
const request = accessToken =>
fetch(
url,
{
method,
mode: 'cors',
withCredentials: true,
processData: false,
headers: {
Accept: 'application/json',
'Content-Type': 'application/json', //'multipart/form-data',
Authorization: `Bearer ${accessToken}`,
},
body: data,
})
.then(res => res.json())
.then(success => console.log('API HELPER: file upload success: ', success)
.catch(err => console.log('API HELPER: error during file upload: ', err)));
return sendRequest(request, resource);
};
Thanks for the help and suggestions, it turned out to be a backend issue but even still I learned a lot in the process. I will post my working code here in case anyone comes across this and finds it useful.
export const uploadAdminFile = (file, path, resource=config.defaultResource) => {
const url = createUrl(resource, path);
const formData = new FormData();
formData.append('file', file[0][0]);
formData.append('filename', file[0][0].name);
const request = accessToken =>
fetch(url,
{
method: 'POST',
headers: {
Accept: 'application/json',
Authorization: `Bearer ${accessToken}`,
},
body: formData,
});
return sendRequest(request, resource);
};
As mentioned, the file name does not need to be sent separately and count be omitted. I am indexing the file this way because I get it from dropzone as an array and I only want a single file (the first one in the array). I hope this helps someone else out and here is a link to the mdn fetch docs (good information) and a good article on using fetch and formData.
Previously I was using the Dropbox API V1 within my web app to upload files my dropbox account. Please note that the app uses only one dropbox account (mine) to upload files.
So Previously:
I created an app on the dropbox developers console
Generated my token from the developers console
Hard coded that token into my server to upload all file to a specific folder within my Dropbox.
This worked perfectly before but as the dropbox API v1 has been deprecated it does not work anymore.
Dropbox V1 Code:
function fileupload(content) {
request.put('https://api-content.dropbox.com/1/files_put/auto/my_reports/report.pdf', {
headers: {
Authorization: 'TOKEN HERE',
'Content-Type': 'application/pdf'
},
body: content
}, function optionalCallback(err, httpResponse, bodymsg) {
if (err) {
console.log(err);
}
else {
console.log("File uploaded to dropbox successfully!");
fs.unlink(temp_dir + 'report.pdf', function(err) {
if (err)
throw err;
else {
console.log("file deleted from server!");
}
})
request.post('https://api.dropboxapi.com/1/shares/auto/MY_reports/report.pdf' + '?short_url=false', {
headers: {
Authorization: 'TOKEN HERE'
}
}, function optionalCallback(err, httpResponse, bodymsg) {
if (err) {
console.log(err);
}
else {
console.log('Shared link 2 ' + JSON.parse(httpResponse.body).url);
}
});
}
});
}
Dropbox V2 Code:
function fileupload(content) {
request.post('https://content.dropboxapi.com/2/files/upload/my_reports', {
headers: {
Authorization: 'TOKEN HERE',
'Content-Type': 'application/pdf'
},
body: content
} ......... (rest of the code is similar to above)
Issue:
What I have tried does not work. I can't seem to upload a file to my dropbox account from within my app. I have tried re-generating my TOKEN from the Dropbox App console but no luck.
Can anyone tell me what am I doing wrong?
Update:
I updated my code to similar structure for v2 of the API but still unable to resolve it.
request.post('https://content.dropboxapi.com/2/files/upload/', {
headers: {
Authorization: 'Bearer TOKEN',
'Dropbox-API-Arg': {"path": "/Homework","mode": "add","autorename": true,"mute": false},
'Content-Type': 'application/pdf'
//'Content-Type': 'application/vnd.openxmlformats-officedocument.presentationml.presentation'
},
body: content
} .... similar code
I encourage you to use existing nodejs dropbox packages, which hides abstraction of an authentication process, etc. under the hood.
Check official dropbox-sdk-js or try my tiny package dropbox-v2-api. Quick example:
const dropboxV2Api = require('dropbox-v2-api');
//create session
const dropbox = dropboxV2Api.authenticate({
token: 'TOKEN HERE'
});
//create upload stream
const uploadStream = dropbox({
resource: 'files/upload',
parameters: {
path: '/dropbox/path/to/file.txt'
}
}, (err, result) => {
// upload completed
});
//use nodejs stream
fs.createReadStream('path/to/file.txt').pipe(uploadStream);
My recommendation is also to use a SDK which abstracts over authentication. CloudRail for Node.js could be very useful here. It's quite easy to use and works for other providers like OneDrive as well.
const cloudrail = require("cloudrail-si");
const service = new cloudrail.services.Dropbox(
cloudrail.RedirectReceivers.getLocalAuthenticator(8082),
"[Dropbox Client Identifier]",
"[Dropbox Client Secret]",
"http://localhost:8082/auth",
"someState"
);
service.upload(
"/myFolder/myFile.png",
readableStream,
1024,
true,
(error) => {
// Check for potential error
}
);
Here is also a short article about the {“error”: “v1_retired”} issue.
My RESTful service allows batching requests.
I'm trying to combine requests into one batch with help of Fetch API:
let req1 = {
url: "/cups/count",
options: {
method: 'GET',
headers: {
'Content-Type': 'application/http'
}
}
},
req2 = {
url: "/spoons/count",
options: {
method: 'GET',
headers: {
'Content-Type': 'application/http'
}
}
},
authToken = "Bearer my_token123",
batchUrl = "http://something.com/batch",
options = {
method: 'POST',
headers: {
'Authorization': authToken,
'Content-Type': 'multipart/mixed'
},
body: {req1, req2}
};
return fetch(batchUrl, options)
.then(response => response.json())
.then(items => dispatch(batchSuccess(items)))
.catch((err) => {
console.log(err)
});
However it returns an error - bad request. I suppose I may combine HTTP requests in wrong way.
Is there simpler way of doing this?
Where in Network Chrome Dev Tools can I see nested HTTP requests?
Your code does not work because it does not follow multipart/mixed request format:
In Content-Type header, there is no boundary information.
The child requests are not divided by boundary, instead they will be sent as plain text of req1 & req2 object.
In order to send valid multipart/mixed request, there is a node.js module batchelor. According to the introduction page, its usage is pretty simple.
If you want to send multipart/mixed request from browser, you can use build tool (gulp, webpack etc.) to compile batchelor into something like "batchelor-compiled.js" and import it in HTML.
For developer tool, I didn't find anything in Chrome, but the child requests are visible in Firefox debug window's "Params" tab.
Here is an example of a batch request using the Fetch API with the Gmail Batch REST API.
This will get the content of several messages at once.
const response = await fetch("https://www.googleapis.com/batch/gmail/v1", {
headers: {
"Content-Type": "multipart/mixed; boundary=batch_boundary",
Authorization: "Bearer <access_token>",
},
method: "POST",
body: `--batch_boundary
Content-Type: application/http
Content-ID: 1
GET /gmail/v1/users/me/messages/{message-id-1}
--batch_boundary
Content-Type: application/http
Content-ID: 2
GET /gmail/v1/users/me/messages/{message-id-2}
--batch_boundary--`,
});
console.log(await response.text());