Like
fetch('state_wise_data.csv')
.then(response => response.json())
.then(data => console.log(data))
.catch(err => console.log(err))
Tried doing this but didn't work.
First of all, CSV it's not a JSON. Fetch does not have CSV support, you will need to download CSV string (you can use response.text()) and use the third party CSV parser.
For parse CSV parser you can use papaparse:
"Isn't parsing CSV just String.split(',')?"
Heavens, no. Papa does it right. Just pass in the CSV string with an optional configuration.
Example:
const response = fetch('state_wise_data.csv')
.then(response => response.text())
.then(v => Papa.parse(v))
.catch(err => console.log(err))
response.then(v => console.log(v))
It also supports file downloading:
Papa.parse('state_wise_data.csv', {
download: true,
complete: results => {
console.log(results);
}
})
Fetch is 100% work with .csv file (or even api with req.query).
'content-type': 'text/csv' must be addressed in the fetch's headers:{}, and use res.text() instead of res.json() to interpret data.
const downloadCsv = async () => {
try {
const target = `https://SOME_DOMAIN.com/data/csv/addresses.csv`; //file
//const target = `https://SOME_DOMAIN.com/api/data/log_csv?$"queryString"`; //target can also be api with req.query
const res = await fetch(target, {
method: 'get',
headers: {
'content-type': 'text/csv;charset=UTF-8',
//'Authorization': //in case you need authorisation
}
});
if (res.status === 200) {
const data = await res.text();
console.log(data);
} else {
console.log(`Error code ${res.status}`);
}
} catch (err) {
console.log(err)
}
}
CSV is not a JSON file type, so you cant parse as a json text. you can check how to parse CSV text in javascript here : Example JavaScript code to parse CSV data
I would use the following method and insert it where you are supposed console.log the data.
const parseCSV = (data) => {
// create empty array
const csvData = [];
// this will return each line as an individual String
const lines = data.split("\n");
// loop through the lines and return an array of individual
// Strings within the line that are separated by a comma
for (let i = 0; i < lines.length; i++) {
csvData[i] = lines[i].split(",");
}
// return an array of arrays 2D array
// e.g [ [1,2,3], [3,4,5], [6,7,8] ]
return csvData;
};
pretty easy,
GET the url, with normal fetch req, and first convert response to text and then it's done
fetch('sample-url.csv')
.then((response) => response.text())
.then((data) => console.log(data));
Related
I am looking for some assistance with my fetch response from the Online Movie Database API . I am able to succesfully get a response to log in the console, which is what I want. However I am trying to manipulate the response.
I want to pull the most popular shows from the API (API sends 100 titles), and trim it down to 8 titles. I did that using the .splice method. It returns an array of 8 strings representing a title id.
Example: '/title/tt11198330'
Lastly I want to trim each 8 of the strings so it gets rid of the /title/ and all I have left is tt11198330. I am trying to do this inside the .then and when I console.log that forEach that is saved as 'trimmer' it gives me undefined instead of the trimmed result I was intending. But if I console.log the element in the forEach it does show the trimmed strings. Any idea why its undefined, and is there maybe a better way to go about this?
// fetch for most popular shows
const options = {
method: 'GET',
headers: {
'X-RapidAPI-Key': 'Z2EvqnO4xwmsh2eY3rMTIV2ivj5hp1QsuGUjsnrYp69UBS4EI5',
'X-RapidAPI-Host': 'online-movie-database.p.rapidapi.com'
}
};
fetch('https://online-movie-database.p.rapidapi.com/title/get-most-popular-tv-shows?currentCountry=US&purchaseCountry=US&homeCountry=US', options)
.then(response => response.json())
.then(response => {
const list = response.splice(0, 8)
let trimmer = list.forEach(element => console.log(element.slice(7, 17)))
console.log(trimmer)
})
.catch(err => console.error(err));
because you are using forEach and it doesn't return data, use map instead.
fetch('https://online-movie-database.p.rapidapi.com/title/get-most-popular-tv-shows?currentCountry=US&purchaseCountry=US&homeCountry=US', options)
.then(response => response.json())
.then(response => {
const list = response.splice(0, 8)
let trimmer = list.map(element => element.slice(7, 17))
console.log(trimmer)
})
.catch(err => console.error(err));
I have a Node Js server, in it, I am fetching a blob data from another web service (which is for a PDF file), now after receiving blob, I want to convert it again into PDF file.
Anyone, who knows how to achieve this please help.
Here is my code block I have tried so far:
const fetch = require('node-fetch');
const Blob = require('fetch-blob');
const fs = require('fs');
fetch(url, options)
.then(res => {
console.log(res);
res.blob().then(async (data) => {
const result = data.stream();
// below line of code saves a blank pdf file
fs.createWriteStream(objectId + '.pdf').write(result);
})
})
.catch(e => {
console.log(e);
});
Modification points:
For fs.createWriteStream(objectId + '.pdf').write(data), please modify res.blob() to res.buffer().
Please modify .then(res => {res.blob().then() to .then(res => res.buffer()).then(.
Modified script:
fetch(url, options)
.then(res => res.buffer())
.then(data => {
fs.createWriteStream(objectId + '.pdf').write(data);
})
.catch(e => {
console.log(e);
});
Note:
In this modification, it supposes that the fetch process using url and options works fine.
References:
node-fetch
write()
How would you add data to a .json file that is already on a website? I kinda of get how you add data to a .json locally but if I need to send it to a file that has already been created and hosted.
This is my code so far in my server.js file
router.post('/themes', async (ctx) => {
const results = await fetch(`https://`+ ctx.cookies.get("shopOrigin") + `/admin/themes.json`, {
headers: {
"X-Shopify-Access-Token": ctx.cookies.get('accessToken'),
},
})
.then(response => response.json());
ctx.body = {
status: 'success',
data: results
};
});
and this is the code in my frontend .js file
async function getUser() {
const query = [
{
"theme": {
"name": "Lemongrass",
"src": "https://codeload.github.com/Shopify/skeleton-theme/zip/master"
}
}
]
var url = "/themes";
var method = 'post';
fetch(url, { method: method, body: query})
.then(response => response.json())
.then(json => console.log(json))
};
I can pull the data aka the themes on my console, however, I need to send the data so I can create a new theme.
you're not very clear but if I understand you properly you're saying
step 1 - you called an endpoint and got a response as JSON
step 2 - you modified it and you want to send that data to a different endpoint.
assuming I'm correct above, this is the answer
let assume after step 1 the data returned is this
let response = [1,2,3,4]
and to modified it to
let newData = [{value: 1},{value: 2},{value: 3},{value: 2}]
then in step - 2 send newData instead of response.
what to take away?
if you make any modifications to a data from an endpoint then it's safer to save your modifications to a new variable in this case newData
I have a problem in nodejs, I make a request to an api using https.request, the response contains an object of 10000 rows.
What happens is that the entire object does not arrive, and parsing gives the error: Unexpected end of JSON input;
Can someone help?
Function to request:
function request({
options,
method,
resource,
queryParams,
bodyParams,
}) {
return new Promise((resolve, reject) => {
const hasBodyParams = !!bodyParams;
const stringifyedQueryParams = strigifyQueryParams(queryParams);
const optionsRequest = {
...options,
method,
path: `${resource}${stringifyedQueryParams}`,
};
const req = https.request(optionsRequest, (res) => {
res.setEncoding(configs.ENCODING);
res.on(events.DATA, data => resolve({
body: JSON.parse(data),
statusCode: res.statusCode,
}));
});
req.on(events.ERROR, error => reject(error) );
hasBodyParams && req.write(bodyParams);
req.end();
});
}
As I suspected in the comments, you're not handling multiple data-events.
When receiving large responses from a request, the data-event is called multiple times, each time with a chunk of data from the response (not the complete response).
When you're parsing a chunk, the complete JSON document hasn't been transmitted yet, so the parsing fails with the "Unexpected end of JSON stream" error
In short, you need to:
Create a variable to collect the complete body
On a data-event, append the new chunk to the complete body
When the end-event is called, parse the full body.
Here is a short example, adopted from the official documentation:
https.request(options, (res) => {
// PARTIAL example
res.setEncoding("utf8"); // makes sure that "chunk" is a string.
let fullBody = "";
res.on("data", data => {
fullBody += data;
});
res.on("end", () => {
const json = JSON.parse(fullBody);
// work with json
});
});
I have a JSON file I want to read from and write to that looks like this:
[
"test#example.com"
]
I want to add info to this file using the fetch() API. So far, I can only read from this file.
let handle = JSON.stringify(`test#example2.com`); // handle is fine irrelevant of "" or ``
const url = `../json/emails.json`; // URL is fine
const options = {
method: `POST`, // PUT return error at second fetch: SyntaxError: "JSON.parse: unexpected character at line 1 column 1 of the JSON data"
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body: handle
};
fetch(url) // this fetch is fine
.then(response => {
if (!response.ok) {
throw new Error(`Error getting the stuff`);
}
return response;
})
.then(r => r.json())
.then(data => {
console.log(`Data: ` + data);
});
fetch(url, options)
.then(response => {
if (!response.ok) {
throw new Error(`Error getting the stuff`);
}
return response;
})
.then(a => a.json())
.then(append => {
console.log(`Data updated: ` + append);
}).catch(err => {
console.error('Request failed: ', err);
});
I get no errors (aside from that PUT comment); ESLint and TSLint don't have any problem with the JS file nor with the JSON file. What am I doing wrong?
fetch() is an API for making HTTP requests.
It can't write to files. In particular, nothing can write to arbitrary URLs. (Imagine if it was possible for any browser to write new data to http://www.google.com/!)
If you want your PUT or POST request to change data on your server, then you must write server-side code to process the request and edit the file.