Develop Reference

How implement types properly?

I'm new to Typescript and have been doing a refactor a colleague code, I'm currently doing a typecheck and removing all any types. The goal is to make an MSGraph API call and return the a JSON file that translated into BirthdayPerson with a name, birthday date and a ID
I've been trying to a assign a type instead of any in the following code, but whether I assign number, string or any other type a different error will show up.
Perhaps I'm not tackling the solution correctly:
graph.ts
* #param accessToken
* #param endpoint url to call from MS Graph
*/
async function callMsGraph(accessToken: string, endpoint: string) {
const headers = new Headers();
const bearer = `Bearer ${accessToken}`;
headers.append('Authorization', bearer);
const options = {
method: 'GET',
headers,
};
try {
return fetch(endpoint, options);
} catch (error) {
console.error(error);
throw error;
}
}
export const callMsGraphWithoutPagination = async (
accessToken: string,
url: string,
dataToReturn: any[] = []
): Promise<any[]> => {
try {
const data = await callMsGraph(accessToken, url);
const dataJson = await data.json();
const newData = dataToReturn.concat(dataJson.value);
if (dataJson['#odata.nextLink']) {
const NEXT_URL = dataJson['#odata.nextLink'].split('/v1.0')[1];
return await callMsGraphWithoutPagination(
accessToken,
process.env.REACT_APP_GRAPH_URL + NEXT_URL,
newData
);
}
return dataToReturn.concat(dataJson.value);
} catch (error) {
/* eslint-disable no-console */
console.error(error);
/* eslint-enable no-console */
throw error;
}
};
export default callMsGraph;
useUsers.tsx
export const useUsers = () => {
const token = useToken();
const [users, setUsers] = React.useState<BirthdayPerson[]>([]);
React.useEffect(() => {
if (token) {
callMsGraphWithoutPagination(token, graphConfig.usersEndpoint).then(async (data: any) => {
const processedData: any[] = await Promise.all(
data.map(async (element: any) => {
const user = await callMsGraph(token, graphConfig.userBirthdayEndpoint(element.id));
const userJson = await user.json();
const image = await callMsGraph(token, graphConfig.userPhotoEndpoint(element.id));
const blob = await image.blob();
const returnElement: BirthdayPerson = {
displayName: element.displayName,
birthday: userJson.value,
id: element.id,
};
if (blob !== null) {
window.URL = window.URL || window.webkitURL;
returnElement.picture = window.URL.createObjectURL(blob);
}
return returnElement;
})
);
setUsers([].concat(...processedData));
});
}
}, [token]);
return users;
};
helpers.ts
interface IUpdateData {
slideCount: number;
}
const sortAndFilterBirthdays = (people: BirthdayPerson[], daysToGet: number) =>
people
.sort((firstEl, secondEl) => sortDate(firstEl.birthday, secondEl.birthday))
.filter(({ birthday }) => filterByAmountOfDays({ date: birthday, daysAfter: daysToGet }));
const getBirthdays: any = (people: BirthdayPerson[], daysToGet: number) => {
const validBirthdays = people.filter((element: any) => {
const year = moment(element.birthday).year();
return year !== 0;
});
const result = sortAndFilterBirthdays(validBirthdays, daysToGet);
// if it's okay
if (result.length > 1 && daysToGet <= 30) {
return result;
}
// if not okay, filters by future dates, concats with 'next year' dates, returns 2 dates
const fallbackResult = validBirthdays
.sort((firstEl, secondEl) => sortDate(firstEl.birthday, secondEl.birthday))
.filter((person: BirthdayPerson) => {
const currentYear = moment().year();
const date = moment(person.birthday, DATE_FORMAT).set('years', currentYear);
return moment().diff(date, 'days') <= 0;
});
return fallbackResult.concat(validBirthdays).splice(0, 2);
};
Any help or indication would be great!
From all the changes I've done another object will complain that Type 'x' is not assignable to type 'string'

Firstly you need to somehow type responses from API, because right now, as you have seen, call to .json() on Response object returns unknown, which make sense because no one knows what response the server returns. You may know what response it is expected to return, but not what it actually does.
Ideally therefore you need a parser that will verify that the response has correct structure and throws an error otherwise. There are libraries such as superstruct, yup, joi and others which you can use for this. Of course this is a lot of work and will need refactoring. Or if you don't care enough you can just cast the response object to appropriate type, but then if server returns something unexpected and the application cannot handle it, it's your fault.
Example with response parsing using superstruct
import {string, number, create, Infer} from 'superstruct'
// I assume `BirthdayUser` is just a string, but you can type more complex objects as well
const BirthdayUser = string()
// This is so that you don't have to list fields twice: once
// for the parser and once for typescript
type BirthdayUser = Infer<typeof BirthdayUser>
// Then use the parser
const response = await callMsGraph(acessToken, url)
const userJson = await response.json()
// user variable has inferred appropriate type, and if the response doesn't
// comply with the definition of `BirthdayUser` an error will be thrown
// Also I assume MS graph doesn't just return plain value but wraps it in an object with `value` field, so writing it here
const user = create(userJson, object({ value: BirthdayUser }))
Example of "I don't care enough" solution
type BirthdayUser = string
const response = await callMsGraph(accessToken, url)
// Same thing with wrapping into object with `value` field
const userJson = (await response.json()) as {value: BirthdayUser}
This is a bit awkward, because API call returns Response object and not the actual data. It might be easier to work with if you move parsing and casting logic inside of callMsGraph. It's not obligatory of course, but I still provide an example because after that you need to type callMsGraphWithoutPagination and it will use the same idea
import {object, create, Struct} from 'superstruct'
async function callMsGraphParsed<T>(
accessToken: string,
url: string,
// Since we need information about object structure at runtime, just making function
// generic is not enough, you need to pass the parser structure as an argument
struct: Struct<T>
) {
// ...
const response = await fetch(...)
const json = await response.json()
// Same, verifies that response complies to provided structure, if it
// does returns type object (of type `T`), otherwise throws an error
return create(json, object({ value: struct }))
}
async function callMsGraphLazy<T>(accessToken: string, url: string) {
// ...
const response = await fetch(...)
const json = await response.json()
return json as {value: T}
}
However I only call .json() here, if you want to use this solution, you will then need either a different function or another argument if you also want it to call .blob() for some API calls.
Now you type callMsGraphWithoutPagination using in the same way:
export const callMsGraphWithoutPaginationParsed = async <T>(
accessToken: string,
url: string,
dataToReturn: T[] = [],
struct: Struct<T>,
): Promise<T[]> => {
// If you rewrote `callMsGraph` to use parsing
const dataJson = await callMsGraph(accessToken, url, struct);
const newData = dataToReturn.concat(dataJson.value);
// ...
}
export const callMsGraphWithoutPaginationLazy= async <T>(
accessToken: string,
url: string,
dataToReturn: T[] = [],
): Promise<T[]> => {
// If you left `callMsGraph` as is
const data = await callMsGraph(accessToken, url);
const dataJson = (await data.json()) as {value: T}
const newData = dataToReturn.concat(dataJson.value);
// ...
}
Then use it
// Not sure if you are requesting `BirthdayUser` array here or some other entity, so change it to whatever you expect to receive
callMsGraphWithoutPagination<BirthdayUser>(token, graphConfig.usersEndpoint).then(async (data) => {
// "data" is inferred to have type BirthdayUser[]
data.map(element => {
// "element" is inferred to have type BirthdayUser
})
})
Also everywhere I wrote "I assume" and "Not sure" is missing info that you should probably have provided in the question. It didn't turn out to be relevant for me, but it could have. Good luck!

How to get both response JSON and headers with Got?

I'm using Got to make requests to a Strapi API from Node, like so:
res.setHeader('Content-Type', 'application/json')
try {
const request = req.query.request
const decodedRequest = Buffer.from(request, 'base64').toString()
const api = process.env.API_URL ? process.env.API_URL.replace(/\/$/, '') : ''
const url = `${api}${decodedRequest}`
const response = await got.get(url)
const body = await got.get(url).json()
const headers = JSON.parse(JSON.stringify(response.headers))
res.status(200).json({
headers: headers,
data: body
})
} catch (e) {
res.status(500).json({})
}
This works but note that I have the request twice because if I do:
res.setHeader('Content-Type', 'application/json')
try {
const request = req.query.request
const decodedRequest = Buffer.from(request, 'base64').toString()
const api = process.env.API_URL ? process.env.API_URL.replace(/\/$/, '') : ''
const url = `${api}${decodedRequest}`
const response = await got.get(url)
const body = response.json()
const headers = JSON.parse(JSON.stringify(response.headers))
res.status(200).json({
headers: headers,
data: body
})
} catch (e) {
res.status(500).json({
error: e
})
}
it Just crashes and the e from the catch returns an empty error so I have no idea what's going on
I need the headers because the pagination info from Strapi is returned there:

This works because of the value you're awaiting. The conventional example:
const body = await got.get("...").json();
is equivalent to:
const res = got.get("...");
const body = await res.json();
// ^ note
but not:
const res = await got.get("...");
// ^ note
const body = res.json();
From the Promise API docs:
The main Got function returns a
Promise.
Although in order to support cancelation,
PCancelable is used
instead of pure Promise.
The json method is attached to this PCancelable object, not the value it resolves to. If you try to call it on the response, therefore, you get TypeError: res.json is not a function.
What you want is something like:
const res = await got.get("...");
const body = JSON.parse(res.body);
const headers = res.headers;
// ...
That said, if you're doing this for pagination reasons you could also look into their API for that.

Strange behaviour of params.append with axios

export const getCharactersAsync = createAsyncThunk('getCharactersAsync', async (data) => {
const response = await axios.get('users', { params: { limit: data.limit } });
return response.data;
});
this code block allows me to control limit attribute.
export const getCharactersAsync = createAsyncThunk('getCharactersAsync', async (data) => {
const params = new FormData();
// const params = new URLSearchParams();
params.append('limit', data.limit);
const response = await axios.get('users', params);
console.log(response);
return response.data;
});
However I cannot control limit with using params.append. I tried URLSearchParams instead of FormData but still cannot manipulate limit attribute of the response. Why they differ from each other?
EDIT: This question has missleading information. I should have mention that i am using react-native. I found that react native doesn't fully support everything the web supports. So i need to install package called react-native-url-polyfill.Here is a github issues link
https://github.com/facebook/react-native/issues/23922#issuecomment-648096619

docs
params are the URL parameters to be sent with the request. Must be a plain object or a URLSearchParams object
It can't be FormData
Solution
You wanted to use { params }, not params
export const getCharactersAsync = createAsyncThunk('getCharactersAsync', async (data) => {
const params = new URLSearchParams();
params.append('limit', data.limit);
const response = await axios.get('users', { params });
console.log(response);
return response.data;
});

(hashlips_art_engine-1.1.2_patch_v5) The "path" argument must be of type string. Received undefined

I'm trying to update some metadata for 10 JSON files, some of them get updated properly and some others give me the following error.
Below you can see 6 files give me errors and 4 files are correctly updated.
error:TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
error:TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
error:TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
error:TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
error:TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
error:TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
5.png uploaded & 5.json updated!
4.png uploaded & 4.json updated!
1.png uploaded & 1.json updated!
7.png uploaded & 7.json updated!
This is the node.js code im using:
onst FormData = require('form-data');
const fetch = require('node-fetch');
const path = require("path");
const basePath = process.cwd();
const fs = require("fs");
fs.readdirSync(`${basePath}/build/images`).
forEach(file => {
const formData = new FormData();
const fileStream = fs.createReadStream(`${basePath}/build/images/${file}`);
formData.append("file", fileStream);
let url = 'https://api.nftport.xyz/v0/files';
let options = {
method: 'POST',
headers: {
Authorization: 'SecretApiCode',
},
body: formData
};
fetch(url, options)
.then(res => res.json())
.then((json) => {
const fileName = path.parse(json.file_name).name;
let rawdata = fs.readFileSync(`${basePath}/build/json/${fileName}.json`);
let metaData = JSON.parse(rawdata);
metaData.file_url = json.ipfs_url;
fs.writeFileSync(`${basePath}/build/json/${fileName}.json`,
JSON.stringify(metaData, null, 2));
console.log(`${json.file_name} uploaded & ${fileName}.json updated!`);
})
.catch(err => console.error('error:' + err));
});
I have 10png files:
1.png
2.png
3.png
4.png
5.png
6.png
7.png
8.png
9.png
10.png
And 10 JSON files
1.json
2.json
3.json
4.json
5.json
6.json
7.json
8.json
9.json
10.json

The issue in your code seems to be with fetching the URL.
This is proven as sometimes, your code fails with an error, and sometimes, it works successfully.
What might be happening is that sometimes, when you are fetching the data from the URL, it might not have the file_name property (because of an error, etc.)
When you are trying to parse the path with the below method:
path.parse(json.file_name);
The file_name might be undefined as there might not be such a key. And, as path.parse() can't parse undefined, it returns an error, saying that in only accepts strings (shown below).
TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
There are many possible solutions, so they are listed below.
Add a check for if file_name is available. If it isn't, then don't try parsing it with path.parse(). You can use the code below to check it (in the second .then() method).
if (!json.file_name) {
console.error("Error fetching file name.");
return;
}
If your API supports it, check if there is an error in the response body. Assuming that the error will be something like .error, you can use the code below for guidance (add it in the second .then() method).
if (!json.file_name && json.error) {
console.log(json.error.message);
return;
}
There may be many other (better) solutions then these, but these solutions are just to give you a starting point.
Solution 1 and 2 are very similar, but they work under different circumstances.

Try this. You can also find it in codeSTACKr/create-10k-nft-collection" git repository if you change tag to v1.2.0.
This is in uploadfiles.js
const FormData = require("form-data");
const fetch = require("node-fetch");
const path = require("path");
const basePath = process.cwd();
const fs = require("fs");
const AUTH = 'Your API KEY GOES HERE';
const TIMEOUT = 1000; // Milliseconds. Extend this if needed to wait for each upload. 1000 = 1 second.
const allMetadata = [];
async function main() {
const files = fs.readdirSync(`${basePath}/build/images`);
files.sort(function(a, b){
return a.split(".")[0] - b.split(".")[0];
});
for (const file of files) {
const fileName = path.parse(file).name;
let jsonFile = fs.readFileSync(`${basePath}/build/json/${fileName}.json`);
let metaData = JSON.parse(jsonFile);
if(!metaData.file_url.includes('https://')) {
const response = await fetchWithRetry(file);
metaData.file_url = response.ipfs_url;
fs.writeFileSync(
`${basePath}/build/json/${fileName}.json`,
JSON.stringify(metaData, null, 2)
);
console.log(`${response.file_name} uploaded & ${fileName}.json updated!`);
} else {
console.log(`${fileName} already uploaded.`);
}
allMetadata.push(metaData);
}
fs.writeFileSync(
`${basePath}/build/json/_metadata.json`,
JSON.stringify(allMetadata, null, 2)
);
}
main();
function timer(ms) {
return new Promise(res => setTimeout(res, ms));
}
async function fetchWithRetry(file) {
await timer(TIMEOUT)
return new Promise((resolve, reject) => {
const fetch_retry = (_file) => {
const formData = new FormData();
const fileStream = fs.createReadStream(`${basePath}/build/images/${_file}`);
formData.append("file", fileStream);
let url = "https://api.nftport.xyz/v0/files";
let options = {
method: "POST",
headers: {
Authorization: AUTH,
},
body: formData,
};
return fetch(url, options).then(async (res) => {
const status = res.status;
if(status === 200) {
return res.json();
}
else {
console.error(`ERROR STATUS: ${status}`)
console.log('Retrying')
await timer(TIMEOUT)
fetch_retry(_file)
}
})
.then(async (json) => {
if(json.response === "OK"){
return resolve(json);
} else {
console.error(`NOK: ${json.error}`)
console.log('Retrying')
await timer(TIMEOUT)
fetch_retry(_file)
}
})
.catch(async (error) => {
console.error(`CATCH ERROR: ${error}`)
console.log('Retrying')
await timer(TIMEOUT)
fetch_retry(_file)
});
}
return fetch_retry(file);
});
}

How do I properly route data through my Node API?

I have the following files:
My routes - where the orders_count route lives:
routes/index.js
const express = require('express');
const router = express.Router();
const transactionsController = require('../controllers/transactionsController');
const ordersController = require('../controllers/ordersController');
const ordersCountController = require('../controllers/ordersCountController');
router.get('/transactions', transactionsController);
router.get('/orders', ordersController);
router.get('/orders_count', ordersCountController);
module.exports = router;
I then have my orders count controller living in the controllers directory:
controllers/ordersCountController.js
const ordersCountService = require('../services/ordersCountService');
const ordersCountController = (req, res) => {
ordersCountService((error, data) => {
if (error) {
return res.send({ error });
}
res.send({ data })
});
};
module.exports = ordersCountController;
My controller then calls my order count service which fetches data from another API.
services/ordersService.js
const fetch = require('node-fetch');
// connect to api and make initial call
const ordersCountService = (req, res) => {
const url = ...;
const settings = { method: 'Get'};
fetch(url, settings)
.then(res => {
if (res.ok) {
res.json().then((data) => {
return data;
});
} else {
throw 'Unable to retrieve data';
}
}).catch(error => {
console.log(error);
});
}
module.exports = ordersCountService;
I'm trying to return the JSON response. I initially had it setup with requests but looking at the NPM site, it appears that it's depreciated so have been digging through how to use node-fetch.
I have tried both 'return data' and res.send({data}), but neither are solving the problem.
I am still new to this so I am likely missing something very obvious, but how come I am not sending the JSON back through so that it displays at the /api/orders_count endpoint?
I keep thinking I messed something up in my controller but have been looking at it for so long and can't seem to figure it out.
Any help would be greatly appreciated and if there is anything I can add for clarity, please don't hesitate to ask.
Best.

please learn promises and await syntax. life will be easier.
never throw a string. always prefer a real error object, like that : throw new Error('xxx'); that way you will always get a stack. its way easier to debug.
avoid the callback hell : http://callbackhell.com/
you need to decide if you want to catch the error in the controller or in the service. no need to do in both.
in the controller you call the service that way :
ordersCountService((error, data) => {
but you declare it like that :
const ordersCountService = (req, res) => {
which is not compatible. it should look like this if you work with callback style :
const ordersCountService = (callback) => {
...
if (error) return callback(error)
...
callback(null, gooddata);
here is an example to flatten your ordersCountService function to await syntax, which allows the "return data" you were trying to do :
const fetch = require('node-fetch');
// connect to api and make initial call
const ordersCountService = async (req, res) => {
const url = ...;
const settings = { method: 'Get'};
try {
const res = await fetch(url, settings);
if (!res.ok) throw new Error('Unable to retrieve data');
return await res.json();
} catch(error) {
console.log(error);
}
}
module.exports = ordersCountService;
in fact i would prefer to error handle in the controller. then this woud be sufficient as a service
const fetch = require('node-fetch');
// connect to api and make initial call
const ordersCountService = async () => {
const url = ...;
const settings = { method: 'Get'};
const res = await fetch(url, settings);
if (!res.ok) throw new Error('Unable to retrieve data');
return await res.json();
}
module.exports = ordersCountService;
then you can call this funtion like this :
try {
const data = await ordersCountService(req, res);
} catch(err) {
console.log(err);
}
//or
ordersCountService(req, res).then((data) => console.log(data)).catch((err) => console.error(err));