Develop Reference

Google App Script pageToken to save attachments to Google Drive

basically what I'm trying to do is to get all the attachments within the received emails to a folder in google Drive (there are many, mostly .PDF). But it says I can't go beyond 500 attached files with search function and that I have to use something called pageToken which I have no idea how to apply to my code. So I need some advice or guide or maybe some examples to do this.
function saveGmailtoGoogleDrive() {
const folderId = '1apaQJjDSK-bNfd3ZgiFqK23cE7SCPqoB'; //Google Drive Folder
const searchQuery = 'label:unread has:attachment'; //Filter
const threads = GmailApp.search(searchQuery, 0, 500);
threads.forEach(thread => {
const messages = thread.getMessages();
messages.forEach(message => {
const attachments = message.getAttachments({
includeInlineImages: false,
includeAttachments: true
});
attachments.forEach(attachment => {
// Insert the attachment to google drive folder
Drive.Files.insert(
{
title: attachment.getName(),
mimeType: attachment.getContentType(),
parents: [{ id: folderId }]
},
attachment.copyBlob()
);
});
});
});
};
function saveGmailtoGoogleDrive() {
const folderId = '1apaQJjDSK-bNfd3ZgiFqK23cE7SCPqoB'; //Google Drive Folder
const searchQuery = 'label:unread has:attachment'; //Filter
const threads = GmailApp.search(searchQuery, 0, 500);
threads.forEach(thread => {
const messages = thread.getMessages();
messages.forEach(message => {
const attachments = message.getAttachments({
includeInlineImages: false,
includeAttachments: true
});
attachments.forEach(attachment => {
// Insert the attachment to google drive folder
Drive.Files.insert(
{
title: attachment.getName(),
mimeType: attachment.getContentType(),
parents: [{ id: folderId }]
},
attachment.copyBlob()
);
});
});
});
};

The arguments of the method of search(query, start, max) are query, start, max. The current maximum value of max is 500. When this value is over, an error like Argument max cannot exceed 500. occurs. And, start is the start position of the search. So I thought that this can be used for achieving your goal. When this is reflected in your script, it becomes as follows.
Modified script:
From:
const threads = GmailApp.search(searchQuery, 0, 500);
To:
let [start, end] = [0, 500];
let threads = [];
do {
const t = GmailApp.search(searchQuery, start, end);
start += end;
threads = [...threads, ...t];
} while (threads.length == start);
By this modification, you can retrieve the emails in threads more than 500.
Reference:
search(query, start, max)

Pagination in TypeORM/NestJS

I have to introduce pagination in findAll() method. I really dont know how to do it. I tried but it is giving so many errors. I used findAndCount() method given by typeorm for that, But I am not sure how it will work.
As of now below method returning all the record. I need to return at a time 10 records. Please suggest what modification I need to do.
async findAll(queryCertificateDto: QueryCertificateDto,page=1): Promise<PaginatedResult> {
let { country, sponser } = queryCertificateDto;
const query = this.certificateRepository.createQueryBuilder('certificate');
if (sponser) {
sponser = sponser.toUpperCase();
query.andWhere('Upper(certificate.sponser)=:sponser', { sponser });
}
if (country) {
country = country.toUpperCase();
query.andWhere('Upper(certificate.country)=:country', { country });
}
const certificates = query.getMany();
return certificates;
}
this is PaginatedResult file.
export class PaginatedResult {
data: any[];
meta: {
total: number;
page: number;
last_page: number;
};
}
I tried changing code of findAll() but where clause is giving error. I am not sure how to handle query.getMany() in pagination.
const take = query.take || 10
const skip = query.skip || 0
const [result, total] = await this.certificateRepository.findAndCount(
{
where: query.getMany(), //this is giving error
take:take,
skip:skip
}
);
return result;
I need to introduce pagination in this method. Any help will be really helpful.

Typeorm has a really nice method specific to your usecase findAndCount
async findAll(queryCertificateDto: QueryCertificateDto): Promise<PaginatedResult> {
const take = queryCertificateDto.take || 10
const skip = queryCertificateDto.skip || 0
const country = queryCertificateDto.keyword || ''
const sponser = queryCertificateDto.sponser || ''
const query = this.certificateRepository.createQueryBuilder('certificate');
const [result, total] = await this.certificateRepository.findAndCount(
{
where: { country: Like('%' + country + '%') AND sponser: Like('%' + sponser + '%') }, order: { name: "DESC" },
take: take,
skip: skip
}
);
return {
data: result,
count: total
};
}
More documentation about Repository class can be found here

You don't need the .getMany() with your where in the last code, the result is an array of the data you need.
From your first code, you can do this:
async findAll(queryCertificateDto: QueryCertificateDto,page=1): Promise<PaginatedResult> {
// let's say limit and offset are passed here too
let { country, sponser, limit, offset } = queryCertificateDto;
const query = this.certificateRepository.createQueryBuilder('certificate');
if (sponser) {
sponser = sponser.toUpperCase();
query.andWhere('certificate.sponser = :sponser', { sponser });
}
if (country) {
country = country.toUpperCase();
query.andWhere('certificate.country = :country', { country });
}
// limit and take mean the same thing, while skip and offset mean the same thing
const certificates = await query
.orderBy("certificate.id", "ASC")
.limit(limit || 10)
.offset(offset || 0)
.getMany();
// if you want to count just replace the `.getMany()` with `.getManyandCount()`;
return certificates;
}```

React: rendering an Array of Data

Working on my first react project and not making any progress on how to loop through and render my data to the front-end.
I am using axios to get a list of stock tickers from a MySQL database, and for each of those i am once again using axios to scrape some values from an external website.
See code snippet below:
const fetchStocks = () => {
let stocksList = []
Axios.get('http://localhost:3001/fetchStocks').then((response) => {
response.data.map((val, key) => {
const url = 'https://www.tradegate.de/orderbuch.php?isin=' + val.stockTicker
Axios.get(url).then((response) => {
let $ = cheerio.load(response.data)
let name = $('#col1_content h2')[0].children[0].data
let last = $('#last')[0].children[0].data
let delta = $('#delta')[0].children[0].data
let high = $('#high')[0].children[0].data
let low = $('#low')[0].children[0].data
stocksList.push({
sId: val.sId,
stockName: name,
stockTicker: val.stockTicker,
stockLast: last,
stockDelta: delta,
stockHigh: high,
stockLow: low
})
})
})
})
}
When i do console.log(stocksList) i pretty much get what i want:
[
{
"sId": 3,
"stockName": "Tesla Inc.",
"stockTicker": "US88160R1014",
"stockLast": "1 049,80",
"stockDelta": "+8,90%",
"stockHigh": "1 049,80",
"stockLow": "966,90"
},
{
"sId": 1,
"stockName": "Apple Inc.",
"stockTicker": "US0378331005",
"stockLast": "128,00",
"stockDelta": "-1,16%",
"stockHigh": "130,28",
"stockLow": "127,70"
},
{
"sId": 2,
"stockName": "Intel Corp.",
"stockTicker": "US4581401001",
"stockLast": "42,725",
"stockDelta": "+0,78%",
"stockHigh": "42,85",
"stockLow": "42,37"
}
]
I would now like to do something like:
{
stocksList.map(stock => {
return (
<li key = {stock.sId}>{stock.stockName}</li>
)
})
}
for each entry in the database, but so far i had no luck and don't quite understand where i'm wrong - im guessing something about how i'm setting up the array?

Since your are returning JSX you may want to use () braces instead of {}.
{
stocksList.map(stock => (
return (
<li key = {stock.sId}>{stock.stockName}</li>
)
))
}
Or simply:
{
stocksList.map(stock => <li key = {stock.sId}>{stock.stockName}</li>)
}
P.S. also I don't see a part of your code where you are trying to render an array

When the component did mount, useEffect will be called then it calls fetchStocks function, finally fetch data from the server and puts at a component state via useState
import React, { useState, useEffect } from 'react';
const FC = () => {
const [stocksList, setStocksList] = useState([])
const fetchStocks = () => {
let list = []
Axios.get('http://localhost:3001/fetchStocks').then((response) => {
response.data.map((val, key) => {
const url = 'https://www.tradegate.de/orderbuch.php?isin=' + val.stockTicker
Axios.get(url).then((response) => {
let $ = cheerio.load(response.data)
let name = $('#col1_content h2')[0].children[0].data
let last = $('#last')[0].children[0].data
let delta = $('#delta')[0].children[0].data
let high = $('#high')[0].children[0].data
let low = $('#low')[0].children[0].data
list.push({
sId: val.sId,
stockName: name,
stockTicker: val.stockTicker,
stockLast: last,
stockDelta: delta,
stockHigh: high,
stockLow: low
})
})
})
})
setStocksList(list)
}
useEffect(() => {
fetchStocks()
}, [])
return stocksList.map(stock => <li key = {stock.sId}>{stock.stockName}</li>)
}

React hook useEffect runs continuously forever/infinite loop, making duplicate of data fetching from APi,

im fetching data from APi and doing filtering from severside, i'm using useInfiniteScroll to fetch only limited amount of data on the first page, and with this im doing pagination too...
const [casesList, setCasesList] = useState<CaseModel[]>([]);
const [isLoading, setIsLoading] = useState<boolean>(false);
const [isFetchingData, setIsFetchingData] = useState<boolean>(true);
const { inputValue } = React.useContext(MenuContext);
const debouncedValue = useDebounce(inputValue, 10);
these are my hook, (casesList) in which im saving all my incoming data from APi, input value is the value that im typing in search box for filtering the data, and debouncedValue is my custom hook, so the inputValue first goes to debouncedValue and then my debouncedValue will get the value of my inputValue,
const [pagination, setPagination] = useState<Pagination>({
continuationToken: "",
hasMoreResults: true,
});
const [isFetchingMore, setIsFetchingMore] = useInfiniteScroll();
these are my pagination and useInfiniteScroll() hooks...
so the actual problem that i'm facing is that,
const getDashboardCases = useCallback(
async (continuationToken: string) => {
setIsLoading(true);
let casesPageLimit = CASES_PAGE_LIMIT;
if (casesList.length === 0) {
const table = document.querySelector("#cases-items");
if (table) {
const caseItemHeight = 80;
const heightDifference =
table?.getBoundingClientRect().y > 0
? window.innerHeight - table?.getBoundingClientRect().y
: -1;
casesPageLimit = Math.max(
casesPageLimit,
Math.ceil(heightDifference / caseItemHeight)
);
}
}
const options: GetCasesListOptions = {
continuationToken,
filter: [],
sort: [],
pageLimit: casesPageLimit,
search: [debouncedValue]
};
const data: IData = await dashboardService.getCasesList(options);
setIsFetchingMore(false);
setIsLoading(false);
if (data.result) {
setIsFetchingData(false)
if (data.continuationToken !== undefined) {
const newContinuationToken = data.continuationToken;
setPagination((prevPagination) => ({
...prevPagination,
hasMoreResults: data.hasMoreResults,
continuationToken: newContinuationToken,
}));
} else {
setPagination((prevPagination) => ({
...prevPagination,
hasMoreResults: false,
}));
}
setCasesList((prevCases) => [...prevCases, ...data.result]);
dispatch(setAllowedClassifications(data.options));
}
},
[casesList.length, dashboardService, debouncedValue]
);
this code is fetching the data from the APi and for filtering i created an Object name Options
const options: GetCasesListOptions = {
continuationToken,
filter: [],
sort: [],
pageLimit: casesPageLimit,
search: [debouncedValue]
};
im saving my debouncedValue to the search Array in the Options object and then im using Options object in APi to filter the data
const data: IData = await dashboardService.getCasesList(options);
for example if i have 15 objects in APi, i need to get first 10 objects, and then i scroll down my callback function executes one more time and get the rest of the data...
if (data.result) {
setIsFetchingData(false)
if (data.continuationToken !== undefined) {
const newContinuationToken = data.continuationToken;
setPagination((prevPagination) => ({
...prevPagination,
hasMoreResults: data.hasMoreResults,
continuationToken: newContinuationToken,
}));
} else {
setPagination((prevPagination) => ({
...prevPagination,
hasMoreResults: false,
}));
}
setCasesList((prevCases) => [...prevCases, ...data.result]);
dispatch(setAllowedClassifications(data.options));
}
it's already done there...
now i want that, if i type something in the search box my api should run again to add my search value in the APi and filters the data...
but i'm facing problems doing this...
im calling my usecallback function like this...
useEffect(() => {
if (isFetchingMore && pagination.hasMoreResults) {
getDashboardCases(pagination.continuationToken);
}
}, [
getDashboardCases,
isFetchingMore,
pagination.continuationToken,
pagination.hasMoreResults,
debouncedValue
]);
if isFetchingMore && pagination.hasMoreResults is true, then it executes the function, but if type something in searchbox it is not running my function again...
i also tried to remove the if condition in the useEffect but it started infinite scrolling making duplicates of data, and i get this error...
Encountered two children with the same key, `d77c39f2-2dcd-4c4e-b7ee-1fde07b6583f`. Keys should be unique so that components maintain their identity across updates
so i need to re-run the function if i type something in search box and not get duplicated data back, and also i want to run the if condition that i typed in the useEffect...
please help, Thank you :)

GetStream creates same activity in different feeds

I built this piece of code to add tags to my tag feed.
buildActivity = (model,obj) => {
return {
...{
actor: `user:model.user`,
verb: 'is',
object: `model:${model.id}`,
foreign_id: `model:${model.id}`,
time: model.createdAt.toDate(),
},
...(obj ? obj : {})
}
}
addActivitiesToTagFeed = async (model,tags) => {
const promises = []
for(let i=0;i<tags.length;i++){
const tag = tags[i]
const activity = buildActivity(model,{target: `tag:${tag}`})
const feed = stream.feed('tag', tag)
promises.push(feed.addActivity(activity))
}
await Promise.all(promises)
}
Which I limited to max 3 tags. I can have like
tag:netflix
tag:films
tag:suspense
The problem is somehow the activity created by addActivity is the same to all tags. The same target, even the same activity id. It is breaking my 'tag_aggregated' that only follows one of those tags.
Ideas anyone how to fix it?

Good news
Feeds (in my case tag feed) can't have differents activities with same foreign_id + time. To fixed I changed my code
buildActivity = (model,obj) => {
return {
...{
actor: `user:model.user`,
verb: 'is',
object: `model:${model.id}`,
foreign_id: `model:${model.id}`,
time: model.createdAt.toDate(),
},
...(obj ? obj : {})
}
}
addActivitiesToTagFeed = async (model,tags) => {
const promises = []
for(let i=0;i<tags.length;i++){
const tag = tags[i]
// THIS LINE changing the foreign_id for each activity
const activity = buildActivity(model,{target: `tag:${tag}`,foreign_id: `model:${model.id}:${tag}`})
const feed = stream.feed('tag', tag)
promises.push(feed.addActivity(activity))
}
await Promise.all(promises)
}
It fixed my problem. In GetStream documentation doesn't mention it but makes sense, because foreign_id + time is a secondary key to find unique record.