Develop Reference

API call to youtube.videos.list failed with error

When I run the following JavaScript through Google Apps script with more then 100 keywords.
function youTubeSearchResults() {
// 1. Retrieve values from column "A".
const sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
const values = sheet.getRange("A2:A" + sheet.getLastRow()).getDisplayValues().filter(([a]) => a);
// 2. Retrieve your current values.
const modifyResults = values.flatMap(([keywords]) => {
const searchResults = YouTube.Search.list("id, snippet", { q: keywords, maxResults: 10, type: "video", order: "viewCount", videoDuration: "short", order: "date" });
const fSearchResults = searchResults.items.filter(function (sr) { return sr.id.kind === "youtube#video" });
return fSearchResults.map(function (sr) { return [keywords, sr.id.videoId, `https://www.youtube.com/watch?v=${sr.id.videoId}`, sr.snippet.title, sr.snippet.publishedAt, sr.snippet.channelTitle, sr.snippet.channelId, `https://www.youtube.com/channel/${sr.snippet.channelId}`, sr.snippet.thumbnails.high.url] });
});
// 3. Retrieve viewCounts and subscriberCounts.
const { videoIds, channelIds } = modifyResults.reduce((o, r) => {
o.videoIds.push(r[1]);
o.channelIds.push(r[6]);
return o;
}, { videoIds: [], channelIds: [] });
const limit = 50;
const { viewCounts, subscriberCounts } = [...Array(Math.ceil(videoIds.length / limit))].reduce((obj, _) => {
const vIds = videoIds.splice(0, limit);
const cIds = channelIds.splice(0, limit);
const res1 = YouTube.Videos.list(["statistics"], { id: vIds, maxResults: limit }).items.map(({ statistics: { viewCount } }) => viewCount);
const obj2 = YouTube.Channels.list(["statistics"], { id: cIds, maxResults: limit }).items.reduce((o, { id, statistics: { subscriberCount } }) => (o[id] = subscriberCount, o), {});
const res2 = cIds.map(e => obj2[e] || null);
obj.viewCounts = [...obj.viewCounts, ...res1];
obj.subscriberCounts = [...obj.subscriberCounts, ...res2];
return obj;
}, { viewCounts: [], subscriberCounts: [] });
const ar = [viewCounts, subscriberCounts];
const rr = ar[0].map((_, c) => ar.map(r => r[c]));
// 4. Merge data.
const res = modifyResults.map((r, i) => [...r, ...rr[i]]);
// 5. Put values on Spreadsheet.
sheet.getRange(2, 2, res.length, res[0].length).setValues(res);
}
it gives me that error
GoogleJsonResponseException: API call to youtube.videos.list failed with error:
The request cannot be completed because you have exceeded your quota.
reduce.viewCounts #code.gs:23
youTubeSearchResults #code.gs:20
I know YouTube have data call limits for example you can call the results of not more then 50 video ids at one time but if you have 1000 video ids in your sheet you can run then loop for first 50 then next so on. Is it anything like that I can do with search results too.
Please help me understand how can I fix this issue.

Note that the endpoint the most expensive in your script is the Search: list one which costs 100 of your 10,000 quota (you can have a look to other endpoint costs here).
You may be interested in the standalone quota-free solution that consists in reverse-engineering the YouTube UI search feature.
Otherwise a temporary solution to Google audit consists in using my no-key service.
With my no-key service:
const searchResults = YouTube.Search.list("id, snippet", { q: keywords, maxResults: 10, type: "video", order: "viewCount", videoDuration: "short", order: "date" });
would become:
const searchResults = JSON.parse(UrlFetchApp.fetch(`https://yt.lemnoslife.com/noKey/search?part=snippet&q=${keywords}&maxResults=10&type=video&order=viewCount&videoDuration=short`).getContentText())
As part=id doesn't add more data to the response and AFAIK using two order isn't supported by YouTube Data API v3.

Pagination in TypeORM/NestJS

I have to introduce pagination in findAll() method. I really dont know how to do it. I tried but it is giving so many errors. I used findAndCount() method given by typeorm for that, But I am not sure how it will work.
As of now below method returning all the record. I need to return at a time 10 records. Please suggest what modification I need to do.
async findAll(queryCertificateDto: QueryCertificateDto,page=1): Promise<PaginatedResult> {
let { country, sponser } = queryCertificateDto;
const query = this.certificateRepository.createQueryBuilder('certificate');
if (sponser) {
sponser = sponser.toUpperCase();
query.andWhere('Upper(certificate.sponser)=:sponser', { sponser });
}
if (country) {
country = country.toUpperCase();
query.andWhere('Upper(certificate.country)=:country', { country });
}
const certificates = query.getMany();
return certificates;
}
this is PaginatedResult file.
export class PaginatedResult {
data: any[];
meta: {
total: number;
page: number;
last_page: number;
};
}
I tried changing code of findAll() but where clause is giving error. I am not sure how to handle query.getMany() in pagination.
const take = query.take || 10
const skip = query.skip || 0
const [result, total] = await this.certificateRepository.findAndCount(
{
where: query.getMany(), //this is giving error
take:take,
skip:skip
}
);
return result;
I need to introduce pagination in this method. Any help will be really helpful.

Typeorm has a really nice method specific to your usecase findAndCount
async findAll(queryCertificateDto: QueryCertificateDto): Promise<PaginatedResult> {
const take = queryCertificateDto.take || 10
const skip = queryCertificateDto.skip || 0
const country = queryCertificateDto.keyword || ''
const sponser = queryCertificateDto.sponser || ''
const query = this.certificateRepository.createQueryBuilder('certificate');
const [result, total] = await this.certificateRepository.findAndCount(
{
where: { country: Like('%' + country + '%') AND sponser: Like('%' + sponser + '%') }, order: { name: "DESC" },
take: take,
skip: skip
}
);
return {
data: result,
count: total
};
}
More documentation about Repository class can be found here

You don't need the .getMany() with your where in the last code, the result is an array of the data you need.
From your first code, you can do this:
async findAll(queryCertificateDto: QueryCertificateDto,page=1): Promise<PaginatedResult> {
// let's say limit and offset are passed here too
let { country, sponser, limit, offset } = queryCertificateDto;
const query = this.certificateRepository.createQueryBuilder('certificate');
if (sponser) {
sponser = sponser.toUpperCase();
query.andWhere('certificate.sponser = :sponser', { sponser });
}
if (country) {
country = country.toUpperCase();
query.andWhere('certificate.country = :country', { country });
}
// limit and take mean the same thing, while skip and offset mean the same thing
const certificates = await query
.orderBy("certificate.id", "ASC")
.limit(limit || 10)
.offset(offset || 0)
.getMany();
// if you want to count just replace the `.getMany()` with `.getManyandCount()`;
return certificates;
}```

how to indicate loaging/searching on Office FabricUI TagPicker for large data sets?

I am using the TagPicker to get data dynamically and present a set of the results that matches with the term. The issue is that looking into the docs there is not clear indication how to determine that the component data is loading or searching. The interface that had those was dropped (ISuggestionsProps) and the loadingText prop does not seem to work for me or I am probably using it wrong.
here is how I was able to load data from a list into the tagpicker:
const filterSuggestedTags = async (filterText: string, tagList: ITag[]) => {
//* possibly here to call an api if needed?
if (filterText) {
const url = 'url'
const resp = await fetch(url,{method:'GET',headers:{Accept:'application/json; odata=verbose'}})
return (await resp.json()).d.results.map(item => ({ key: item, name: item.Title }));
} else return []
};
codepen:
https://codepen.io/deleite/pen/MWjBMjY?editors=1111
This obviously has a lot of problems, first and the worst every keystroke is a promise fired. So, question is how to call an api using the search term and result the suggestions?
Thank you all.

I am ussing Office ui Fabric react v5 ("office-ui-fabric-react": "^5.135.5").
I am using TagPicker to laod external API data (long resolve time, large data set).
Loading suggestions is delayed for 700ms (after key pressed).
Loading suggestions is fired after 3 chars are typed.
During loading there is loading circle visible. I am loading suggestion in pages for 20 suggestions, if there is more items to be loaded on the bottom there is Loading more anchor which loads another page and add new suggestions to already loaded. I had to extend IBasePickerSuggestionsProps interface for moreSuggestionsAvailable?: boolean; in BasePicker.types.d.ts based on this issue: https://github.com/microsoft/fluentui/issues/6582
Doc: https://developer.microsoft.com/en-us/fluentui#/components/pickers
Codepen: https://codepen.io/matej4386/pen/ZEpqwQv
Here is my code:
const {disabled} = this.props;
const {
selectedItems,
errorMessage
} = this.state;
<TagPicker
onResolveSuggestions={this.onFilterChanged}
getTextFromItem={this.getTextFromItem}
resolveDelay={700}
pickerSuggestionsProps={{
suggestionsHeaderText: strings.suggestionsHeaderText,
noResultsFoundText: strings.noresultsFoundText,
searchForMoreText: strings.moreSuggestions,
moreSuggestionsAvailable: this.state.loadmore
}}
onGetMoreResults={this.onGetMoreResults}
onRenderSuggestionsItem={this.onRenderSuggestionsItem}
selectedItems={selectedItems}
onChange={this.onItemChanged}
itemLimit={1}
disabled={disabled}
inputProps={{
placeholder: strings.TextFormFieldPlaceholder
}}
/>
private onFilterChanged = async (filterText: string, tagList:IPickerItem[]) => {
if (filterText.length >= 3) {
let resolvedSugestions: IPickerItem[] = await this.loadListItems(filterText);
const {
selectedItems
} = this.state;
// Filter out the already retrieved items, so that they cannot be selected again
if (selectedItems && selectedItems.length > 0) {
let filteredSuggestions = [];
for (const suggestion of resolvedSugestions) {
const exists = selectedItems.filter(sItem => sItem.key === suggestion.key);
if (!exists || exists.length === 0) {
filteredSuggestions.push(suggestion);
}
}
resolvedSugestions = filteredSuggestions;
}
if (resolvedSugestions) {
this.setState({
errorMessage: "",
showError: false,
suggestions: resolvedSugestions,
loadmore: true,
loadMorePageNumber: 1
});
return resolvedSugestions;
} else {
return [];
}
} else {
return null
}
}
private onGetMoreResults = async (filterText: string, selectedItems?: any[]): Promise<IPickerItem[]> => {
let arrayItems: IPickerItem[] = [];
try {
let listItems: IOrganization[] = await this.GetOrganizations(this.Identity[0].id, filterText, 1);
...
private loadListItems = async (filterText: string): Promise<IPickerItem[]> => {
let { webUrl, filter, substringSearch } = this.props;
let arrayItems: IPickerItem[] = [];
try {
...

Ok, you can mitigate this problem with the prop 'resolveDelay' but still I did not find any standard way to handle items from an api this is the closest I came up.
Any samples or ideas would be appreciated.

React - Check for duplicates in an array

I'm working on simple list where you can simply add your words to the list.
Main problem is duplicates, I tried many solutions but they weren't even close.
state = {
people: [{ name: null, count: null }]
}
handleSubmit = (e) => {
this.setState(({ count }) => ({
count: count + 1
}));
this.props.addHuman(this.state);
}
addHuman = (human) => {
let people = [...this.state.people, human];
this.setState({
people: people
});
}
I hope for solution which will check if there is any duplicate already in the array

You could make a check if there is someone with the same name already in the array. A better property to check would be an email adresse.
find takes a callback function as parameter. Inside this function, I compare the name properties. If it's a match, find returns true, then a do an early return in the next line and the person isn't added.
addHuman = (human) => {
const exists = this.state.people.find(p => p.name === human.name);
if (exists) return;
let people = [...this.state.people, human];
this.setState({
people: people
});
}
https://developer.mozilla.org/de/docs/Web/JavaScript/Reference/Global_Objects/Array/find

if you want to de duplicate the array...you can do it in this way
cont array = [1,1.2,3,5,5];
[...new Set(array)]

This will work as well, there are plenty of ways to achieve the desired outcome.
addHuman = human => {
const updatedPeople = people.includes(human) ? people : [ ...people, human ];
this.setState({
people: updatedPeople
});
}

Remove_Duplicate_recs(filteredRecord)
{
var records = [];
for (let record of filteredRecord)
{
const duplicate_recorde_exists = records.find(r => r.MovementId ===
record.MovementId);
if (duplicate_recorde_exists)
{
continue ;
}
else
{
records.push(record);
}
}
return records;
}

Optimize repeated expensive computations in Rxjs given an Observable of Array

Given the following script:
type Data = {
id: string
value: number
}
// This is a pure function: given val, res will always be the same
const veryExpensiveCalc = (val: number) => {
const res = // ... 5 seconds of sync computation ...
return res
}
const array$ = new ReplaySubject<Array<Data>>(1)
const allComputedValues$ = array$.pipe(
map(arr =>
arr.map(item =>
// I want this veryExpensiveCalc to be performed only when
// item.value changes for this item.id, or the item was not
// there before
veryExpensiveCalc(item.value)
)
)
)
allComputedValues$
.pipe(
tap(newStruct => {
console.log('newStruct', newStruct)
})
)
.subscribe()
I'd like to optimize how allComputedValues$ is calculated. In particular let's say that an item is added or removed to array$, or the order of the elements change: then veryExpensiveCalc is executed again on every item of the array, even though that's not needed at all. I just need allComputedValues$ from the previous computation with the result of veryExpensiveCalc applied only to the newly added element (in case of addition).
What's the best way to write this in a functional reactive fashion? It should work also if there are multiple elements edited/added/removed at the same time (so that veryExpensiveCalc is executed only on the elements that have a different value given the same id).

I think you can use the scan() operator here:
const allComputedValues$ = array$.pipe(
scan(
(acc, crtArr) => {
// with these we 'automatically' remove the current items(from `acc`) whose ids are not in the current array
const newIds = {};
const newValues = {};
crtArr.forEach(crt => {
const { id: currentId, value: crtValue } = crt;
// if new or edited
if (!acc.ids[currentId] || acc.values[currentId] !== crtValue) {
newIds[currentId] = true;
newValues[currentId] = veryExpensiveCalc(crtValue);
}
});
return { ids: newIds, values: newValues };
},
{ ids: {}, values: {} },
),
)