Develop Reference

API call to youtube.videos.list failed with error

When I run the following JavaScript through Google Apps script with more then 100 keywords.
function youTubeSearchResults() {
// 1. Retrieve values from column "A".
const sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
const values = sheet.getRange("A2:A" + sheet.getLastRow()).getDisplayValues().filter(([a]) => a);
// 2. Retrieve your current values.
const modifyResults = values.flatMap(([keywords]) => {
const searchResults = YouTube.Search.list("id, snippet", { q: keywords, maxResults: 10, type: "video", order: "viewCount", videoDuration: "short", order: "date" });
const fSearchResults = searchResults.items.filter(function (sr) { return sr.id.kind === "youtube#video" });
return fSearchResults.map(function (sr) { return [keywords, sr.id.videoId, `https://www.youtube.com/watch?v=${sr.id.videoId}`, sr.snippet.title, sr.snippet.publishedAt, sr.snippet.channelTitle, sr.snippet.channelId, `https://www.youtube.com/channel/${sr.snippet.channelId}`, sr.snippet.thumbnails.high.url] });
});
// 3. Retrieve viewCounts and subscriberCounts.
const { videoIds, channelIds } = modifyResults.reduce((o, r) => {
o.videoIds.push(r[1]);
o.channelIds.push(r[6]);
return o;
}, { videoIds: [], channelIds: [] });
const limit = 50;
const { viewCounts, subscriberCounts } = [...Array(Math.ceil(videoIds.length / limit))].reduce((obj, _) => {
const vIds = videoIds.splice(0, limit);
const cIds = channelIds.splice(0, limit);
const res1 = YouTube.Videos.list(["statistics"], { id: vIds, maxResults: limit }).items.map(({ statistics: { viewCount } }) => viewCount);
const obj2 = YouTube.Channels.list(["statistics"], { id: cIds, maxResults: limit }).items.reduce((o, { id, statistics: { subscriberCount } }) => (o[id] = subscriberCount, o), {});
const res2 = cIds.map(e => obj2[e] || null);
obj.viewCounts = [...obj.viewCounts, ...res1];
obj.subscriberCounts = [...obj.subscriberCounts, ...res2];
return obj;
}, { viewCounts: [], subscriberCounts: [] });
const ar = [viewCounts, subscriberCounts];
const rr = ar[0].map((_, c) => ar.map(r => r[c]));
// 4. Merge data.
const res = modifyResults.map((r, i) => [...r, ...rr[i]]);
// 5. Put values on Spreadsheet.
sheet.getRange(2, 2, res.length, res[0].length).setValues(res);
}
it gives me that error
GoogleJsonResponseException: API call to youtube.videos.list failed with error:
The request cannot be completed because you have exceeded your quota.
reduce.viewCounts #code.gs:23
youTubeSearchResults #code.gs:20
I know YouTube have data call limits for example you can call the results of not more then 50 video ids at one time but if you have 1000 video ids in your sheet you can run then loop for first 50 then next so on. Is it anything like that I can do with search results too.
Please help me understand how can I fix this issue.

Note that the endpoint the most expensive in your script is the Search: list one which costs 100 of your 10,000 quota (you can have a look to other endpoint costs here).
You may be interested in the standalone quota-free solution that consists in reverse-engineering the YouTube UI search feature.
Otherwise a temporary solution to Google audit consists in using my no-key service.
With my no-key service:
const searchResults = YouTube.Search.list("id, snippet", { q: keywords, maxResults: 10, type: "video", order: "viewCount", videoDuration: "short", order: "date" });
would become:
const searchResults = JSON.parse(UrlFetchApp.fetch(`https://yt.lemnoslife.com/noKey/search?part=snippet&q=${keywords}&maxResults=10&type=video&order=viewCount&videoDuration=short`).getContentText())
As part=id doesn't add more data to the response and AFAIK using two order isn't supported by YouTube Data API v3.

Javascript Conditional return object

I used aws-sns to create one webhook. Two lambda functions are checked by this webhook. One of the lambda functions publishes 'orderId' and'startTime', while another publishes 'orderId' and 'roundName'. Both lambdas fire at different times. As a result, publishing can happen at two different times. One or both of the'startTime' and 'roundName' parameters may be undefined.
If 'roundName' exists, the 'updateOrder' variable will return 'roundName,' and the database will be updated. When'startTime' is set and 'roundName' is left blank, the 'roundName' will be rewritten from the database, which I don't want. Because if there is a 'roundName,' there will always be a 'roundName,' the value of 'roundName' can change but it will never be undefined.If startTime changes as well as roundName change then it will update the database. But my current logic is wrong. Struggling to implementing diffrent scenario logic.
const data = {
Records: [
{
Sns: {
Message:
'[{\n "orderId": "a4013438-926f-4fdc-8f6a-a7aa402b40ea",\n "roundName": "RO1"}]',
},
},
],
};
const existingData = [
{
modifiedAt: "2022-03-09T13:18:06.211Z",
lastMile: "progress",
createdAt: "2022-02-26T06:38:50.967+00:00",
orderId: "a4013438-926f-4fdc-8f6a-a7aa402b40ea",
},
];
// parse the data
const parseData = data.Records.flatMap((record) =>
JSON.parse(record.Sns.Message)
);
// check if the data exist or not
const existingOrder = existingData.filter(
(o1) => parseData.some((o2) => o1.orderId === o2.orderId)
);
// if there is no existingOrder then return false
if (existingOrder.length === 0) return;
// if there is exisiting order then add roundName and startTime from SNS event
const updateOrder = existingOrder.map((i) => {
const roundName = parseData.find((r) => {
return r.orderId === i.orderId;
}).roundName;
const startTime = parseData.find((r) => {
return r.orderId === i.orderId;
}).startTime;
return {
roundName: roundName ?? "",
startTime: startTime ?? "",
};
});
console.log(updateOrder);

Pagination in TypeORM/NestJS

I have to introduce pagination in findAll() method. I really dont know how to do it. I tried but it is giving so many errors. I used findAndCount() method given by typeorm for that, But I am not sure how it will work.
As of now below method returning all the record. I need to return at a time 10 records. Please suggest what modification I need to do.
async findAll(queryCertificateDto: QueryCertificateDto,page=1): Promise<PaginatedResult> {
let { country, sponser } = queryCertificateDto;
const query = this.certificateRepository.createQueryBuilder('certificate');
if (sponser) {
sponser = sponser.toUpperCase();
query.andWhere('Upper(certificate.sponser)=:sponser', { sponser });
}
if (country) {
country = country.toUpperCase();
query.andWhere('Upper(certificate.country)=:country', { country });
}
const certificates = query.getMany();
return certificates;
}
this is PaginatedResult file.
export class PaginatedResult {
data: any[];
meta: {
total: number;
page: number;
last_page: number;
};
}
I tried changing code of findAll() but where clause is giving error. I am not sure how to handle query.getMany() in pagination.
const take = query.take || 10
const skip = query.skip || 0
const [result, total] = await this.certificateRepository.findAndCount(
{
where: query.getMany(), //this is giving error
take:take,
skip:skip
}
);
return result;
I need to introduce pagination in this method. Any help will be really helpful.

Typeorm has a really nice method specific to your usecase findAndCount
async findAll(queryCertificateDto: QueryCertificateDto): Promise<PaginatedResult> {
const take = queryCertificateDto.take || 10
const skip = queryCertificateDto.skip || 0
const country = queryCertificateDto.keyword || ''
const sponser = queryCertificateDto.sponser || ''
const query = this.certificateRepository.createQueryBuilder('certificate');
const [result, total] = await this.certificateRepository.findAndCount(
{
where: { country: Like('%' + country + '%') AND sponser: Like('%' + sponser + '%') }, order: { name: "DESC" },
take: take,
skip: skip
}
);
return {
data: result,
count: total
};
}
More documentation about Repository class can be found here

You don't need the .getMany() with your where in the last code, the result is an array of the data you need.
From your first code, you can do this:
async findAll(queryCertificateDto: QueryCertificateDto,page=1): Promise<PaginatedResult> {
// let's say limit and offset are passed here too
let { country, sponser, limit, offset } = queryCertificateDto;
const query = this.certificateRepository.createQueryBuilder('certificate');
if (sponser) {
sponser = sponser.toUpperCase();
query.andWhere('certificate.sponser = :sponser', { sponser });
}
if (country) {
country = country.toUpperCase();
query.andWhere('certificate.country = :country', { country });
}
// limit and take mean the same thing, while skip and offset mean the same thing
const certificates = await query
.orderBy("certificate.id", "ASC")
.limit(limit || 10)
.offset(offset || 0)
.getMany();
// if you want to count just replace the `.getMany()` with `.getManyandCount()`;
return certificates;
}```

GetStream creates same activity in different feeds

I built this piece of code to add tags to my tag feed.
buildActivity = (model,obj) => {
return {
...{
actor: `user:model.user`,
verb: 'is',
object: `model:${model.id}`,
foreign_id: `model:${model.id}`,
time: model.createdAt.toDate(),
},
...(obj ? obj : {})
}
}
addActivitiesToTagFeed = async (model,tags) => {
const promises = []
for(let i=0;i<tags.length;i++){
const tag = tags[i]
const activity = buildActivity(model,{target: `tag:${tag}`})
const feed = stream.feed('tag', tag)
promises.push(feed.addActivity(activity))
}
await Promise.all(promises)
}
Which I limited to max 3 tags. I can have like
tag:netflix
tag:films
tag:suspense
The problem is somehow the activity created by addActivity is the same to all tags. The same target, even the same activity id. It is breaking my 'tag_aggregated' that only follows one of those tags.
Ideas anyone how to fix it?

Good news
Feeds (in my case tag feed) can't have differents activities with same foreign_id + time. To fixed I changed my code
buildActivity = (model,obj) => {
return {
...{
actor: `user:model.user`,
verb: 'is',
object: `model:${model.id}`,
foreign_id: `model:${model.id}`,
time: model.createdAt.toDate(),
},
...(obj ? obj : {})
}
}
addActivitiesToTagFeed = async (model,tags) => {
const promises = []
for(let i=0;i<tags.length;i++){
const tag = tags[i]
// THIS LINE changing the foreign_id for each activity
const activity = buildActivity(model,{target: `tag:${tag}`,foreign_id: `model:${model.id}:${tag}`})
const feed = stream.feed('tag', tag)
promises.push(feed.addActivity(activity))
}
await Promise.all(promises)
}
It fixed my problem. In GetStream documentation doesn't mention it but makes sense, because foreign_id + time is a secondary key to find unique record.

how to indicate loaging/searching on Office FabricUI TagPicker for large data sets?

I am using the TagPicker to get data dynamically and present a set of the results that matches with the term. The issue is that looking into the docs there is not clear indication how to determine that the component data is loading or searching. The interface that had those was dropped (ISuggestionsProps) and the loadingText prop does not seem to work for me or I am probably using it wrong.
here is how I was able to load data from a list into the tagpicker:
const filterSuggestedTags = async (filterText: string, tagList: ITag[]) => {
//* possibly here to call an api if needed?
if (filterText) {
const url = 'url'
const resp = await fetch(url,{method:'GET',headers:{Accept:'application/json; odata=verbose'}})
return (await resp.json()).d.results.map(item => ({ key: item, name: item.Title }));
} else return []
};
codepen:
https://codepen.io/deleite/pen/MWjBMjY?editors=1111
This obviously has a lot of problems, first and the worst every keystroke is a promise fired. So, question is how to call an api using the search term and result the suggestions?
Thank you all.

I am ussing Office ui Fabric react v5 ("office-ui-fabric-react": "^5.135.5").
I am using TagPicker to laod external API data (long resolve time, large data set).
Loading suggestions is delayed for 700ms (after key pressed).
Loading suggestions is fired after 3 chars are typed.
During loading there is loading circle visible. I am loading suggestion in pages for 20 suggestions, if there is more items to be loaded on the bottom there is Loading more anchor which loads another page and add new suggestions to already loaded. I had to extend IBasePickerSuggestionsProps interface for moreSuggestionsAvailable?: boolean; in BasePicker.types.d.ts based on this issue: https://github.com/microsoft/fluentui/issues/6582
Doc: https://developer.microsoft.com/en-us/fluentui#/components/pickers
Codepen: https://codepen.io/matej4386/pen/ZEpqwQv
Here is my code:
const {disabled} = this.props;
const {
selectedItems,
errorMessage
} = this.state;
<TagPicker
onResolveSuggestions={this.onFilterChanged}
getTextFromItem={this.getTextFromItem}
resolveDelay={700}
pickerSuggestionsProps={{
suggestionsHeaderText: strings.suggestionsHeaderText,
noResultsFoundText: strings.noresultsFoundText,
searchForMoreText: strings.moreSuggestions,
moreSuggestionsAvailable: this.state.loadmore
}}
onGetMoreResults={this.onGetMoreResults}
onRenderSuggestionsItem={this.onRenderSuggestionsItem}
selectedItems={selectedItems}
onChange={this.onItemChanged}
itemLimit={1}
disabled={disabled}
inputProps={{
placeholder: strings.TextFormFieldPlaceholder
}}
/>
private onFilterChanged = async (filterText: string, tagList:IPickerItem[]) => {
if (filterText.length >= 3) {
let resolvedSugestions: IPickerItem[] = await this.loadListItems(filterText);
const {
selectedItems
} = this.state;
// Filter out the already retrieved items, so that they cannot be selected again
if (selectedItems && selectedItems.length > 0) {
let filteredSuggestions = [];
for (const suggestion of resolvedSugestions) {
const exists = selectedItems.filter(sItem => sItem.key === suggestion.key);
if (!exists || exists.length === 0) {
filteredSuggestions.push(suggestion);
}
}
resolvedSugestions = filteredSuggestions;
}
if (resolvedSugestions) {
this.setState({
errorMessage: "",
showError: false,
suggestions: resolvedSugestions,
loadmore: true,
loadMorePageNumber: 1
});
return resolvedSugestions;
} else {
return [];
}
} else {
return null
}
}
private onGetMoreResults = async (filterText: string, selectedItems?: any[]): Promise<IPickerItem[]> => {
let arrayItems: IPickerItem[] = [];
try {
let listItems: IOrganization[] = await this.GetOrganizations(this.Identity[0].id, filterText, 1);
...
private loadListItems = async (filterText: string): Promise<IPickerItem[]> => {
let { webUrl, filter, substringSearch } = this.props;
let arrayItems: IPickerItem[] = [];
try {
...

Ok, you can mitigate this problem with the prop 'resolveDelay' but still I did not find any standard way to handle items from an api this is the closest I came up.
Any samples or ideas would be appreciated.