Develop Reference

API call to youtube.videos.list failed with error

When I run the following JavaScript through Google Apps script with more then 100 keywords.
function youTubeSearchResults() {
// 1. Retrieve values from column "A".
const sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
const values = sheet.getRange("A2:A" + sheet.getLastRow()).getDisplayValues().filter(([a]) => a);
// 2. Retrieve your current values.
const modifyResults = values.flatMap(([keywords]) => {
const searchResults = YouTube.Search.list("id, snippet", { q: keywords, maxResults: 10, type: "video", order: "viewCount", videoDuration: "short", order: "date" });
const fSearchResults = searchResults.items.filter(function (sr) { return sr.id.kind === "youtube#video" });
return fSearchResults.map(function (sr) { return [keywords, sr.id.videoId, `https://www.youtube.com/watch?v=${sr.id.videoId}`, sr.snippet.title, sr.snippet.publishedAt, sr.snippet.channelTitle, sr.snippet.channelId, `https://www.youtube.com/channel/${sr.snippet.channelId}`, sr.snippet.thumbnails.high.url] });
});
// 3. Retrieve viewCounts and subscriberCounts.
const { videoIds, channelIds } = modifyResults.reduce((o, r) => {
o.videoIds.push(r[1]);
o.channelIds.push(r[6]);
return o;
}, { videoIds: [], channelIds: [] });
const limit = 50;
const { viewCounts, subscriberCounts } = [...Array(Math.ceil(videoIds.length / limit))].reduce((obj, _) => {
const vIds = videoIds.splice(0, limit);
const cIds = channelIds.splice(0, limit);
const res1 = YouTube.Videos.list(["statistics"], { id: vIds, maxResults: limit }).items.map(({ statistics: { viewCount } }) => viewCount);
const obj2 = YouTube.Channels.list(["statistics"], { id: cIds, maxResults: limit }).items.reduce((o, { id, statistics: { subscriberCount } }) => (o[id] = subscriberCount, o), {});
const res2 = cIds.map(e => obj2[e] || null);
obj.viewCounts = [...obj.viewCounts, ...res1];
obj.subscriberCounts = [...obj.subscriberCounts, ...res2];
return obj;
}, { viewCounts: [], subscriberCounts: [] });
const ar = [viewCounts, subscriberCounts];
const rr = ar[0].map((_, c) => ar.map(r => r[c]));
// 4. Merge data.
const res = modifyResults.map((r, i) => [...r, ...rr[i]]);
// 5. Put values on Spreadsheet.
sheet.getRange(2, 2, res.length, res[0].length).setValues(res);
}
it gives me that error
GoogleJsonResponseException: API call to youtube.videos.list failed with error:
The request cannot be completed because you have exceeded your quota.
reduce.viewCounts #code.gs:23
youTubeSearchResults #code.gs:20
I know YouTube have data call limits for example you can call the results of not more then 50 video ids at one time but if you have 1000 video ids in your sheet you can run then loop for first 50 then next so on. Is it anything like that I can do with search results too.
Please help me understand how can I fix this issue.

Note that the endpoint the most expensive in your script is the Search: list one which costs 100 of your 10,000 quota (you can have a look to other endpoint costs here).
You may be interested in the standalone quota-free solution that consists in reverse-engineering the YouTube UI search feature.
Otherwise a temporary solution to Google audit consists in using my no-key service.
With my no-key service:
const searchResults = YouTube.Search.list("id, snippet", { q: keywords, maxResults: 10, type: "video", order: "viewCount", videoDuration: "short", order: "date" });
would become:
const searchResults = JSON.parse(UrlFetchApp.fetch(`https://yt.lemnoslife.com/noKey/search?part=snippet&q=${keywords}&maxResults=10&type=video&order=viewCount&videoDuration=short`).getContentText())
As part=id doesn't add more data to the response and AFAIK using two order isn't supported by YouTube Data API v3.

Do I need to use PrevState even if I spread the state into a variable?

I am testing some code to try and understand the race condition regarding the use of setState().
my code can be found here
my code below:
import React from "react";
export default class App extends React.Component {
state = {
id: "",
ids: [{ id: 7 }, { id: 14 }]
};
// here is where I create the id numbers
uniqueIdCreatorHandler = incrementAmount => {
let ids = [...this.state.ids];
let highestId = 0;
if (ids.length > 0) {
highestId = ids
.map(value => {
return value.id;
})
.reduce((a, b) => {
return Math.max(a, b);
});
}
let newId = highestId + incrementAmount;
ids.push({ id: newId });
this.setState({ ids: ids });
};
idDeleterHanlder = currentIndex => {
let ids = this.state.ids;
ids.splice(currentIndex, 1);
this.setState({ ids: ids });
};
//below is when I test performing the function twice, in order to figure if the result would be a race condition
double = (firstIncrementAmount, secondIncrementAmount) => {
this.uniqueIdCreatorHandler(firstIncrementAmount);
this.uniqueIdCreatorHandler(secondIncrementAmount);
};
render() {
let ids = this.state.ids.map((id, index) => {
return (
<p onClick={() => this.idDeleterHanlder(index)} key={id.id}>
id:{id.id}
</p>
);
});
return (
<div className="App">
<button onClick={() => this.uniqueIdCreatorHandler(1)}>
Push new id
</button>
<button onClick={() => this.double(1, 2)}>Add some Ids</button>
<p>all ids below:</p>
{ids}
</div>
);
}
}
when invoking the double function on the second button only the secondIncrementAmount works. You can test it by changing the argument values on the call made on the onClick method.
I think that I should somehow use prevState on this.setState in order to fix this.
How could I avoid this issue here? This matter started at CodeReview but I did not realize how could I fix this.
There is also a recommendation to spread the mapped ids into Math.max and I could not figure out how and Why to do it. Isn't the creation of the new array by mapping the spreaded key values safe enough?

.splice and .push mutate the array. Thus the current state then does not match the currently rendered version anymore. Instead, use .slice (or .filter) and [...old, new] for immutable stateupdates:
deleteId = index => {
this.setState(({ ids }) => ({ ids: ids.filter((id, i) => i !== index) }));
};
uniqueIdCreatorHandler = increment => {
const highest = Math.max(0, ...this.state.ids.map(it => it.id ));
this.setState(({ ids }) => ({ ids: [...ids, { id: highest + increment }] }));
};

setState can be asynchronous, batching up multiple changes and then applying them all at once. So when you spread the state you might be spreading an old version of the state and throwing out a change that should have happened.
The function version of setState avoids this. React guarantees that you will be passed in the most recent state, even if there's some other state update that you didn't know about. And then you can product the new state based on that.
There is also a recommendation to spread the mapped ids into Math.max and I could not figure out how and Why to do it
That's just to simplify the code for finding the max. Math.max can be passed an abitrary number of arguments, rather than just two at a time, so you don't need to use reduce to get the maximum of an array.
uniqueIdCreatorHandler = incrementAmount => {
this.setState(prevState => {
let ids = [...prevState.ids];
let highestId = Math.max(...ids.map(value => value.id));
let newId = highestId + incrementAmount;
ids.push({ id: newId });
this.setState({ ids: ids });
});
};

This isn't the most elegant solution but you can pass a callback to setState(see https://reactjs.org/docs/react-component.html#setstate).
If you modify uniqueIdCreatorHandler like this:
uniqueIdCreatorHandler = (incrementAmount, next) => {
let ids = [...this.state.ids];
let highestId = 0;
if (ids.length > 0) {
highestId = ids
.map(value => {
return value.id;
})
.reduce((a, b) => {
return Math.max(a, b);
});
}
let newId = highestId + incrementAmount;
ids.push({ id: newId });
this.setState({ ids: ids }, next); //next will be called once the setState is finished
};
You can call it inside double like this.
double = (firstIncrementAmount, secondIncrementAmount) => {
this.uniqueIdCreatorHandler(
firstIncrementAmount,
() => this.uniqueIdCreatorHandler(secondIncrementAmount)
);
};

Assigning values received from api calls for each element in the loop iteration using Observables

I have a foreach loop through which I am iterating and want to call functions which would in turn make async api calls and return value which can be rendered in the html.
The 1st function call getCurrentValue() would return currentTemperatureRef which I finally want to assign receivedCurrentValue and render in html
The 2nd function call getDesiredValue1() would return desiredValueToBeReturned which I finally want to assign receivedDesiredValue1 and render in html
ts
myObj = { droppedItem: [] };
elements = [
{ elementId: 1, elementName: "name1" },
{ elementId: 2, elementName: "name2" },
{ elementId: 3, elementName: "name3" },
{ elementId: 4, elementName: "name4" }
];
this.elements.forEach(element => {
let receivedCurrentValue = this.getCurrentValue(element.name);
let receivedDesiredValue1 = this.getDesiredValue1(element.id);
this.myObj.droppedItem.push(receivedCurrentValue)
this.myObj.droppedItem.push(receivedDesiredValue1)
}
getCurrentValue(eleName){
//1st api(async api call)
var ref = this.helperService.getPointIdbyTags(this.existingObj, ['current',
'temp'], eleName)[0];
//2nd api(async api call which expects ref value from above api call)
this.siteService.getHisPointData(ref, 'current')
.pipe(
map(this.helperService.stripHaystackTypeMapping),
)
.subscribe(({ rows }) => {
if (rows.length > 0) {
this.currentTemperatureRef = rows[0].val;
}
});
}
getDesiredValue1(eleId){
//1st async api call
this.siteService.getScheduleParamsByRoomRef('temp and air and desired and
heating', eleId)
.subscribe(function (a) {
let row = a;
let pointId = this.helperService.stripHaystackTypeMapping(row['id']).split(' ')[0];
//2nd async api call expecting pointId from above api call
this.siteService.getHisPointData(pointId, 'current')
.subscribe(function (a) {
let rows = a.rows,
if (rows.length > 0) {
let desiredValueToBeReturned = rows[0].val;
)
}
)
}
}
html
<div *ngFor="let existingItem of myObj?.droppedItem">
<span>{{existingItem.receivedValue}}</span>
<span>{{existingItem.receivedDesiredValue1}}</span>
<span>{{existingItem.receivedDesiredValue2}}</span>
</div>
Update
when I try to
getCurrentValue(eleName){
let roomObj = this.getRoomObj(eleName);
let equipRef = roomObj.map(equip => equip.entities.filter(entity => entity.entities.length > 0)[0])[0];
return this.helperService.getPointIdbyTags(this.buildings, ['current',
'temp'], equipRef.referenceIDs.room)[0].pipe(switchMap((res:any)=>{
//we don't want "res" else the response of
return this.siteService.getHisPointData(res, 'current')
.pipe(
map(this.helperService.stripHaystackTypeMapping),
)
}));
}
I get an error on line => return this.helperService.getPointIdbyTags(this.buildings, ['current',
'temp'], equipRef.referenceIDs.room)[0].pipe(switchMap(
ERROR TypeError: this.helperService.getPointIdbyTags(...)[0].pipe is
not a function

I don't undestand so much the question, but you need understand some about forkJoin and switchMap. SwitchMap it's usefull when you need make two calls one depending the response of the another. The construction becomes like
callOne.pipe(
switchMap(resposeOfCallOne=>{
return callTwo(responseOfCallOne)
})
If subscribe you received the response of callTwo
forkJoin get an array of calls and return the result in an array
forkJoin([callOne,callTwo])
if subscribe you received an array: res[0] has the response of callOne and res[1] the response of callTwo
Well, First convert your functions getCurrentValue and getDesiredValue1 to return observables
getCurrentValue(eleName){
return this.helperService.getPointIdbyTags(this.existingObj, ['current',
'temp'], eleName)[0].pipe(switchMap((res:any)=>{
//we don't want "res" else the response of
return this.siteService.getHisPointData(ref, 'current')
.pipe(
map(this.helperService.stripHaystackTypeMapping),
)
};
}
getDesiredValue1(eleId){
return this.siteService.getScheduleParamsByRoomRef('temp and air and desired and
heating', eleId).pipe(
switchMap((a:any)=>{
let row = a;
let pointId = this.helperService.stripHaystackTypeMapping(row['id']).split(' ')[0];
return this.siteService.getHisPointData(pointId, 'current')
}))
Well, when we has an element we want create the two calls, we are going to use forkjoin
We want to make, forEach element create two calls, so we can make
this.elements.forEach(element => {
forkJoin([this.getCurrentValue(element.name),this.getDesiredValue1(element.id)])
.subscribe(([current,desired])=>{
element.current=current;
element.desired=desired;
})
})
I user in subscribe ([current,desired]) but we can use res and use element.current=res[0],element.desired=res[1]
If we want, we can even make only one subscription -now we has so many subscriptions as element we have-
arrayOfCalls=[]
this.elements.forEach(element => {
arrayOfCalls.push(
forkJoin([this.getCurrentValue(element.name),this.getDesiredValue1(element.id)])
)
}
//then subscribe
arrayOfCalls.subscribe((fullRes:any[])=>{
fullRes.map((res,index)=>{
this.elements[index].currentValue=res[0]
this.elements[index].desiredValue=res[1]
})
})

tranforming RxJS Observable

I use angularFirestore to query on firebase and I want join data from multiple documents using the DocumentReference.
The first operator map in the pipe return an array of IOrderModelTable, the second operator, i.e, the switchMap iterate over array an for each element use the id contained in each element to query data in other table.
The problem is that in the swithMap I obtain an array of observable due to anidated map operators. How I can obtain an array of IOrderModelTable and then return an observable of this array.
The code is:
getDataTableOperatorsFromDB(): Observable<IOrderModelTable[]> {
const observable = this.tableOperatorsCollectionsRef.snapshotChanges().pipe(
map(actions => {
return actions.map(a => {
const data = a.payload.doc.data() as IOrdersModelDatabase;
const id = a.payload.doc.id;
data.ot = id;
return data;
});
}),
switchMap(data => {
const result = data.map(element => {
return this.afs.collection('Orders/').doc(element.orderNumberReference.id).valueChanges().pipe(map(order => {
return {
otNumber: element.ot,
clientName: '',
clientReference: order.clientReference,
id: element.orderNumberReference,
};
}));
});
// Result must be an IOrderModelTable[] but is a Observable<IOrderModelTable>[]
return of(result);
})
);

You can use to Array operator to transform a stream to an array, but make sure your stream will end.
The trick is to choose the right stream.
For you problem, the natural source would be the list received by your first call. In a schematic way I can put it , you get a list of ids, that you transform into a list of augmented information :
first input ...snapshopChanges():
----[A, B, C]------>
each element is transformed through ...valueChanges():
-------Call A -------------DataA-------->
-------Call B ------------------------DataB----->
-------Call C --------------------DataC----->
Then reduced using toArray() to :
----------------------------------------------[DataA, DataC, DataB]-------->
Code:
getDataTableOperatorsFromDB(): Observable<IOrderModelTable[]> { {
return this.tableOperatorsCollectionsRef.snapshotChanges()
.pipe(
map(actions => {
from(data).pipe(
map(action => {
const data = a.payload.doc.data() as IOrdersModelDatabase;
const id = a.payload.doc.id;
data.ot = id;
return data;
}),
mergeMap(element => {
return this.afs.collection('Orders/').doc(element.orderNumberReference.id).valueChanges().pipe(
map(order => {
return {
otNumber: element.ot,
clientName: '',
clientReference: order.clientReference,
id: element.orderNumberReference,
};
})
);
}),
toArray()
);
})
)
}
Important : I replaced switchMap by mergeMap, otherwise some information could be thrown away.

#madjaoue
You're right, mergeMap is the correct operator in this case because with switchMap for each event emitted the inner observable is destroyed so in the subscribe you only get the final event emitted, i.e, the last row. This observable is long lived, never complete, so also use the operator take with the length of the actions which is the array that contains the list of documents.
Thank you very much for the help. :D
getDataTableOperatorsFromDB(): Observable<IOrderModelTable[]> {
const observable = this.tableOperatorsCollectionsRef.snapshotChanges().pipe(
switchMap(actions => {
return from(actions).pipe(
mergeMap(action => {
console.log(action);
const data = action.payload.doc.data() as IOrdersModelDatabase;
const otNumber = action.payload.doc.id;
return this.afs.collection('Orders/').doc(data.orderNumberReference.id).valueChanges().pipe(map(order => {
return {
otNumber: otNumber,
clientName: '',
clientReference: order.clientReference,
id: data.orderNumberReference,
};
}));
}),
mergeMap(order => {
console.log(order);
return this.afs.collection('Clients/').doc(order.clientReference.id).valueChanges().pipe(map(client => {
return {
otNumber: order.otNumber,
clientName: client.name,
clientReference: order.clientReference,
id: order.id,
};
}));
}),
take(actions.length),
toArray(),
tap(console.log),
);
}),

In Rx.js, how can I distinguish which stream triggers the combineLatest method?

I'm writing my own version of who to follow?. Clicking refreshButton will fetching suggestions list and refresh <Suggestion-List />, and closeButton will resue the data from suggestions list and refresh <Suggestion-List-Item />.
I want to let the closeClick$ and suggestions$ combine together to driving subscribers.
Demo code here:
var refreshClick$ = Rx.Observable
.fromEvent(document.querySelector('.refresh'), 'click')
var closeClick$ = Rx.Observable.merge(
Rx.Observable.fromEvent(document.querySelector('.close1'), 'click').mapTo(1),
Rx.Observable.fromEvent(document.querySelector('.close2'), 'click').mapTo(2),
Rx.Observable.fromEvent(document.querySelector('.close3'), 'click').mapTo(3)
)
var suggestions$ = refreshClick$
.debounceTime(250)
.map(() => `https://api.github.com/users?since=${Math.floor(Math.random()*500)}`)
.startWith('https://api.github.com/users')
.switchMap(requestUrl => Rx.Observable.fromPromise($.getJSON(requestUrl)))
Rx.Observable.combineLatest(closeClick$, suggestions$, (closeTarget, suggestions) => {
if (/* the latest stream is closeClick$ */) {
return [{
target: clickTarget,
suggestion: suggestions[Math.floor(Math.random() * suggestions.length)]
}]
}
if (/* the latest stream is suggestions$ */) {
return [1, 2, 3].map(clickTarget => ({
target: clickTarget,
suggestion: suggestions[Math.floor(Math.random() * suggestions.length)]
}))
}
})
Rx.Observable.merge(renderDataCollectionFromSuggestions$, renderDataCollectionFromCloseClick$)
.subscribe(renderDataCollection => {
renderDataCollection.forEach(renderData => {
var suggestionEl = document.querySelector('.suggestion' + renderData.target)
if (renderData.suggestion === null) {
suggestionEl.style.visibility = 'hidden'
} else {
suggestionEl.style.visibility = 'visible'
var usernameEl = suggestionEl.querySelector('.username')
usernameEl.href = renderData.suggestion.html_url
usernameEl.textContent = renderData.suggestion.login
var imgEl = suggestionEl.querySelector('img')
imgEl.src = "";
imgEl.src = renderData.suggestion.avatar_url
}
})
})
You can find it in JsFiddle.
You should note the comments in condition judgment, closeClick$ emits [{ target: x, suggestion: randomSuggestionX }], suggestions$ emits [{ target: 1, suggestion: randomSuggestion1 }, { target: 2, suggestion: randomSuggestion2 }, { target: 3, suggestion: randomSuggestion3 }]. Subsriber render interface according to the emitted data.
May there are some ways/hacks to distinguish the latest stream in combineLatest or elegant modifications?

I think the easiest way would be to use the scan() operator and always keep the previous state in an array:
Observable.combineLatest(obs1$, obs2$, obs3$)
.scan((acc, results) => {
if (acc.length === 2) {
acc.shift();
}
acc.push(results);
return acc;
}, [])
.do(states => {
// states[0] - previous state
// states[1] - current state
// here you can compare the two states to see what has triggered the change
})
Instead of do() you can use whatever operator you want of course.
Or maybe instead of the scan() operator you could use just bufferCount(2, 1) that should emit the same two arrays... (I didn't test it)