I'm trying to get a full list of watched videos for a given user in my YouTube API application. I want to add up total duration of all videos.
When I get the list of videos from history playlist, the API caps it at 50 items. There's pagination but total amount of items is 50 (not just per page); I can't access more data with the API it appears.
Is there any way I can get this playlist without the data cap? I'm hoping for another method (of using the API) or a way to do it without the API. I know YouTube stores this data because I can view my entire history (far more that 50 videos).
I'm using this code:
var requestOptions = {
playlistId: playlistId,
part: 'snippet',
maxResults: 50
};
gapi.client.youtube.playlistItems.list(requestOptions);
where playlistId is the id of the history playlist I got from a gapi.client.youtube.channels.list request.
Edit (2017): I want to clarify that it was always my intention to download my own history, just out of interest to see how much time I have spent watching videos. I still have not been able to do this.
The API currently only retrieves the last two weeks of Watch History. For more information refer to the Bug Issue reported: https://code.google.com/p/gdata-issues/issues/detail?id=4642
Note:
There is a similar question on SO asked here: YouTube API v3 returns truncated watch history
I wrote a scraper(in Python 2.7(updated for 3.5) and Scrapy) for this task a while ago.
Sans official API, it uses a logged in session cookie and html parsing. Dumps to SQLite by default.
https://github.com/zvodd/Youtube-Watch-History-Scraper
How it's done: essentially it opens the url
https://www.youtube.com/feed/history'
with a valid(logged in) session cookie taken from Chrome. Scrapes all video entries for name, vid(url), channel/user, description, length. Then it finds the button at the bottom of the page with the attribute data-uix-load-more-href which contains the link to the next page, something like:
"/browse_ajax?action_continuation=1&continuation=98h32hfoasau0fu928hf2hf908h98hr%253D%253D&target_id=item-section-552363&direct_render=1"
... re-scrapes the video entries from there and dumps them all into an sqlite database; which you can search entries by any of the fields (name, length, user, description, etc).
So until they change their feed/history page, it's doable and done.
I might even update it.
While this isn't currently possible using just the YouTube API, there is an (albeit slightly involved) method to calculate your watch time):
download a list of your watch history as a JSON file using Google Takeout.
Unfortunately the JSON file doesn't include the video durations, so the next step is to extract the video IDs (the part after "watch?v=" in the "titleURL" object
Now take your list of video IDs, and send a request to the youtube API that looks something like this:
function execute() {
return gapi.client.youtube.videos.list({
"part": [
"contentDetails"
],
"id": [
"VIDEO IDs"
],
"fields": "items(contentDetails(duration))"
})
(Code created using YouTube API Explorer)
Note: You may need to break the list of video IDs into smaller lists (I had to) or the API may reject the request. As [pointed out by stvar in the comments] the ID list maximum length is 50, so this is the maximum length your lists can be. (full disclosure: I was using Python to send the requests)
Finally, just extract the duration values and add them up (though this might not be quite as easy as it sounds)
The best part of this is I don't believe this actually violates any ToS.
It seems like this is a known bug originally reported in 2013. The exact same behavior is explained on a Google Code thread: https://code.google.com/p/gdata-issues/issues/detail?id=4642
Brainstorming, never tried: Have you tried not using the API and instead parsing the https://www.youtube.com/feed/history URL?
Theoretically, the user browsing could be emulated, including the pagination. I am not aware of how hard though (probably very), since you need to deal with authentication and YouTube probably tries to verify that a human is browsing.
I was looking for some way to get the list of YouTube history.
I just found out that Google has a tool for this. In Google Takeout you have a option taht you can get the entire list of watched videos. My list went back util 2011.
To get explanation short there are two videos explaining how to do this:
https://www.youtube.com/watch?v=zlzzO1e6dws
https://www.youtube.com/watch?v=dto8jGMxHxY
Related
The Goal and the Problem
I am working on a sharing solution in our React-based video and audio search apps. My goal is that when sharing a single item from a search result set to a social media app (e.g. Facebook, Messenger, LinkedIn, Twitter) and then clicked by someone else, the user lands within our search app/experience with only a single result showing (the shared item), which is done using query parameters in our search apps. Imagine using Google image search, and creating a little share icon on each search result so that when hovered/clicked/tapped and shared to social media then clicked by someone on said social media channel, the user lands in Google image search with just that single shared item in the result set.
We've been challenged to find a third party solution that uses custom query parameters for share links on each search result to help us achieve our goal. They all seem to use the base url structure (window.location.origin + window.location.pathname e.g. https://www.clipstock.com/audio) which won't work properly to show a single search result within our search experience which needs a query parameter as shown in the example below. Note: this url approach works fine for sharing a full page "node" with just the single video or audio result on the page, which we're also doing separately.
We've built a custom solution in our React based search apps that does achieve our goal of having the user land within our search app with just a single search result although we expect our custom solution to break if/when sharing urls in social apps update which seems to happen yearly or so. We figure a third party JavaScript library or third party social share service will fix these issues faster and cheaper without our development time if/when the sharing URLs break.
Has anyone seen a third party solution to this?
Examples
Example link for landing on a single search result in our audio search app, achieved by the search query q="Reunited" in the url: https://www.clipstock.com/audio?q=%22Reunited%22&size=n_12_n
Example share link of our custom solution for sharing the above search result on Facebook today, which seems to change by the year (note the https://www.facebook.com/sharer/sharer.php?u= part of the url, which is the part that may break from Facebook updates): https://www.facebook.com/sharer/sharer.php?u=https://www.clipstock.com/audio?q=%22Reunited%22&size=n_12_n
Example link of a single video clip node page, which works well with any third party sharing solution, although isn't the goal of this question which is to land in our search app(s) with just the single shared item in the result set which is achieved using query parameters as seen in the examples above: https://www.clipstock.com/clip/young-male-swimmer-diving-starting-block-olympic-size-pool-lanes-slow-motion
The below approach to sharing works for us, although is the custom sharing solution we created that may break over the years. In this example, on mobile, the Facebook app opens, and on non-mobile, the Facebook web app opens, which is our goal. Either of the Facebook share urls (and the share urls for each other social media app like Messenger, LinkedIn, Twitter) could change over time since they're essentially API integrations and we think it would be cheaper & faster to invest in a third party sharing solution than to monitor for these breaks and fix the breaks ourselves.
Note: some setup code in the below example is not included such as the DrupalContext. We aren't getting errors in the code, rather, we're looking for a third-party solution so that we don't have to maintain custom code.
const drupalContext = useContext(DrupalContext);
const queryString = type === "sfx" ? drupalContext.sfxQueryString : "";
const baseUrl = window.location.origin.concat(window.location.pathname);
const sharingLink =
type === "video"
? baseUrl.concat("?q=").concat(nodeId)
: baseUrl
.concat("?q=")
.concat(encodeURI('"'))
.concat(encodeURI(title))
.concat(encodeURI('"'))
.concat(queryString);
const shareViaFacebook = () => {
let shareUrl = "";
if (platform?.os?.family === ("iOS" || "Android" || "Windows Phone")) {
shareUrl = "fb://share/?link=" + sharingLink;
} else {
shareUrl = "https://www.facebook.com/sharer/sharer.php?u=" + sharingLink;
}
window.open(shareUrl);
};
We've created similar shareVia... methods for each social media app we're sharing to which currenly consists of Facebook, Messenger, LinkedIn, Twitter, Email, and "Copy Link". Email and "Copy Link" are what we're going to fall back to using if we can't find a reliable third-party solution (removing Facebook, Messenger, LinkedIn, and Twitter sharing options) since Email and "Copy Link" reliably work and we don't expect these two options to break over time.
Recently , the youtube API has been updated to V3 , and I have a question:
How can I get the video details (views, name , description etc..) through Javascript using the url of the video ?
Example :
I have this url https://www.youtube.com/watch?v=DQ5w8LBI0kI&ab_channel=YT
and I want to get via Javascript the name and description of it.
How can I do this ?
You can find a detailed information about how to retrieve any kind of data you want in the official documentation of youtube api.Here
The API documentation is quite involved. Two good points to start from are the following:
YouTube Data API Overview, and
JavaScript Quickstart.
Upon reading them -- and surely more for that matter -- do come back with concrete (programming!) questions.
One more hint: the API endpoint that provides you with the information attached to a given video is Videos.list. But that doc page becomes meaningful only after you familiarize yourself with the surrounding (programming) environment.
Steps to achieve:
See the answer here to see how to get/parse video Id from a youtube video url.
After you get a videoId e.g. DQ5w8LBI0kI, you have to use that in the below API to get views, name, description etc.
https://www.googleapis.com/youtube/v3/videos?part=snippet%2CcontentDetails%2Cstatistics&id=DQ5w8LBI0kI&key=[YOUR_API_KEY]
See more about this Youtube API here
I'm using an API in JSON format to display the standings of sports teams in a league. Here's an example of the JSON data I'm working with:
{ "data":[
{
"position":1,
"team_id":53,
"team_name":"Celtic",
"group_id":null,
"group_name":null,
"overall":{
"games_played":25,
"won":18,
"draw":6,
"lost":1,
"goals_scored":54,
"goals_against":17
},
"home":{
"games_played":13,
"won":9,
"draw":4,
"lost":0,
"goals_scored":30,
"goals_against":8
},
"away":{
"games_played":12,
"won":9,
"draw":2,
"lost":1,
"goals_scored":24,
"goals_against":9
},
"total":{
"goal_difference":"+37",
"points":60
},
"result":"Promotion - Premiership (Championship Group)",
"points":60,
"recent_form":"WWWDW",
"status":"same"
}
]}
I have a way to associate the team_ID with specific team data from the API. What I'm trying to do is to make the teams in my Standings tables clickable, so when the user clicks on a team, they can go to a page where I display data for that specific team (the info would, of course, be extracted from the API). Now, I'm developing the website on top of WordPress, and I'm wondering how I can auto-generate pages for every team, so that site users can go to specific team pages directly from the Standings tables.
It's not convenient for me to manually create pages with a piece of code and upload them to the FTP server for each team, because there'll be so many teams (eventually thousands of teams), and it's definitely not feasible.
Can anyone please guide me on the way to auto-create pages for the teams so my team links don't return 404 Not Found pages? An important point is that I'm using JavaScript to manipulate the data.
I'm using the team_id property in the API to create my team links, but I don' know how to use my team_id to automatically create team pages on my WordPress site.
Please help if you can. :)
You can use ajax method on click team-id, in ajax call function, check, is this team-id exists or not, if exists redirect to respective page or not create a new page and display require information.
I have worked same type project, I am using CPT method for the game and used the custom taxonomy for team and player.
I need a little assistance with my app. I'm not asking for a handout, just some guidance as to where to begin. The basic idea is for logged in users to be able to favorite videos, and to have them persist when logged in.
My app uses the YouTube API (Playlist Items) to display videos from my public playlists within my app. I'm using firebase auth to register and login users, but I have yet to implement the RTD. This is where I need some assistance in structuring my data & organizing my app.
I don't want favorite lists created for every user by default, nor do I want to store false values and have to loop through them. I'd only like to set a favorites list if the user requests to do so, and the values are true. I'm open to suggestions regarding structuring my data, but I was thinking something simple like this:
"favorites": {
"John Doe": {
"video1ID": true,
"video2ID": true,
}
}
Videos are contained within cards using a .each function from within the API response. Included in these cards are "favorite" toggle switches that I'd like a user to be able to toggle and add a favorite video to their list.
YouTube provides Video ID's from within their JSON response. I was thinking that assigning a boolean to that video ID would get the job done, but I have no idea where to begin.
Something like:
function writeFavoritesList (name, videoID, toggleValue) {
firebase.database().ref('favorites/' + userId).set({
name: displayName,
videoID: videoID,
toggleValue: true
});
}
I'm very much a newb to anything outside of WordPress, so I hope I'm on the right track. Any help appreciated. Thanks! :)
Looks great. If this were another database, you could consider storing the video IDs in an array, but this being the firebase RealTime Database, you're much better off with objects, which you've already got.
You could modify your structure slightly to take advantage of RTDs push() key generation if you ever intend on sorting your favourite videos. To do so, instead of making the key the videoID and the value the boolean status, you could generate a key using firebase's push() key generation and make the value the videoID. "The unique key generated by push() are ordered by the current time, so the resulting list of items will be chronologically sorted. The keys are also designed to be unguessable (they contain 72 random bits of entropy)."
"favorites": {
"uid1": {
"uniqueKey1": videoID1,
"uniqueKey2": videoID2,
}
}
To generate a push() key, use: const key = firebase.database().ref().push().key.
More info: https://firebase.google.com/docs/reference/js/firebase.database.Reference#push
Saw your Guru post. I think the best way for you to learn is to delve deep into the documentation and figure this out for yourself. If you're truly committed to learning this stuff you'd be doing yourself a disservice to have someone else write the code for you.
I'd start with the GCP(Google Cloud Platform) cloud firestore docs and read through the Concepts section in its entirety:
https://cloud.google.com/firestore/docs/concepts
The firebase site mirrors parts of the GCP documenation, but also covers client implementations:
https://firebase.google.com/docs/firestore/
To get the most out of these docs use the nav sidebar on the left to drill down into all the various Cloud Firestore topics. They go into how to structure your database and provide sample code for you to analyse and play with.
You'll see the terms Documents and Collections thrown around a lot. A Document is somewhat equivalent to a JSON Object. A Collection is a list of documents; similar to an array of JSON objects. But here's where things get interesting; Documents can reference Collections (aka Subcollections):
So I would structure your database as follows:
Create a Users collection
Whenever a new user signs into your app, create a user document and add it to the Users collection.
The first time a user selects a favorite video create a Favorites collection and add it to the user document; then add favorite documents to the Favorites collection for this user
There is a Javascript/Web client (you've seem to already have it loaded from what I've seen in the repo link you provided on Guru). Here's the reference documentation for it:
https://firebase.google.com/docs/reference/js/firebase.firestore
The classes, methods and properties defined in those reference docs are what you'll be calling from within your jquery code blocks.
Good luck and stick with it.
I'm using the Instagram REST API and trying to make a simple paginated profile. According to the endpoint docs for /v1/users/{user-id}/media/recent/?access_token=ACCESS-TOKEN - the request can take a max_id... which is supposed to be supplied in the pagination object as per "envelope" response seen here. However when I receive my response, the pagination object is empty. See response below that I'm getting...
{
pagination: {},
meta: { code: 200 },
data:[ ... photos ... ]
}
I've read other S.O. questions about this and they all claim that "it's empty when there are no more photos" — BUT this is not the case. The account I'm testing with has several hundred photos, and I have no way to get beyond the first 20. I even tried using the id of the last object in the data array as my max_id, but still to no avail.
One other final bit of info I can provide is that my account is still in Sandbox mode — don't know if that has anything to do with it (eg: maybe they disable pagination for sandbox accounts).
Anyway, help is much appreciated!
Nevermind. It's not allowed in Sandbox mode :/