Commit Graph

636 Commits

Author SHA1 Message Date
Philipp Hagemeister 9684f17cde Merge remote-tracking branch 'akretz/youtube_fix' 2014-12-11 16:28:10 +01:00
Philipp Hagemeister e52a40abf7 [youtube] Add test case for #4431 2014-12-11 16:28:07 +01:00
Adrian Kretz 24270b0301 [youtube] The case that 'url_encoded_fmt_stream_map' or 'adaptive_fmts' is the empty string is handled accordingly (fixes #4431) 2014-12-11 16:00:46 +01:00
Philipp Hagemeister 4bb4a18876 [youtube] Fix imports 2014-12-11 10:08:17 +01:00
Philipp Hagemeister da27660014 [youtube] Pass in all variables to DASH manifest (Fixes #4424) 2014-12-10 14:39:00 +01:00
Philipp Hagemeister 774e208f94 [youtube] Handle missing DASH manifest (Fixes #4421, fixes #4420) 2014-12-10 13:21:24 +01:00
Philipp Hagemeister b82f815f37 Allow iterators for playlist result entries 2014-12-06 14:02:19 +01:00
Philipp Hagemeister 9ff6772790 [youtube] Modernize 2014-12-06 12:20:54 +01:00
Philipp Hagemeister 810fb84d5e pep8 and minor beautification all around 2014-12-04 08:27:40 +01:00
Jaime Marquínez Ferrándiz c41a9650c3 [youtube] Extract framerate from the dash manifest
Not all videos have 60 fps, for example they can have 48 fps.
2014-12-01 17:36:12 +01:00
Jaime Marquínez Ferrándiz c522adb1f0 [youtube] Add a normal age-gate test video 2014-11-30 21:45:49 +01:00
Jaime Marquínez Ferrándiz 7160532d41 [youtube] Simplify code for getting the dash manifest url
video_info contains now the 'ytplayer.config.args' dictionary
2014-11-30 21:07:50 +01:00
Jaime Marquínez Ferrándiz 4e62ebe250 [youtube] Try to extract the video_info from the webpage before requesting the 'get_video_info' pages
The YouTube player doesn't seem to use them except for embedded videos, so we can skip a network request.
But they still provide better error mesagges (for removed videos for example).
2014-11-30 20:56:32 +01:00
Jaime Marquínez Ferrándiz b766eb2707 [youtube] Update test 2014-11-30 19:18:39 +01:00
Jaime Marquínez Ferrándiz 10a404c335 [youtube] Add format 313 (fixes #4339) 2014-11-30 18:56:14 +01:00
Jaime Marquínez Ferrándiz 42939b6129 [youtube] Use a cookie for seeting the language
This way, we don't have to do an aditional request
2014-11-30 00:03:59 +01:00
Jaime Marquínez Ferrándiz 37ea8164d3 [youtube] Don't confirm age when initializing
It seems that all the videos with age restriction use now the age gate method, which doesn't require any confirmation.
2014-11-29 23:46:39 +01:00
Philipp Hagemeister 0792d5634e [youtube] Remove useless u prefixes 2014-11-26 12:41:53 +01:00
Philipp Hagemeister 27f8b0994e Merge remote-tracking branch 'jtwaleson/master' 2014-11-23 22:10:26 +01:00
Jouke Waleson 9e1a5b8455 PEP8: applied even more rules 2014-11-23 21:39:15 +01:00
Philipp Hagemeister 784b6d3a9b Merge remote-tracking branch 'jtwaleson/master' 2014-11-23 21:33:31 +01:00
Jouke Waleson 8bcc875676 PEP8: more applied 2014-11-23 21:20:46 +01:00
Jouke Waleson 5f6a1245ff PEP8 applied 2014-11-23 20:41:03 +01:00
Philipp Hagemeister f3a3407226 [youtube] Clarify keywords 2014-11-23 20:09:10 +01:00
Philipp Hagemeister aa79ac0c82 [youtube] Support controversy videos (Fixes #4275) 2014-11-23 09:59:02 +01:00
Philipp Hagemeister 0857baade3 [youtube] Add webm audio formats (Fixes #4229) 2014-11-18 11:08:37 +01:00
Philipp Hagemeister 23ad44b57b [youtube] Better error message for DASH manifest 2014-11-17 20:12:52 +01:00
Aaron Wojnowski 894dd8682e Fix YouTube signature extraction. 2014-11-13 00:33:27 -06:00
Philipp Hagemeister a1cf99d03a [YoutubeDL] Add playlist_id and playlist_title fields (Fixes #4139) 2014-11-09 22:32:35 +01:00
Philipp Hagemeister 20436c30c9 [youtube] Clarify output 2014-11-04 23:35:34 +01:00
Philipp Hagemeister 94bd361318 [youtube] Skip sts if missing (Fixes #4095, fixes #4103) 2014-11-04 22:45:43 +01:00
Philipp Hagemeister 0d2c141865 [youtube] Detect formats 298 et al as mp4 (Fixes #4066) 2014-10-31 11:13:02 +01:00
Philipp Hagemeister 7b6de3728a [youtube] Add format 266 (Fixes #4055) 2014-10-30 09:53:43 +01:00
Philipp Hagemeister a51d3aa001 [youtube] Add support for formats 302 and 303 (Fixes #4060) 2014-10-30 09:43:11 +01:00
Philipp Hagemeister fbb21cf528 [youtube] Add formats 298, 299 (Fixes #4056) 2014-10-30 09:34:13 +01:00
Philipp Hagemeister f46a8702cc [youtube:playlist] Fix test title 2014-10-27 00:06:47 +01:00
Philipp Hagemeister bfc2bedcfc [youtube] Make confirm_age non-fatal (#4042) 2014-10-26 21:49:29 +01:00
Philipp Hagemeister 6b445558ff [youtube] Only set language when necessary
We only need to set the language when we are using a login; otherwise it will always be English.
2014-10-23 00:44:22 +02:00
Philipp Hagemeister 1cc887cbf0 [youtube] Add support for format 278 2014-10-13 00:09:19 +02:00
Philipp Hagemeister 203fb43f36 [youtube] Download DASH manifest by default (Closes #3887) 2014-10-13 00:03:08 +02:00
Philipp Hagemeister a1f934b171 [youtube] Correct language cookie handling 2014-09-29 02:04:16 +02:00
Philipp Hagemeister e2dce53781 [youtube] Always request webpage in English (Fixes #3844) 2014-09-29 01:39:26 +02:00
Philipp Hagemeister 9c44d2429b [vimeo:likes] Support large like lists (Fixes #3847) 2014-09-29 00:36:06 +02:00
Philipp Hagemeister 6b08cdf626 [youtube] Support for embedded /p players (Fixes #3821) 2014-09-25 01:59:02 +02:00
Philipp Hagemeister ac7553d031 [youtube] Support embed/videoseries URLs (#3821) 2014-09-24 10:34:29 +02:00
Philipp Hagemeister cdc628a498 [youtube] Move more tests to extractors 2014-09-24 10:25:47 +02:00
Philipp Hagemeister 69ea8ca42c [youtube] Remove superfluous unicode specifiers 2014-09-24 09:51:45 +02:00
Philipp Hagemeister 4bc3a23ec5 [youtube] Modernize 2014-09-24 09:49:53 +02:00
Philipp Hagemeister 78caa52aea [youtube] Modernize 2014-09-13 07:51:06 +02:00
Philipp Hagemeister 310d75d141 [youtube] Correct description in test case 2014-09-13 07:49:26 +02:00
Philipp Hagemeister 9291475f7d [youtube] Move more tests to Playlist IE, and simply suitable. 2014-09-13 07:35:14 +02:00
Philipp Hagemeister 81127aa55d [youtube] Move playlist test to extractor 2014-09-13 07:19:20 +02:00
Philipp Hagemeister 9b583dca4c Merge remote-tracking branch 'drags/yt-feed-loadmore' 2014-09-13 07:14:19 +02:00
Jaime Marquínez Ferrándiz edb53e2dc3 [youtube] Don't make the url protocol optional
The generic extractor will add it.
2014-09-11 21:47:25 +02:00
Philipp Hagemeister a0e07d3161 [youtube] Move cache into its own module 2014-09-03 17:29:19 +02:00
Philipp Hagemeister c9cc0bf57b [youtube:playlist] PEP8 2014-09-01 01:00:40 +02:00
Jaime Marquínez Ferrándiz 9ea9b61448 Merge branch 'master' of github.com:rg3/youtube-dl 2014-08-31 23:47:26 +02:00
Jaime Marquínez Ferrándiz 1ed5b5c9c8 [youtube:subscriptions] Fix extraction (fixes #2975) 2014-08-31 23:44:43 +02:00
Philipp Hagemeister 55f7bd2dcc [youtube] Fix category extraction 2014-08-31 23:26:19 +02:00
Philipp Hagemeister 3e7c12240c [youtube] Test for like_count and dislike_count (#3633) 2014-08-31 18:10:05 +02:00
Jaime Marquínez Ferrándiz f30a38be8b [youtube] Fix extraction of like and dislike count (fixes #3633) 2014-08-31 12:38:00 +02:00
Philipp Hagemeister 9480d1a566 Merge remote-tracking branch 'riking/twofactor' 2014-08-24 07:14:23 +02:00
Philipp Hagemeister bd9820c937 Merge remote-tracking branch 'liudongmiao/patch-subtitle' 2014-08-22 02:45:21 +02:00
Philipp Hagemeister 55db73efdf [youtube] tag 171 is 128KBits (Fixes #3542) 2014-08-21 13:13:26 +02:00
Philipp Hagemeister 55c49908d2 [youtube] Handle incorrectly written cache files (#3549) 2014-08-21 12:15:51 +02:00
riking 83317f6938 [youtube] Add two-factor account signin (TOTP only)
Additional work is required to prompt the user for the SMS or phone call codes, as there is no framework currently to prompt the user during an extraction operation.

Fixes #3533
2014-08-16 14:48:17 -07:00
Philipp Hagemeister 62af3a0eb5 [youtube] Use new signature cache ID for in-memory cache as well 2014-08-02 12:23:18 +02:00
Philipp Hagemeister 60064c53f1 [youtube] Make cache ID a tuple of lengths instead of just the whole length 2014-08-02 12:21:53 +02:00
Philipp Hagemeister 98eb1c3fa2 [youtube] Clean up -v signature output 2014-08-02 11:55:20 +02:00
Philipp Hagemeister 201e9eaa0e [youtube] Show format ID in signature deobfuscation -v output 2014-08-02 06:35:18 +02:00
Tim Sogard 1a9b9649fb Check load_more_widget_html for feed paging 2014-07-27 17:19:26 -07:00
Philipp Hagemeister b081cebefa [youtube] Fix player ID display 2014-07-25 06:49:26 +02:00
Jaime Marquínez Ferrándiz c084c93402 [youtube] Extract the 'sts' parameter from the webpage (fixes #3327) 2014-07-23 12:16:26 +02:00
Liu DongMiao 7e660ac113 if there is more than one subtitle for the language, use the first one 2014-07-23 10:56:09 +08:00
Philipp Hagemeister c081b35c27 [youtube] Support new player URLs (Fixes #3326) 2014-07-23 02:19:33 +02:00
Philipp Hagemeister 246168bd72 Remove unused imports 2014-07-20 23:38:44 +02:00
Jaime Marquínez Ferrándiz 2c57c7fa5a [youtube] Fix extraction of age gate videos (closes #3270)
Setting the correct value of the 'sts' paramater in the 'get_video_info' url gives the correct urls.
Removed parameters that are not needed.
2014-07-20 21:05:02 +02:00
Philipp Hagemeister b8c74d606a [youtube] fix display of swf player id 2014-07-20 20:20:42 +02:00
Philipp Hagemeister 5425626790 [youtube] Move swfinterp into its own file 2014-07-18 10:24:28 +02:00
Philipp Hagemeister 5dc3552d85 [youtube] Add support for classes in swf parser 2014-07-18 00:54:17 +02:00
Philipp Hagemeister 3fbd27f73e [youtube] SWF parser: Add opcode 86
Yes, I know we need 96, but an implementation of 86 could help avoid a similar issue.
2014-07-17 23:22:49 +02:00
Philipp Hagemeister cf01013161 [youtube] Find more swf players (Closes #3270, refer #3271) 2014-07-17 16:28:36 +02:00
Philipp Hagemeister c8bf86d50d [youtube] Correct signature extraction error detection 2014-07-11 10:44:39 +02:00
Philipp Hagemeister b3a8878080 [youtube] Remove static signatures
The always fail by now. Instead, use only automatic signature extraction
2014-07-11 10:23:19 +02:00
Sergey M․ 6feb2d5e80 [youtube:search_url] Update regexes 2014-07-04 19:21:19 +07:00
Philipp Hagemeister dc2fc73691 [youtube:truncated_url] Move test to extractor 2014-07-01 15:49:34 +02:00
Philipp Hagemeister c4808c6009 [youtube_truncated_url] Add support for truncated watch URLs with annotations (#3178) 2014-07-01 15:49:16 +02:00
Petr Půlpán 18061bbab0 [Youtube] add DASH format 272 (fixes #3128) 2014-06-21 12:03:27 +02:00
Jaime Marquínez Ferrándiz c26e9ac4b2 [youtube] Recognize signature functions that contain '$' (fixes #3104) 2014-06-19 16:42:49 +02:00
pulpe 7d568f5ab8 [Youtube] Recognize playlists with LL 2014-06-14 13:23:28 +02:00
pulpe 3c80377b69 [Youtube] Add format code 271 (1440p webm) 2014-06-07 14:31:10 +02:00
Jaime Marquínez Ferrándiz 70e322695d [youtube:playlist] Fix mixes extraction (fixes #3051)
The username seems to be empty now.
2014-06-05 21:23:27 +02:00
Anton Novosyolov 263bd4ec50 Recognize a third format of the upload_date in the 'watch-uploader-info' element 2014-06-02 13:30:23 +04:00
Jaime Marquínez Ferrándiz 76e92371ac [youtube] Recognize a second format of the upload_date in the 'watch-uploader-info' element (#2911) 2014-05-16 22:12:52 +02:00
Jaime Marquínez Ferrándiz 05ee2b6dad [youtube] Fix extraction of the feed 'paging' values (fixes #2925) 2014-05-16 16:01:13 +02:00
Philipp Hagemeister beee53de06 [youtube] Look for published-on date if uploaded-on is not found
Fixes #2911
2014-05-16 13:21:44 +02:00
Jaime Marquínez Ferrándiz 01ed5c9be3 [youtube] Fix typo 2014-05-15 13:43:29 +02:00
Philipp Hagemeister ad3bc6acd5 Document and test categories (#2923) 2014-05-15 12:41:42 +02:00
Dario Guarascio ec8deefc27 [youtube] Video categories added to metadata 2014-05-15 13:59:27 +07:00
Jaime Marquínez Ferrándiz e399853d0c [youtube:playlist] Improve detection of private lists (#2840) 2014-05-12 07:59:33 +02:00
Jaime Marquínez Ferrándiz 10c0e2d818 [youtube:playlist] Raise an error if the list doesn't exist or is private (closes #2840) 2014-05-01 15:40:35 +02:00
AGSPhoenix e75cafe9fb Clean up format list for consistency
This should make the format list output look a bit nicer.
2014-04-30 01:52:05 +02:00
Jaime Marquínez Ferrándiz aa92f06308 [youtube] Don't call 'unquote_plus' on the video title (fixes #2799)
It's already unquoted after calling 'compat_parse_qs'.
It replaced '+' with spaces, for example in https://www.youtube.com/watch?v=XC0b5YexO-I.
2014-04-25 13:19:03 +02:00
Jaime Marquínez Ferrándiz e00c9cf599 [youtube] Update test description field 2014-04-25 13:14:15 +02:00
Jaime Marquínez Ferrándiz ba60a3ebe0 [youtube] Update test description field 2014-04-25 12:57:04 +02:00
Philipp Hagemeister d11271dd29 [youtube] Include video Id in common error message (Fixes #2786) 2014-04-21 20:34:03 +02:00
Philipp Hagemeister 1db2666916 [youtube:playlist] Correct playlist ID output
The ID now starts with PL, so we don't need to output that twice.
2014-04-15 17:55:52 +02:00
Philipp Hagemeister b9c76aa1a9 [youtube] Add support for cleanvideosearch.com (Fixes #2734) 2014-04-11 13:53:05 +02:00
Philipp Hagemeister 68eb8e90e6 [youtube:playlist] Fix playlists for logged-in users (Fixes #2690) 2014-04-04 02:23:36 +02:00
Jaime Marquínez Ferrándiz e45d40b171 [youtube:subscriptions] Add space to the description 2014-04-03 15:13:52 +02:00
Jaime Marquínez Ferrándiz f6177462db [youtube] feeds: Also look for the html in the 'content_html' field (fixes #2671) 2014-04-02 14:13:08 +02:00
Philipp Hagemeister 5912c639df [youtube] Transform google's JSON dialect (fixes #2663) 2014-04-01 05:56:56 +02:00
Philipp Hagemeister 83d548ef0f [youtube] Encode ytsearch query 2014-03-30 15:57:35 +02:00
Philipp Hagemeister acd213ed6d Remove unusued imports 2014-03-30 07:16:07 +02:00
Philipp Hagemeister 2b25cb5d76 [youtube] Move JavaScript interpreter into its own module 2014-03-30 07:02:58 +02:00
Philipp Hagemeister 43b81eb98a [youtube] Remove useless resolution fields from format definitions
These can be - and are - calculated automatically by the YoutubeDL core.
2014-03-22 14:22:41 +01:00
hurda 278229d195 itag 160 is 144p, not 192p 2014-03-22 12:15:45 +01:00
Philipp Hagemeister 7e8c0af004 Add --prefer-insecure option (Fixes #2364) 2014-03-21 00:37:10 +01:00
Jaime Marquínez Ferrándiz bc6d597828 Add bestvideo and worstvideo to special format names (#2163) 2014-03-14 17:01:47 +01:00
Philipp Hagemeister ae7ed92057 [youtube] Fix up invalid JSON 2014-03-10 13:35:45 +01:00
Philipp Hagemeister c9ae7b9565 [youtube] Add support for search result URLs (Fixes #2495) 2014-03-04 03:32:28 +01:00
AGSPhoenix 60d3a2e0f8 Fix incorrect format codes
Corrects the descriptions for the DASH video format codes 264 and 138
(1440p and 2160p, respectively).
2014-02-24 21:29:37 -05:00
Sergey M. a2dafe2887 [youtube] Fix mix video regex
Attributes' order in <li> is arbitrary and changes every time playlist
page is fetched, so we can't rely on `data-index` to be before
`data-video-username`.
2014-02-24 12:52:02 +07:00
Jaime Marquínez Ferrándiz bc2f773b4f [youtube:playlist] Fix mixes extraction (fixes #2444) 2014-02-23 17:17:36 +01:00
Philipp Hagemeister 521ee82334 Fix imports 2014-02-22 23:03:12 +01:00
Jaime Marquínez Ferrándiz 3489b7d26c [youtube] Simplify the decryption process for the manifest urls and add a test (closes #2422) 2014-02-21 15:15:58 +01:00
Jaime Marquínez Ferrándiz acd2bcc384 Merge branch 'youtube-dash' of github.com:m0vie/youtube-dl 2014-02-21 15:02:47 +01:00
Sergey M. dbb94fb044 [youtube] Fix playlist extraction (Closes #2423, #2424, #2425) 2014-02-21 17:19:55 +07:00
m0viefreak d68f0cdb23 [youtube] decrypt signature when downloading dash manifest 2014-02-21 03:24:56 +01:00
Sergey M. f7000f3a1b [youtube] Add support for yourepeat.com URLs (Closes #2397) 2014-02-19 02:00:54 +07:00
Philipp Hagemeister 2eb5d315d4 [youtube] Match more truncated URLs (Closes #2402) 2014-02-17 14:56:21 +01:00
Philipp Hagemeister 07ad22b8af [youtube:search] Mark "no results found" error as expected 2014-02-15 16:30:11 +01:00
Filippo Valsorda e70dc1d14b [youtube] Correct a minor regex typo 2014-02-10 01:30:47 +01:00
Philipp Hagemeister 81c2f20b53 [youtube] Correct invalid JSON (Fixes #2353) 2014-02-09 17:56:10 +01:00
Philipp Hagemeister 9766538124 [jadorecettepub] Add extractor (Fixes #2148) 2014-02-08 19:20:23 +01:00
Jaime Marquínez Ferrándiz d67cc9fa7c [youtube:playlist] Recognize ‘top tracks’ urls (closes #2332)
The list parameter starts with ‘MC’ and can have more characters after it, including dots
2014-02-06 19:46:26 +01:00
Jaime Marquínez Ferrándiz 20650c8654 [youtube] signatures: Recognize javascript functions that contain “$” (fixes #2304) 2014-02-04 21:38:50 +01:00
Philipp Hagemeister b11cec4162 [youtube:user] Fix id key (Fixes #1745) 2014-01-29 11:16:12 +01:00
Jaime Marquínez Ferrándiz 5700e7792a [youtube] Encode the data when submitting the form for confirming the age
Needed on python 3
2014-01-25 17:22:41 +01:00
Jaime Marquínez Ferrándiz 38c2e5b8d5 [youtube] Use https: in more urls 2014-01-25 17:11:55 +01:00
Jaime Marquínez Ferrándiz beddbc2ad1 [youtube:toplist] Make the regex for finding the playlist link more flexible
`title={foo}` may not be at the end of the `href` string.
2014-01-25 15:47:03 +01:00
Philipp Hagemeister 2c5bae429a [youtube] Fix new formats 2014-01-24 00:06:26 +01:00
Philipp Hagemeister 1394ce65b4 [youtube] Add new formats (Fixes #2221) 2014-01-23 23:54:06 +01:00
Philipp Hagemeister 0b65e5d40f [youtube] Do not break upon unknown formats 2014-01-23 23:21:42 +01:00
Philipp Hagemeister 975d35dbab [youtube:truncated_url] Also match mail subscription links (#2214) 2014-01-23 16:14:54 +01:00
Philipp Hagemeister 4919603f66 [youtube] Make DASH manifest download conditional for now
DASH download fails on many videos (all with encrypted signatures? not sure yet), for example 07FYdnEawAQ, with a 403.
2014-01-22 21:56:38 +01:00
Philipp Hagemeister 65697b3bf3 Merge branch 'paged-lists'
Conflicts:
	test/test_utils.py
	youtube_dl/extractor/youtube.py
2014-01-22 20:00:16 +01:00
Philipp Hagemeister 50317b111d Merge branch 'youtube-dash-manifest'
Conflicts:
	youtube_dl/extractor/youtube.py
2014-01-22 19:58:31 +01:00
Philipp Hagemeister b7ab059084 Add infrastructure for paged lists
This commit allows to download pages in playlists as needed instead of all at once.
Before this commit,
    youtube-dl http://www.youtube.com/user/ANNnewsCH/videos --playlist-end 2 --skip-download
took quite some time - now it's almost instantaneous.
As an example, the youtube:user extractor has been converted.
Fixes #2175
2014-01-20 11:36:47 +01:00
Philipp Hagemeister c91778f8c0 [youtube] Fall back to header if playlist title is not available
Sometimes (in about 10% of requests), the og:title is missing for a weird reason.
See #2170 for an example
2014-01-20 02:45:51 +01:00
Philipp Hagemeister dd27fd1739 [youtube] Download DASH manifest
If given, download and parse the DASH manifest file, in order to get ultra-HQ formats.
Fixes #2166
2014-01-19 05:47:20 +01:00
Philipp Hagemeister 484aaeb204 [everyonesmixtape] Add support (Fixes #2161) 2014-01-17 02:56:13 +01:00
Philipp Hagemeister ce6b9a2dba [youtube] Add a pseudo format for rtmp videos (#2123) 2014-01-09 02:38:50 +01:00
Philipp Hagemeister c3197e3e5c [youtube] Correct subtitle URL (Fixes #2120) 2014-01-09 01:36:21 +01:00
Philipp Hagemeister b4622a328b Use double quotes in error message (#2112)
On Windows, double quotes are required, because single quotes get served to youtube-dl. (Yes, cmd.exe is crazy like that).
On other system, both double and single quotes are fine, unless the string contains a dollar sign (then you need single quotes).
Since virtually no URLs contain dollar signs, double quotes should do.
2014-01-08 00:05:11 +01:00
Jaime Marquínez Ferrándiz 8fa8a6299b [youtube] Add itag 264 (closes #2063)
It has a better bitrate than 137 but the same resolution
2014-01-01 13:45:33 +01:00
Philipp Hagemeister 2c62dc26c8 [youtube] Simplify format specification 2013-12-24 12:34:09 +01:00
Philipp Hagemeister 4bcc7bd1f2 Add temporary _sort_formats helper function 2013-12-24 12:31:42 +01:00
Philipp Hagemeister d80044c235 [youtube] Prefer videos with sound 2013-12-23 04:51:42 +01:00
Philipp Hagemeister f82b18efc1 Merge remote-tracking branch 'rzhxeo/youtube' 2013-12-23 04:37:40 +01:00
Philipp Hagemeister e302f9ce32 [youtube:user] Speed up --match-title 2013-12-22 03:57:42 +01:00
rzhxeo e56f22ae20 [YoutubeIE] Sort formats by resolution 2013-12-18 21:22:37 +01:00
rzhxeo dbd1988ed9 [YoutubeIE] Add width and height to format dict 2013-12-18 21:21:25 +01:00
rzhxeo 4ea3be0a5c [YoutubeIE] Externalize format selection 2013-12-18 03:30:55 +01:00
Philipp Hagemeister 46374a56b2 [youtube] Do not warn for videos with allow_rating=0
This fixes #1982
Test video: http://www.youtube.com/watch?v=gi2uH3YxohU
2013-12-17 02:49:56 +01:00
Philipp Hagemeister b466b7029d [youtube] Make duration an integer or None 2013-12-16 04:09:05 +01:00
Philipp Hagemeister ffa8f0df0a Merge remote-tracking branch 'jaimeMF/yt-toplists' 2013-12-09 04:49:32 +01:00
Philipp Hagemeister 7cc3570e53 Add fatal=False parameter to _download_* functions.
This allows us to simplify the calls in the youtube extractor even further.
2013-12-09 01:49:03 +01:00
Jaime Marquínez Ferrándiz baa7b1978b Remove the calls to 'compat_urllib_request.urlopen' in a few extractors 2013-12-08 22:24:55 +01:00
Philipp Hagemeister 31812a9e0e [youtube:channel] Fix automated channel detection 2013-12-08 07:30:42 +01:00
Jaime Marquínez Ferrándiz 715c8e7bdb [youtube:playlist] Recognize mix ids for direct use (fixes #1295) 2013-12-06 19:52:41 +01:00
Jaime Marquínez Ferrándiz 7d4afc557f [youtube:playlist] Support mix ids longer than 13 (#1295) 2013-12-06 19:48:54 +01:00
Jaime Marquínez Ferrándiz 336c3a69bd [youtube] Extract like and dislike count (#1895) 2013-12-06 13:22:27 +01:00
Philipp Hagemeister 27dcce1904 [youtube] Resolve URLs in comments 2013-12-04 14:18:49 +01:00
Jaime Marquínez Ferrándiz cb7fb54600 Change the ie_name of YoutubeSearchDateIE
It produced a duplicate entry when listing the extractors with '--list-extractors' and generates noise in the commit log when generating the supported sites webpage (like in 09f355f73b)
2013-12-03 13:55:25 +01:00
Jaime Marquínez Ferrándiz 0a688bc0b2 [youtube] Add support for downloading top lists (fixes #1868)
It needs to know the channel and the title of the list, because the ids change every time you browse the channels and are attached to a 'VISITOR_INFO1_LIVE' cookie.
2013-11-30 14:56:51 +01:00
Jaime Marquínez Ferrándiz 0e44d8381a [youtube:feeds] Use the 'paging' value from the downloaded json information (fixes #1845) 2013-11-28 00:33:27 +01:00
Jaime Marquínez Ferrándiz 76d1700b28 [youtube:playlist] Fix the extraction of the title for some mixes (#1844)
Like https://www.youtube.com/watch?v=g8jDB5xOiuE&list=RDIh2gxLqR7HM
2013-11-27 20:01:51 +01:00
Jaime Marquínez Ferrándiz 652cdaa269 [youtube:playlist] Add support for YouTube mixes (fixes #1839) 2013-11-26 21:35:03 +01:00
Jaime Marquínez Ferrándiz e26f871228 Use the new '_download_xml' helper in more extractors 2013-11-26 19:17:25 +01:00
Jaime Marquínez Ferrándiz 6e47b51eef [youtube:playlist] Remove the link with index 0
It's not the first video of the playlist, it appears in the 'Play all' button (see the test course for an example)
2013-11-26 19:09:14 +01:00
Philipp Hagemeister ea36cbac5e Merge remote-tracking branch 'rbrito/swap-dimensions' 2013-11-25 06:19:15 +01:00
Jaime Marquínez Ferrándiz f459d17018 [youtube] Add an extractor for downloading the watch history (closes #1821) 2013-11-24 14:33:50 +01:00
Philipp Hagemeister 66ec019240 [youtube] do not use variable name twice 2013-11-24 06:54:26 +01:00
Philipp Hagemeister 7012b23c94 Match --download-archive during playlist processing (Fixes #1745) 2013-11-22 22:46:46 +01:00
Philipp Hagemeister 3f8ced5144 Merge remote-tracking branch 'jaimeMF/yt-playlists' 2013-11-22 20:11:54 +01:00
Philipp Hagemeister cb7dfeeac4 [youtube] only allow domain name to be upper-case (#1786) 2013-11-18 16:42:35 +01:00
Philipp Hagemeister 83aa529330 Support protocol-independent URLs (#1787) 2013-11-18 13:18:17 +01:00
Philipp Hagemeister fccd377198 Suppor embed-only videos (Fixes #1746) 2013-11-18 13:05:18 +01:00
Philipp Hagemeister 1d699755e0 [youtube] Add view_count (Fixes #1781) 2013-11-17 11:06:16 +01:00
Jaime Marquínez Ferrándiz b9643eed7c [youtube:channel] Fix the extraction of autogenerated channels
The ajax pages are empty, now it looks directly in the channel's /videos page
2013-11-15 11:51:45 +01:00
Philipp Hagemeister 0e145dd541 Merge branch 'master' of github.com:rg3/youtube-dl 2013-11-15 01:46:50 +01:00
Philipp Hagemeister 9f9be844fc [youtube] Fix protocol-independent URLs (Fixes #1768) 2013-11-15 01:45:39 +01:00
Jaime Marquínez Ferrándiz ca715127a2 Don't assume the 'subtitlesformat' is set in the params dict (fixes #1750) 2013-11-13 17:14:10 +01:00
Jaime Marquínez Ferrándiz 880e1c529d [youtube:playlist] Login into youtube if requested (fixes #1757)
Allows to download private playlists
2013-11-13 16:39:11 +01:00
Jaime Marquínez Ferrándiz dcbb45803f [youtube:playlist] Don't use the gdata api (closes #1508)
Parse the playlist pages instead
2013-11-13 16:26:50 +01:00
Jaime Marquínez Ferrándiz 231516b6c9 Merge pull request #1705 from iemejia/master
[ted] support for subtitles
2013-11-10 11:54:18 +01:00
Jaime Marquínez Ferrándiz 4f045eef8f [youtube:channel] Fix the extraction
The page don't include the 'load more' button anymore, now we directly get the 'c4_browse_ajax' pages.
2013-11-06 21:42:33 +01:00
Philipp Hagemeister 08fb86c49b [youtube] Add description for YoutubeSearchDateIE (#1710) 2013-11-03 15:59:10 +01:00
Philipp Hagemeister 3633d77c0f Merge remote-tracking branch 'CBGoodBuddy/ytsearchtime' 2013-11-03 15:56:55 +01:00
Jaime Marquínez Ferrándiz 9103bbc5cd Add the 'webpage_url' field to info_dict
The url for the video page, it must allow to reproduce the result.
It's automatically set by YoutubeDL if it's missing.
2013-11-03 12:11:13 +01:00
Craig Markwardt a3dd924871 Add YoutubeSearchDateIE extractor to youtube.py & __init__.py, which searches by publication date. 2013-11-02 22:40:48 -04:00
Jaime Marquínez Ferrándiz cf51923545 [youtube] Remove vevo test
The video is no longer available and it seems that vevo video don't use encrypted signatures anymore.
2013-11-02 20:46:26 +01:00
Ismaël Mejía 38db46794f Merge branch 'ted_subtitles' 2013-11-02 19:50:45 +01:00
Ismaël Mejía 1f343eaabb [subtitles] refactor to support websites with subtitle information the
webpage.

I added the parameter webpage, so now it's similar to the way automatic
captions are handled. This is an improvement needed for websites like
TED.
2013-11-02 19:29:25 +01:00
Philipp Hagemeister 8eddf3e91d [youtube] Encode subtitle track name in request (Fixes #1700) 2013-11-02 11:21:05 +01:00
Philipp Hagemeister 795f28f871 [youtube] Fix login (Fixes #1681) 2013-10-29 06:45:54 +01:00
Jaime Marquínez Ferrándiz 600cc1a4f0 [youtube] Set the format_id field to the itag of the format (closes #1624) 2013-10-25 17:17:46 +02:00
Jaime Marquínez Ferrándiz ea32fbacc8 Fix the extensions of two tests with youtube videos
The best quality is now a mp4 video.
2013-10-25 16:55:37 +02:00
Jaime Marquínez Ferrándiz 00fe14fc75 [youtube] Also use the 'adaptative_fmts' field from the /get_video_info page (fixes #1649)
The 'adaptative_fmts' field from the video page is not added to the 'url_encoded_fmt_stream_map'
2013-10-25 16:52:58 +02:00
Philipp Hagemeister 6803655ced Merge pull request #1622 from rbrito/fix-extension
extractor: youtube: Set extension of AAC audio formats to m4a.
2013-10-22 15:16:26 -07:00
Jaime Marquínez Ferrándiz f6a54188c2 [youtube] Use 'node is None' when checking if the video has automatic captions
It had stopped working and it reports a FutureWarning
2013-10-21 16:28:55 +02:00
Rogério Brito d5a9bb4ea9 extractor: youtube: Swap video dimensions to match standard practice.
While working on this, I thought about simplifying things like changing
480x854 to 480p, and that seemed like a good option, until I realized that
people (me included) usually link the concept of some number followed by a p
with the video being 16:9.

So, we would be losing some information and, as we all know,
[explicit is better than implicit][*].

[*]: http://www.python.org/dev/peps/pep-0020/

This closes #1446.

Signed-off-by: Rogério Brito <rbrito@ime.usp.br>
2013-10-19 14:04:44 -03:00
Rogério Brito f6f1fc9286 extractor: youtube: Fix extension of dash formats.
While we are at it, separate the audio formats from the video formats.

Signed-off-by: Rogério Brito <rbrito@ime.usp.br>
2013-10-18 18:53:00 -03:00
Rogério Brito 16f36a6fc9 extractor: youtube: Set extension of AAC audio formats to m4a.
This, in particular, eases downloading both audio and videos in DASH formats
before muxing them, which alleviates the problem that I exposed on issue

Furthermore, one may argue that this is, indeed, the case for correctness's
sake.

Signed-off-by: Rogério Brito <rbrito@ime.usp.br>
2013-10-18 17:50:55 -03:00
Philipp Hagemeister a733eb6c53 [youtube] Do not crash if caption info is missing altogether (Fixes #1610) 2013-10-17 02:19:19 +02:00
Jai Grimshaw 1fb07d10a3 [youtube] Adds #1312 Download annotations
Adds #1321 Download annotations from youtube
Annotations are downloaded and written to a .annotations.xml file using the https://www.youtube.com/annotations_invideo?features=1&legacy=1&video_id=$VIDEOID API.
Added unit test for annotations.
2013-10-14 16:22:27 +11:00
Philipp Hagemeister 57da92b7df [youtube] Do not recognize attribution link as user (Fixes #1573) 2013-10-09 23:50:38 +02:00
Jaime Marquínez Ferrándiz a34c2faae4 [youtube] set the 'name' parameter in the subtitles url (fixes #1577) 2013-10-09 16:41:36 +02:00
Philipp Hagemeister 15870e90b0 Restore warning when user forgets to quote URL (#1396) 2013-10-07 12:21:24 +02:00
Philipp Hagemeister cfadd183c4 Call extracted property age_limit everywhere 2013-10-06 16:23:06 +02:00
Philipp Hagemeister 8dbe9899a9 Allow users to specify an age limit (fixes #1545)
With these changes, users can now restrict what videos are downloaded by the intented audience, by specifying their age with --age-limit YEARS .
Add rudimentary support in youtube, pornotube, and youporn.
2013-10-06 06:08:56 +02:00
Philipp Hagemeister 7f8ae73a5d Include length in player cache ID
Some videos use the same player with IDs of multiple lengths.
See https://travis-ci.org/rg3/youtube-dl/jobs/12126506#L319 for an example.
2013-10-04 11:36:06 +02:00
Philipp Hagemeister c21315f273 [youtube] new static 82 signature 2013-10-04 00:43:01 +02:00
Philipp Hagemeister c38b1e776d [youtube] Simplify cache_dir code (#1529) 2013-10-02 08:41:14 +02:00
Philipp Hagemeister 4f8bf17f23 Merge remote-tracking branch 'holomorph/master' 2013-10-02 08:23:53 +02:00
Philipp Hagemeister ca40186c75 [youtube] Fix static 82 signature (Closes #1539) 2013-10-02 08:20:00 +02:00
Philipp Hagemeister a8c6b24155 [youtube] Support videos without a title (Fixes #1391, Closes #1542) 2013-10-02 07:25:35 +02:00
Filippo Valsorda bd8e5c7ca2 Merge pull request #1531 from rg3/no-playlist
[youtube] implement --no-playlist to only download current video
2013-10-01 10:08:20 -07:00
Filippo Valsorda 7c61bd36bb [youtube] correct --no-playlist for python3 2013-10-01 11:58:13 -04:00
Philipp Hagemeister 44d466559e Properly handle stream meap not being present 2013-10-01 14:44:09 +02:00
Philipp Hagemeister f10503db67 Handle videos without url_encoded_fmt_stream_map (Fixes #1535) 2013-10-01 11:39:11 +02:00
Mark Oteiza fa55675593 Support XDG base directory specification 2013-09-30 18:22:38 -04:00
Filippo Valsorda 47192f92d8 implement --no-playlist to only download current video - closes #755 2013-09-30 16:26:25 -04:00
Jaime Marquínez Ferrándiz 9abb32045a [youtube] Add hlsvp to the error message if it can't be found and remove the live stream test
It's no longer available, other olympics streams have the same problem.
2013-09-27 15:06:27 +02:00
Jaime Marquínez Ferrándiz f490e77e77 [youtube] Set the thumbnail to None if it can't be extracted 2013-09-27 14:22:36 +02:00
Jaime Marquínez Ferrándiz 2dc592991a [youtube] update description of test 2013-09-27 14:20:52 +02:00
Philipp Hagemeister 920de7a27d [youtube] Fix 83 signature (Closes #1511) 2013-09-27 06:15:21 +02:00
Philipp Hagemeister 695dc094ab Merge branch 'automatic-signatures' 2013-09-24 21:40:08 +02:00
Philipp Hagemeister f2c327fd39 Fix 86 signature (#1494) 2013-09-24 21:20:42 +02:00
Philipp Hagemeister e35e4ddc9a Fix output of --youtube-print-sig-code when counting down to 0 2013-09-24 21:18:03 +02:00
Philipp Hagemeister c3c88a2664 Allow opts.cachedir == None to disable cache 2013-09-24 21:04:43 +02:00
Jaime Marquínez Ferrándiz bb0eee71e7 [youtube] Update one of the test's description 2013-09-24 21:04:13 +02:00
Jaime Marquínez Ferrándiz 6f56389b88 [youtube] update algos for length 86 and 84 (fixes #1494) 2013-09-24 21:02:00 +02:00
Philipp Hagemeister c705320f48 Correct test strings 2013-09-22 12:18:16 +02:00
Philipp Hagemeister d2d8f89531 Do not warn if fallback is without alternatives (because we did not get the flash player URL) 2013-09-22 12:18:10 +02:00
Philipp Hagemeister bdde940e90 [youtube] Improve flash player URL handling 2013-09-22 12:17:42 +02:00
Philipp Hagemeister 13dc64ce74 [youtube] Remove _decrypt_signature_age_gate 2013-09-22 11:17:21 +02:00
Philipp Hagemeister f8061589e6 [youtube] Actually pass in cachedir option 2013-09-22 10:51:33 +02:00
Philipp Hagemeister 0ca96d48c7 [youtube] Improve source code quality 2013-09-22 10:37:23 +02:00
Philipp Hagemeister 4ba146f35d Update static signatures 2013-09-22 10:31:25 +02:00
Philipp Hagemeister edf3e38ebd [youtube] Improve cache and add an option to print the extracted signatures 2013-09-22 10:30:02 +02:00
Philipp Hagemeister c4417ddb61 [youtube] Add filesystem signature cache 2013-09-22 00:35:03 +02:00
Philipp Hagemeister 2f2ffea9ca Clarify a couple of calls 2013-09-21 15:34:29 +02:00
Philipp Hagemeister ba552f542f Use reader instead of indexing 2013-09-21 15:32:37 +02:00
Philipp Hagemeister 8379969834 Prepare signature function caching 2013-09-21 15:19:48 +02:00
Philipp Hagemeister 95dbd2f990 Change test target (Verified with node.js) 2013-09-21 15:10:38 +02:00
Philipp Hagemeister a7177865b1 Implement more opcodes 2013-09-21 14:48:12 +02:00
Philipp Hagemeister e0df6211cc Restore accidentally deleted commits
That's what happens if you let Windows machines write :(
2013-09-21 14:40:35 +02:00
Jaime Marquínez Ferrándiz 38d025b3f0 [youtube] add algo for length 91 2013-09-20 14:43:16 +02:00
patrickslin bc4b900898 Unable to decrypt signature length 93 (fixes #1461) 2013-09-19 21:49:06 -07:00
Jaime Marquínez Ferrándiz 71c82637e7 [youtube] apply the fix for lists with number of videos multiple of _MAX_RESULTS to user extraction
Copied from the playlist extractor.
2013-09-18 23:00:32 +02:00
Jaime Marquínez Ferrándiz 4a67aafb7e [youtube] Don't search the flash player version for videos with age gate activated 2013-09-17 20:59:55 +02:00
Philipp Hagemeister 6ae8ee3f54 Update 85 signature (Fixes #1449)
This is the first signature algorithm to have been parsed automatically, although that only works for HTML5 players for now, and is not yet integrated into master.
2013-09-17 16:59:13 +02:00
Jaime Marquínez Ferrándiz e69ae5b9e7 [youtube] support youtube.googleapis.com/v/* urls (fixes #1425) 2013-09-15 12:14:59 +02:00
Jaime Marquínez Ferrándiz ce85f022d2 [youtube] update algo for length 82 (fixes #1416) 2013-09-12 22:04:09 +02:00
Jaime Marquínez Ferrándiz e3dc22ca3a [youtube] Fix detection of videos with automatic captions 2013-09-11 19:24:56 +02:00
Jaime Marquínez Ferrándiz d665f8d3cb [subtitles] Also list the available automatic captions languages with '--list-sub' 2013-09-11 19:17:30 +02:00
Jaime Marquínez Ferrándiz 055e6f3657 [youtube] Support automatic captions with original language different from English (fixes #1225) and download in multiple languages. 2013-09-11 19:08:43 +02:00
Jaime Marquínez Ferrándiz 7fad1c6328 [subtitles] Use self._download_webpage for extracting the subtitles
It raises ExtractorError for the same exceptions we have to catch.
2013-09-11 16:24:47 +02:00
Jaime Marquínez Ferrándiz d82134c339 [subtitles] Simplify the extraction of subtitles in subclasses and remove NoAutoSubtitlesInfoExtractor
Subclasses just need to call the method extract_subtitles, which will call _extract_subtitles and _request_automatic_caption
Now the default implementation of _request_automatic_caption returns {}.
2013-09-11 16:05:49 +02:00
Jaime Marquínez Ferrándiz 54d39d8b2f [subtitles] rename SubitlesIE to SubtitlesInfoExtractor
Otherwise it can be automatically detected as a IE ready for use.
2013-09-11 15:51:04 +02:00
Jaime Marquínez Ferrándiz de7f3446e0 [youtube] move subtitles methods from the base extractor to YoutubeIE 2013-09-11 15:48:23 +02:00
Ismael Mejia f8e52269c1 [subtitles] made inheritance hierarchy flat as requested 2013-09-11 15:21:09 +02:00
Ismael Mejia cf1dd0c59e Merge branch 'master' into subtitles_rework 2013-09-11 14:26:48 +02:00
Jaime Marquínez Ferrándiz aa8f2641da [youtube] update algo for length 85 (fixes #1408 and fixes #1406) 2013-09-11 11:24:58 +02:00
Jaime Marquínez Ferrándiz 8963d9c266 [youtube] Modify the regex to match ids of length 11 (fixes #1396)
In urls like http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930 you can't split the query string and ids always have that length.
2013-09-09 10:33:12 +02:00
Jaime Marquínez Ferrándiz 890f62e868 Revert "[youtube] Fix detection of tags from HLS videos."
They have undo the change

This reverts commit 0638ad9999.
2013-09-08 18:50:07 +02:00
Ismael Mejia 72836fcee4 Merge branch 'master' into subtitles_rework 2013-09-06 23:24:41 +02:00
Ismael Mejia d6e203b3dc [subtitles] fixed multiple subtitles language separated by comma after merge
As mentioned in the pull request, I forgot to include this changes.
aa6a10c44a
2013-09-06 16:30:13 +02:00
Jaime Marquínez Ferrándiz e3ea479087 [youtube] Fix some issues with the detection of playlist/channel urls (reported in #1374)
They were being caught by YoutubeUserIE, now it only extracts a url if the rest of extractors aren't suitable.
Now the url tests check that the urls can only be extracted with an specific extractor.
2013-09-06 16:24:24 +02:00
Jaime Marquínez Ferrándiz faab1d3836 [youtube] Fix detection of feeds urls (fixes #1294)
Urls like https://www.youtube.com/feed/watch_later were being as users (before the last changes to YoutubeUserIE, as videos)
2013-09-06 14:45:49 +02:00
Jaime Marquínez Ferrándiz fd9cf73836 [youtube] Users: download from the api in json to simplify extraction (fixes #1358)
There could be duplicate videos or other videos if the description have links.
2013-09-06 10:43:02 +02:00
Jaime Marquínez Ferrándiz 0638ad9999 [youtube] Fix detection of tags from HLS videos. 2013-09-06 10:25:31 +02:00
patrickslin 1cf911bc82 Vevo sig changed again, please update for us! Thanks very much! (fixes #1375) 2013-09-05 17:38:03 -07:00
Jaime Marquínez Ferrándiz f4b052321b [youtube] Urls like youtube.com/NASA are now interpreted as users (fixes #1069)
Video urls like http://youtube.com/BaW_jenozKc are not valid, but http://youtu.be/BaW_jenozKc is correct.
2013-09-05 22:39:15 +02:00
Jaime Marquínez Ferrándiz c215217e39 [youtube] Playlists: extract the videos id from ['media$group']['yt$videoid'] (fixes #1374)
'media$player' is not defined for private videos.
2013-09-05 21:40:04 +02:00
Allan Zhou 96fb5605b2 AHLS -> Apple HTTP Live Streaming 2013-09-03 18:49:35 -07:00
Allan Zhou 85f03346eb Merge remote-tracking branch 'upstream/master' 2013-08-30 17:51:59 -07:00
Allan Zhou bdc6b3fc64 add support for "-f mp4" for YouTube 2013-08-30 17:51:50 -07:00
Jaime Marquínez Ferrándiz 2e756879f1 [youtube] update algo for length 86 2013-08-30 20:49:51 +02:00
Jaime Marquínez Ferrándiz c7a7750d3b [youtube] Fix typo in the _VALID_URL for YoutubeFavouritesIE, it was intended to also match :ytfavourites 2013-08-30 20:13:05 +02:00
Jaime Marquínez Ferrándiz 23b00bc0e4 [youtube] update algo for length 84
Only appears sometimes, nearly identical to length 86.
2013-08-29 22:44:29 +02:00
Jaime Marquínez Ferrándiz 52e1eea18b [youtube] update algo for length 86 (fixes #1349) 2013-08-29 22:33:58 +02:00
Jaime Marquínez Ferrándiz c7bf7366bc Update descriptions checksum for some test for Unistra and Youtube 2013-08-29 13:41:59 +02:00
Jaime Marquínez Ferrándiz 3e223834d9 [youtube] update algo for length 88, thanks to @Ramhack (fixes #1328) 2013-08-28 10:26:44 +02:00
Ismael Mejia 06a401c845 Merge branch 'master' into subtitles_rework 2013-08-28 00:33:12 +02:00
Jaime Marquínez Ferrándiz f2aeefe29c [youtube] update algo for length 84 2013-08-24 10:48:12 +02:00
Jaime Marquínez Ferrándiz 306170518f [youtube] update algo for length 86 (fixes #1302) 2013-08-23 22:36:59 +02:00
Jaime Marquínez Ferrándiz aa6a10c44a Allow to specify multiple subtitles languages separated by commas (closes #518) 2013-08-23 18:34:57 +02:00