mirror of
https://codeberg.org/polarisfm/youtube-dl
synced 2025-01-08 14:17:54 +01:00
Merge branch 'master' of https://github.com/rg3/youtube-dl
This commit is contained in:
commit
edc7e70d9f
6
.github/ISSUE_TEMPLATE/1_broken_site.md
vendored
6
.github/ISSUE_TEMPLATE/1_broken_site.md
vendored
@ -18,7 +18,7 @@ title: ''
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
||||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2019.07.02. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2019.07.30. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||||
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
||||||
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
|
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
|
||||||
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
||||||
@ -26,7 +26,7 @@ Carefully read and work through this check list in order to prevent the most com
|
|||||||
-->
|
-->
|
||||||
|
|
||||||
- [ ] I'm reporting a broken site support
|
- [ ] I'm reporting a broken site support
|
||||||
- [ ] I've verified that I'm running youtube-dl version **2019.07.02**
|
- [ ] I've verified that I'm running youtube-dl version **2019.07.30**
|
||||||
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
||||||
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
|
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
|
||||||
- [ ] I've searched the bugtracker for similar issues including closed ones
|
- [ ] I've searched the bugtracker for similar issues including closed ones
|
||||||
@ -41,7 +41,7 @@ Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <
|
|||||||
[debug] User config: []
|
[debug] User config: []
|
||||||
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
||||||
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
||||||
[debug] youtube-dl version 2019.07.02
|
[debug] youtube-dl version 2019.07.30
|
||||||
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
||||||
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
||||||
[debug] Proxy map: {}
|
[debug] Proxy map: {}
|
||||||
|
@ -19,7 +19,7 @@ labels: 'site-support-request'
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
||||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2019.07.02. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2019.07.30. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||||
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
||||||
- Make sure that site you are requesting is not dedicated to copyright infringement, see https://yt-dl.org/copyright-infringement. youtube-dl does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
|
- Make sure that site you are requesting is not dedicated to copyright infringement, see https://yt-dl.org/copyright-infringement. youtube-dl does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
|
||||||
- Search the bugtracker for similar site support requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
- Search the bugtracker for similar site support requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
||||||
@ -27,7 +27,7 @@ Carefully read and work through this check list in order to prevent the most com
|
|||||||
-->
|
-->
|
||||||
|
|
||||||
- [ ] I'm reporting a new site support request
|
- [ ] I'm reporting a new site support request
|
||||||
- [ ] I've verified that I'm running youtube-dl version **2019.07.02**
|
- [ ] I've verified that I'm running youtube-dl version **2019.07.30**
|
||||||
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
||||||
- [ ] I've checked that none of provided URLs violate any copyrights
|
- [ ] I've checked that none of provided URLs violate any copyrights
|
||||||
- [ ] I've searched the bugtracker for similar site support requests including closed ones
|
- [ ] I've searched the bugtracker for similar site support requests including closed ones
|
||||||
|
@ -18,13 +18,13 @@ title: ''
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
||||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2019.07.02. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2019.07.30. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||||
- Search the bugtracker for similar site feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
- Search the bugtracker for similar site feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
||||||
- Finally, put x into all relevant boxes (like this [x])
|
- Finally, put x into all relevant boxes (like this [x])
|
||||||
-->
|
-->
|
||||||
|
|
||||||
- [ ] I'm reporting a site feature request
|
- [ ] I'm reporting a site feature request
|
||||||
- [ ] I've verified that I'm running youtube-dl version **2019.07.02**
|
- [ ] I've verified that I'm running youtube-dl version **2019.07.30**
|
||||||
- [ ] I've searched the bugtracker for similar site feature requests including closed ones
|
- [ ] I've searched the bugtracker for similar site feature requests including closed ones
|
||||||
|
|
||||||
|
|
||||||
|
6
.github/ISSUE_TEMPLATE/4_bug_report.md
vendored
6
.github/ISSUE_TEMPLATE/4_bug_report.md
vendored
@ -18,7 +18,7 @@ title: ''
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
||||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2019.07.02. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2019.07.30. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||||
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
- Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
|
||||||
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
|
- Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in http://yt-dl.org/escape.
|
||||||
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
- Search the bugtracker for similar issues: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
||||||
@ -27,7 +27,7 @@ Carefully read and work through this check list in order to prevent the most com
|
|||||||
-->
|
-->
|
||||||
|
|
||||||
- [ ] I'm reporting a broken site support issue
|
- [ ] I'm reporting a broken site support issue
|
||||||
- [ ] I've verified that I'm running youtube-dl version **2019.07.02**
|
- [ ] I've verified that I'm running youtube-dl version **2019.07.30**
|
||||||
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
- [ ] I've checked that all provided URLs are alive and playable in a browser
|
||||||
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
|
- [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
|
||||||
- [ ] I've searched the bugtracker for similar bug reports including closed ones
|
- [ ] I've searched the bugtracker for similar bug reports including closed ones
|
||||||
@ -43,7 +43,7 @@ Add the `-v` flag to your command line you run youtube-dl with (`youtube-dl -v <
|
|||||||
[debug] User config: []
|
[debug] User config: []
|
||||||
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
|
||||||
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
|
||||||
[debug] youtube-dl version 2019.07.02
|
[debug] youtube-dl version 2019.07.30
|
||||||
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
|
||||||
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
|
||||||
[debug] Proxy map: {}
|
[debug] Proxy map: {}
|
||||||
|
4
.github/ISSUE_TEMPLATE/5_feature_request.md
vendored
4
.github/ISSUE_TEMPLATE/5_feature_request.md
vendored
@ -19,13 +19,13 @@ labels: 'request'
|
|||||||
|
|
||||||
<!--
|
<!--
|
||||||
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
Carefully read and work through this check list in order to prevent the most common mistakes and misuse of youtube-dl:
|
||||||
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2019.07.02. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
- First of, make sure you are using the latest version of youtube-dl. Run `youtube-dl --version` and ensure your version is 2019.07.30. If it's not, see https://yt-dl.org/update on how to update. Issues with outdated version will be REJECTED.
|
||||||
- Search the bugtracker for similar feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
- Search the bugtracker for similar feature requests: http://yt-dl.org/search-issues. DO NOT post duplicates.
|
||||||
- Finally, put x into all relevant boxes (like this [x])
|
- Finally, put x into all relevant boxes (like this [x])
|
||||||
-->
|
-->
|
||||||
|
|
||||||
- [ ] I'm reporting a feature request
|
- [ ] I'm reporting a feature request
|
||||||
- [ ] I've verified that I'm running youtube-dl version **2019.07.02**
|
- [ ] I've verified that I'm running youtube-dl version **2019.07.30**
|
||||||
- [ ] I've searched the bugtracker for similar feature requests including closed ones
|
- [ ] I've searched the bugtracker for similar feature requests including closed ones
|
||||||
|
|
||||||
|
|
||||||
|
81
ChangeLog
81
ChangeLog
@ -1,3 +1,84 @@
|
|||||||
|
version 2019.07.30
|
||||||
|
|
||||||
|
Extractors
|
||||||
|
* [youtube] Fix and improve title and description extraction (#21934)
|
||||||
|
|
||||||
|
|
||||||
|
version 2019.07.27
|
||||||
|
|
||||||
|
Extractors
|
||||||
|
+ [yahoo:japannews] Add support for yahoo.co.jp (#21698, #21265)
|
||||||
|
+ [discovery] Add support go.discovery.com URLs
|
||||||
|
* [youtube:playlist] Relax video regular expression (#21844)
|
||||||
|
* [generic] Restrict --default-search schemeless URLs detection pattern
|
||||||
|
(#21842)
|
||||||
|
* [vrv] Fix CMS signing query extraction (#21809)
|
||||||
|
|
||||||
|
|
||||||
|
version 2019.07.16
|
||||||
|
|
||||||
|
Extractors
|
||||||
|
+ [asiancrush] Add support for yuyutv.com, midnightpulp.com and cocoro.tv
|
||||||
|
(#21281, #21290)
|
||||||
|
* [kaltura] Check source format URL (#21290)
|
||||||
|
* [ctsnews] Fix YouTube embeds extraction (#21678)
|
||||||
|
+ [einthusan] Add support for einthusan.com (#21748, #21775)
|
||||||
|
+ [youtube] Add support for invidious.mastodon.host (#21777)
|
||||||
|
+ [gfycat] Extend URL regular expression (#21779, #21780)
|
||||||
|
* [youtube] Restrict is_live extraction (#21782)
|
||||||
|
|
||||||
|
|
||||||
|
version 2019.07.14
|
||||||
|
|
||||||
|
Extractors
|
||||||
|
* [porn91] Fix extraction (#21312)
|
||||||
|
+ [yandexmusic] Extract track number and disk number (#21421)
|
||||||
|
+ [yandexmusic] Add support for multi disk albums (#21420, #21421)
|
||||||
|
* [lynda] Handle missing subtitles (#20490, #20513)
|
||||||
|
+ [youtube] Add more invidious instances to URL regular expression (#21694)
|
||||||
|
* [twitter] Improve uploader id extraction (#21705)
|
||||||
|
* [spankbang] Fix and improve metadata extraction
|
||||||
|
* [spankbang] Fix extraction (#21763, #21764)
|
||||||
|
+ [dlive] Add support for dlive.tv (#18080)
|
||||||
|
+ [livejournal] Add support for livejournal.com (#21526)
|
||||||
|
* [roosterteeth] Fix free episode extraction (#16094)
|
||||||
|
* [dbtv] Fix extraction
|
||||||
|
* [bellator] Fix extraction
|
||||||
|
- [rudo] Remove extractor (#18430, #18474)
|
||||||
|
* [facebook] Fallback to twitter:image meta for thumbnail extraction (#21224)
|
||||||
|
* [bleacherreport] Fix Bleacher Report CMS extraction
|
||||||
|
* [espn] Fix fivethirtyeight.com extraction
|
||||||
|
* [5tv] Relax video URL regular expression and support https URLs
|
||||||
|
* [youtube] Fix is_live extraction (#21734)
|
||||||
|
* [youtube] Fix authentication (#11270)
|
||||||
|
|
||||||
|
|
||||||
|
version 2019.07.12
|
||||||
|
|
||||||
|
Core
|
||||||
|
+ [adobepass] Add support for AT&T U-verse (mso ATT) (#13938, #21016)
|
||||||
|
|
||||||
|
Extractors
|
||||||
|
+ [mgtv] Pass Referer HTTP header for format URLs (#21726)
|
||||||
|
+ [beeg] Add support for api/v6 v2 URLs without t argument (#21701)
|
||||||
|
* [voxmedia:volume] Improvevox embed extraction (#16846)
|
||||||
|
* [funnyordie] Move extraction to VoxMedia extractor (#16846)
|
||||||
|
* [gameinformer] Fix extraction (#8895, #15363, #17206)
|
||||||
|
* [funk] Fix extraction (#17915)
|
||||||
|
* [packtpub] Relax lesson URL regular expression (#21695)
|
||||||
|
* [packtpub] Fix extraction (#21268)
|
||||||
|
* [philharmoniedeparis] Relax URL regular expression (#21672)
|
||||||
|
* [peertube] Detect embed URLs in generic extraction (#21666)
|
||||||
|
* [mixer:vod] Relax URL regular expression (#21657, #21658)
|
||||||
|
+ [lecturio] Add support id based URLs (#21630)
|
||||||
|
+ [go] Add site info for disneynow (#21613)
|
||||||
|
* [ted] Restrict info regular expression (#21631)
|
||||||
|
* [twitch:vod] Actualize m3u8 URL (#21538, #21607)
|
||||||
|
* [vzaar] Fix videos with empty title (#21606)
|
||||||
|
* [tvland] Fix extraction (#21384)
|
||||||
|
* [arte] Clean extractor (#15583, #21614)
|
||||||
|
|
||||||
|
|
||||||
version 2019.07.02
|
version 2019.07.02
|
||||||
|
|
||||||
Core
|
Core
|
||||||
|
@ -58,16 +58,8 @@
|
|||||||
- **ARD:mediathek**
|
- **ARD:mediathek**
|
||||||
- **ARDBetaMediathek**
|
- **ARDBetaMediathek**
|
||||||
- **Arkena**
|
- **Arkena**
|
||||||
- **arte.tv**
|
|
||||||
- **arte.tv:+7**
|
- **arte.tv:+7**
|
||||||
- **arte.tv:cinema**
|
|
||||||
- **arte.tv:concert**
|
|
||||||
- **arte.tv:creative**
|
|
||||||
- **arte.tv:ddc**
|
|
||||||
- **arte.tv:embed**
|
- **arte.tv:embed**
|
||||||
- **arte.tv:future**
|
|
||||||
- **arte.tv:info**
|
|
||||||
- **arte.tv:magazine**
|
|
||||||
- **arte.tv:playlist**
|
- **arte.tv:playlist**
|
||||||
- **AsianCrush**
|
- **AsianCrush**
|
||||||
- **AsianCrushPlaylist**
|
- **AsianCrushPlaylist**
|
||||||
@ -231,6 +223,8 @@
|
|||||||
- **DiscoveryNetworksDe**
|
- **DiscoveryNetworksDe**
|
||||||
- **DiscoveryVR**
|
- **DiscoveryVR**
|
||||||
- **Disney**
|
- **Disney**
|
||||||
|
- **dlive:stream**
|
||||||
|
- **dlive:vod**
|
||||||
- **Dotsub**
|
- **Dotsub**
|
||||||
- **DouyuShow**
|
- **DouyuShow**
|
||||||
- **DouyuTV**: 斗鱼
|
- **DouyuTV**: 斗鱼
|
||||||
@ -313,9 +307,7 @@
|
|||||||
- **FrontendMastersCourse**
|
- **FrontendMastersCourse**
|
||||||
- **FrontendMastersLesson**
|
- **FrontendMastersLesson**
|
||||||
- **Funimation**
|
- **Funimation**
|
||||||
- **FunkChannel**
|
- **Funk**
|
||||||
- **FunkMix**
|
|
||||||
- **FunnyOrDie**
|
|
||||||
- **Fusion**
|
- **Fusion**
|
||||||
- **Fux**
|
- **Fux**
|
||||||
- **FXNetworks**
|
- **FXNetworks**
|
||||||
@ -458,6 +450,7 @@
|
|||||||
- **linkedin:learning:course**
|
- **linkedin:learning:course**
|
||||||
- **LinuxAcademy**
|
- **LinuxAcademy**
|
||||||
- **LiTV**
|
- **LiTV**
|
||||||
|
- **LiveJournal**
|
||||||
- **LiveLeak**
|
- **LiveLeak**
|
||||||
- **LiveLeakEmbed**
|
- **LiveLeakEmbed**
|
||||||
- **livestream**
|
- **livestream**
|
||||||
@ -764,7 +757,6 @@
|
|||||||
- **rtve.es:television**
|
- **rtve.es:television**
|
||||||
- **RTVNH**
|
- **RTVNH**
|
||||||
- **RTVS**
|
- **RTVS**
|
||||||
- **Rudo**
|
|
||||||
- **RUHD**
|
- **RUHD**
|
||||||
- **rutube**: Rutube videos
|
- **rutube**: Rutube videos
|
||||||
- **rutube:channel**: Rutube channels
|
- **rutube:channel**: Rutube channels
|
||||||
@ -896,7 +888,6 @@
|
|||||||
- **TF1**
|
- **TF1**
|
||||||
- **TFO**
|
- **TFO**
|
||||||
- **TheIntercept**
|
- **TheIntercept**
|
||||||
- **theoperaplatform**
|
|
||||||
- **ThePlatform**
|
- **ThePlatform**
|
||||||
- **ThePlatformFeed**
|
- **ThePlatformFeed**
|
||||||
- **TheScene**
|
- **TheScene**
|
||||||
@ -1126,6 +1117,7 @@
|
|||||||
- **Yahoo**: Yahoo screen and movies
|
- **Yahoo**: Yahoo screen and movies
|
||||||
- **yahoo:gyao**
|
- **yahoo:gyao**
|
||||||
- **yahoo:gyao:player**
|
- **yahoo:gyao:player**
|
||||||
|
- **yahoo:japannews**: Yahoo! Japan News
|
||||||
- **YandexDisk**
|
- **YandexDisk**
|
||||||
- **yandexmusic:album**: Яндекс.Музыка - Альбом
|
- **yandexmusic:album**: Яндекс.Музыка - Альбом
|
||||||
- **yandexmusic:playlist**: Яндекс.Музыка - Плейлист
|
- **yandexmusic:playlist**: Яндекс.Музыка - Плейлист
|
||||||
|
@ -53,7 +53,7 @@ class DashSegmentsFD(FragmentFD):
|
|||||||
except compat_urllib_error.HTTPError as err:
|
except compat_urllib_error.HTTPError as err:
|
||||||
# YouTube may often return 404 HTTP error for a fragment causing the
|
# YouTube may often return 404 HTTP error for a fragment causing the
|
||||||
# whole download to fail. However if the same fragment is immediately
|
# whole download to fail. However if the same fragment is immediately
|
||||||
# retried with the same request data this usually succeeds (1-2 attemps
|
# retried with the same request data this usually succeeds (1-2 attempts
|
||||||
# is usually enough) thus allowing to download the whole file successfully.
|
# is usually enough) thus allowing to download the whole file successfully.
|
||||||
# To be future-proof we will retry all fragments that fail with any
|
# To be future-proof we will retry all fragments that fail with any
|
||||||
# HTTP error.
|
# HTTP error.
|
||||||
|
@ -146,7 +146,7 @@ def write_piff_header(stream, params):
|
|||||||
sps, pps = codec_private_data.split(u32.pack(1))[1:]
|
sps, pps = codec_private_data.split(u32.pack(1))[1:]
|
||||||
avcc_payload = u8.pack(1) # configuration version
|
avcc_payload = u8.pack(1) # configuration version
|
||||||
avcc_payload += sps[1:4] # avc profile indication + profile compatibility + avc level indication
|
avcc_payload += sps[1:4] # avc profile indication + profile compatibility + avc level indication
|
||||||
avcc_payload += u8.pack(0xfc | (params.get('nal_unit_length_field', 4) - 1)) # complete represenation (1) + reserved (11111) + length size minus one
|
avcc_payload += u8.pack(0xfc | (params.get('nal_unit_length_field', 4) - 1)) # complete representation (1) + reserved (11111) + length size minus one
|
||||||
avcc_payload += u8.pack(1) # reserved (0) + number of sps (0000001)
|
avcc_payload += u8.pack(1) # reserved (0) + number of sps (0000001)
|
||||||
avcc_payload += u16.pack(len(sps))
|
avcc_payload += u16.pack(len(sps))
|
||||||
avcc_payload += sps
|
avcc_payload += sps
|
||||||
|
@ -15,10 +15,13 @@ class AbcNewsVideoIE(AMPIE):
|
|||||||
IE_NAME = 'abcnews:video'
|
IE_NAME = 'abcnews:video'
|
||||||
_VALID_URL = r'''(?x)
|
_VALID_URL = r'''(?x)
|
||||||
https?://
|
https?://
|
||||||
abcnews\.go\.com/
|
|
||||||
(?:
|
(?:
|
||||||
[^/]+/video/(?P<display_id>[0-9a-z-]+)-|
|
abcnews\.go\.com/
|
||||||
video/embed\?.*?\bid=
|
(?:
|
||||||
|
[^/]+/video/(?P<display_id>[0-9a-z-]+)-|
|
||||||
|
video/embed\?.*?\bid=
|
||||||
|
)|
|
||||||
|
fivethirtyeight\.abcnews\.go\.com/video/embed/\d+/
|
||||||
)
|
)
|
||||||
(?P<id>\d+)
|
(?P<id>\d+)
|
||||||
'''
|
'''
|
||||||
|
@ -5,14 +5,12 @@ import re
|
|||||||
|
|
||||||
from .common import InfoExtractor
|
from .common import InfoExtractor
|
||||||
from .kaltura import KalturaIE
|
from .kaltura import KalturaIE
|
||||||
from ..utils import (
|
from ..utils import extract_attributes
|
||||||
extract_attributes,
|
|
||||||
remove_end,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class AsianCrushIE(InfoExtractor):
|
class AsianCrushIE(InfoExtractor):
|
||||||
_VALID_URL = r'https?://(?:www\.)?asiancrush\.com/video/(?:[^/]+/)?0+(?P<id>\d+)v\b'
|
_VALID_URL_BASE = r'https?://(?:www\.)?(?P<host>(?:(?:asiancrush|yuyutv|midnightpulp)\.com|cocoro\.tv))'
|
||||||
|
_VALID_URL = r'%s/video/(?:[^/]+/)?0+(?P<id>\d+)v\b' % _VALID_URL_BASE
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'https://www.asiancrush.com/video/012869v/women-who-flirt/',
|
'url': 'https://www.asiancrush.com/video/012869v/women-who-flirt/',
|
||||||
'md5': 'c3b740e48d0ba002a42c0b72857beae6',
|
'md5': 'c3b740e48d0ba002a42c0b72857beae6',
|
||||||
@ -20,7 +18,7 @@ class AsianCrushIE(InfoExtractor):
|
|||||||
'id': '1_y4tmjm5r',
|
'id': '1_y4tmjm5r',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'title': 'Women Who Flirt',
|
'title': 'Women Who Flirt',
|
||||||
'description': 'md5:3db14e9186197857e7063522cb89a805',
|
'description': 'md5:7e986615808bcfb11756eb503a751487',
|
||||||
'timestamp': 1496936429,
|
'timestamp': 1496936429,
|
||||||
'upload_date': '20170608',
|
'upload_date': '20170608',
|
||||||
'uploader_id': 'craig@crifkin.com',
|
'uploader_id': 'craig@crifkin.com',
|
||||||
@ -28,10 +26,27 @@ class AsianCrushIE(InfoExtractor):
|
|||||||
}, {
|
}, {
|
||||||
'url': 'https://www.asiancrush.com/video/she-was-pretty/011886v-pretty-episode-3/',
|
'url': 'https://www.asiancrush.com/video/she-was-pretty/011886v-pretty-episode-3/',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://www.yuyutv.com/video/013886v/the-act-of-killing/',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://www.yuyutv.com/video/peep-show/013922v-warring-factions/',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://www.midnightpulp.com/video/010400v/drifters/',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://www.midnightpulp.com/video/mononoke/016378v-zashikiwarashi-part-1/',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://www.cocoro.tv/video/the-wonderful-wizard-of-oz/008878v-the-wonderful-wizard-of-oz-ep01/',
|
||||||
|
'only_matching': True,
|
||||||
}]
|
}]
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
video_id = self._match_id(url)
|
mobj = re.match(self._VALID_URL, url)
|
||||||
|
host = mobj.group('host')
|
||||||
|
video_id = mobj.group('id')
|
||||||
|
|
||||||
webpage = self._download_webpage(url, video_id)
|
webpage = self._download_webpage(url, video_id)
|
||||||
|
|
||||||
@ -51,7 +66,7 @@ class AsianCrushIE(InfoExtractor):
|
|||||||
r'\bentry_id["\']\s*:\s*["\'](\d+)', webpage, 'entry id')
|
r'\bentry_id["\']\s*:\s*["\'](\d+)', webpage, 'entry id')
|
||||||
|
|
||||||
player = self._download_webpage(
|
player = self._download_webpage(
|
||||||
'https://api.asiancrush.com/embeddedVideoPlayer', video_id,
|
'https://api.%s/embeddedVideoPlayer' % host, video_id,
|
||||||
query={'id': entry_id})
|
query={'id': entry_id})
|
||||||
|
|
||||||
kaltura_id = self._search_regex(
|
kaltura_id = self._search_regex(
|
||||||
@ -63,15 +78,23 @@ class AsianCrushIE(InfoExtractor):
|
|||||||
r'/p(?:artner_id)?/(\d+)', player, 'partner id',
|
r'/p(?:artner_id)?/(\d+)', player, 'partner id',
|
||||||
default='513551')
|
default='513551')
|
||||||
|
|
||||||
return self.url_result(
|
description = self._html_search_regex(
|
||||||
'kaltura:%s:%s' % (partner_id, kaltura_id),
|
r'(?s)<div[^>]+\bclass=["\']description["\'][^>]*>(.+?)</div>',
|
||||||
ie=KalturaIE.ie_key(), video_id=kaltura_id,
|
webpage, 'description', fatal=False)
|
||||||
video_title=title)
|
|
||||||
|
return {
|
||||||
|
'_type': 'url_transparent',
|
||||||
|
'url': 'kaltura:%s:%s' % (partner_id, kaltura_id),
|
||||||
|
'ie_key': KalturaIE.ie_key(),
|
||||||
|
'id': video_id,
|
||||||
|
'title': title,
|
||||||
|
'description': description,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
class AsianCrushPlaylistIE(InfoExtractor):
|
class AsianCrushPlaylistIE(InfoExtractor):
|
||||||
_VALID_URL = r'https?://(?:www\.)?asiancrush\.com/series/0+(?P<id>\d+)s\b'
|
_VALID_URL = r'%s/series/0+(?P<id>\d+)s\b' % AsianCrushIE._VALID_URL_BASE
|
||||||
_TEST = {
|
_TESTS = [{
|
||||||
'url': 'https://www.asiancrush.com/series/012481s/scholar-walks-night/',
|
'url': 'https://www.asiancrush.com/series/012481s/scholar-walks-night/',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '12481',
|
'id': '12481',
|
||||||
@ -79,7 +102,16 @@ class AsianCrushPlaylistIE(InfoExtractor):
|
|||||||
'description': 'md5:7addd7c5132a09fd4741152d96cce886',
|
'description': 'md5:7addd7c5132a09fd4741152d96cce886',
|
||||||
},
|
},
|
||||||
'playlist_count': 20,
|
'playlist_count': 20,
|
||||||
}
|
}, {
|
||||||
|
'url': 'https://www.yuyutv.com/series/013920s/peep-show/',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://www.midnightpulp.com/series/016375s/mononoke/',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://www.cocoro.tv/series/008549s/the-wonderful-wizard-of-oz/',
|
||||||
|
'only_matching': True,
|
||||||
|
}]
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
playlist_id = self._match_id(url)
|
playlist_id = self._match_id(url)
|
||||||
@ -96,15 +128,15 @@ class AsianCrushPlaylistIE(InfoExtractor):
|
|||||||
entries.append(self.url_result(
|
entries.append(self.url_result(
|
||||||
mobj.group('url'), ie=AsianCrushIE.ie_key()))
|
mobj.group('url'), ie=AsianCrushIE.ie_key()))
|
||||||
|
|
||||||
title = remove_end(
|
title = self._html_search_regex(
|
||||||
self._html_search_regex(
|
r'(?s)<h1\b[^>]\bid=["\']movieTitle[^>]+>(.+?)</h1>', webpage,
|
||||||
r'(?s)<h1\b[^>]\bid=["\']movieTitle[^>]+>(.+?)</h1>', webpage,
|
'title', default=None) or self._og_search_title(
|
||||||
'title', default=None) or self._og_search_title(
|
webpage, default=None) or self._html_search_meta(
|
||||||
webpage, default=None) or self._html_search_meta(
|
'twitter:title', webpage, 'title',
|
||||||
'twitter:title', webpage, 'title',
|
default=None) or self._search_regex(
|
||||||
default=None) or self._search_regex(
|
r'<title>([^<]+)</title>', webpage, 'title', fatal=False)
|
||||||
r'<title>([^<]+)</title>', webpage, 'title', fatal=False),
|
if title:
|
||||||
' | AsianCrush')
|
title = re.sub(r'\s*\|\s*.+?$', '', title)
|
||||||
|
|
||||||
description = self._og_search_description(
|
description = self._og_search_description(
|
||||||
webpage, default=None) or self._html_search_meta(
|
webpage, default=None) or self._html_search_meta(
|
||||||
|
@ -32,6 +32,10 @@ class BeegIE(InfoExtractor):
|
|||||||
# api/v6 v2
|
# api/v6 v2
|
||||||
'url': 'https://beeg.com/1941093077?t=911-1391',
|
'url': 'https://beeg.com/1941093077?t=911-1391',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
# api/v6 v2 w/o t
|
||||||
|
'url': 'https://beeg.com/1277207756',
|
||||||
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
'url': 'https://beeg.porn/video/5416503',
|
'url': 'https://beeg.porn/video/5416503',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
@ -49,14 +53,17 @@ class BeegIE(InfoExtractor):
|
|||||||
r'beeg_version\s*=\s*([\da-zA-Z_-]+)', webpage, 'beeg version',
|
r'beeg_version\s*=\s*([\da-zA-Z_-]+)', webpage, 'beeg version',
|
||||||
default='1546225636701')
|
default='1546225636701')
|
||||||
|
|
||||||
qs = compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
|
if len(video_id) >= 10:
|
||||||
t = qs.get('t', [''])[0].split('-')
|
|
||||||
if len(t) > 1:
|
|
||||||
query = {
|
query = {
|
||||||
'v': 2,
|
'v': 2,
|
||||||
's': t[0],
|
|
||||||
'e': t[1],
|
|
||||||
}
|
}
|
||||||
|
qs = compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
|
||||||
|
t = qs.get('t', [''])[0].split('-')
|
||||||
|
if len(t) > 1:
|
||||||
|
query.update({
|
||||||
|
's': t[0],
|
||||||
|
'e': t[1],
|
||||||
|
})
|
||||||
else:
|
else:
|
||||||
query = {'v': 1}
|
query = {'v': 1}
|
||||||
|
|
||||||
|
@ -6,7 +6,6 @@ from ..utils import (
|
|||||||
ExtractorError,
|
ExtractorError,
|
||||||
remove_end,
|
remove_end,
|
||||||
)
|
)
|
||||||
from .rudo import RudoIE
|
|
||||||
|
|
||||||
|
|
||||||
class BioBioChileTVIE(InfoExtractor):
|
class BioBioChileTVIE(InfoExtractor):
|
||||||
@ -41,11 +40,15 @@ class BioBioChileTVIE(InfoExtractor):
|
|||||||
}, {
|
}, {
|
||||||
'url': 'http://www.biobiochile.cl/noticias/bbtv/comentarios-bio-bio/2016/07/08/edecanes-del-congreso-figuras-decorativas-que-le-cuestan-muy-caro-a-los-chilenos.shtml',
|
'url': 'http://www.biobiochile.cl/noticias/bbtv/comentarios-bio-bio/2016/07/08/edecanes-del-congreso-figuras-decorativas-que-le-cuestan-muy-caro-a-los-chilenos.shtml',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'edecanes-del-congreso-figuras-decorativas-que-le-cuestan-muy-caro-a-los-chilenos',
|
'id': 'b4xd0LK3SK',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'uploader': '(none)',
|
# TODO: fix url_transparent information overriding
|
||||||
'upload_date': '20160708',
|
# 'uploader': 'Juan Pablo Echenique',
|
||||||
'title': 'Edecanes del Congreso: Figuras decorativas que le cuestan muy caro a los chilenos',
|
'title': 'Comentario Oscar Cáceres',
|
||||||
|
},
|
||||||
|
'params': {
|
||||||
|
# empty m3u8 manifest
|
||||||
|
'skip_download': True,
|
||||||
},
|
},
|
||||||
}, {
|
}, {
|
||||||
'url': 'http://tv.biobiochile.cl/notas/2015/10/22/ninos-transexuales-de-quien-es-la-decision.shtml',
|
'url': 'http://tv.biobiochile.cl/notas/2015/10/22/ninos-transexuales-de-quien-es-la-decision.shtml',
|
||||||
@ -60,7 +63,9 @@ class BioBioChileTVIE(InfoExtractor):
|
|||||||
|
|
||||||
webpage = self._download_webpage(url, video_id)
|
webpage = self._download_webpage(url, video_id)
|
||||||
|
|
||||||
rudo_url = RudoIE._extract_url(webpage)
|
rudo_url = self._search_regex(
|
||||||
|
r'<iframe[^>]+src=(?P<q1>[\'"])(?P<url>(?:https?:)?//rudo\.video/vod/[0-9a-zA-Z]+)(?P=q1)',
|
||||||
|
webpage, 'embed URL', None, group='url')
|
||||||
if not rudo_url:
|
if not rudo_url:
|
||||||
raise ExtractorError('No videos found')
|
raise ExtractorError('No videos found')
|
||||||
|
|
||||||
@ -68,7 +73,7 @@ class BioBioChileTVIE(InfoExtractor):
|
|||||||
|
|
||||||
thumbnail = self._og_search_thumbnail(webpage)
|
thumbnail = self._og_search_thumbnail(webpage)
|
||||||
uploader = self._html_search_regex(
|
uploader = self._html_search_regex(
|
||||||
r'<a[^>]+href=["\']https?://(?:busca|www)\.biobiochile\.cl/(?:lista/)?(?:author|autor)[^>]+>(.+?)</a>',
|
r'<a[^>]+href=["\'](?:https?://(?:busca|www)\.biobiochile\.cl)?/(?:lista/)?(?:author|autor)[^>]+>(.+?)</a>',
|
||||||
webpage, 'uploader', fatal=False)
|
webpage, 'uploader', fatal=False)
|
||||||
|
|
||||||
return {
|
return {
|
||||||
|
@ -71,7 +71,7 @@ class BleacherReportIE(InfoExtractor):
|
|||||||
video = article_data.get('video')
|
video = article_data.get('video')
|
||||||
if video:
|
if video:
|
||||||
video_type = video['type']
|
video_type = video['type']
|
||||||
if video_type == 'cms.bleacherreport.com':
|
if video_type in ('cms.bleacherreport.com', 'vid.bleacherreport.com'):
|
||||||
info['url'] = 'http://bleacherreport.com/video_embed?id=%s' % video['id']
|
info['url'] = 'http://bleacherreport.com/video_embed?id=%s' % video['id']
|
||||||
elif video_type == 'ooyala.com':
|
elif video_type == 'ooyala.com':
|
||||||
info['url'] = 'ooyala:%s' % video['id']
|
info['url'] = 'ooyala:%s' % video['id']
|
||||||
@ -87,9 +87,9 @@ class BleacherReportIE(InfoExtractor):
|
|||||||
|
|
||||||
|
|
||||||
class BleacherReportCMSIE(AMPIE):
|
class BleacherReportCMSIE(AMPIE):
|
||||||
_VALID_URL = r'https?://(?:www\.)?bleacherreport\.com/video_embed\?id=(?P<id>[0-9a-f-]{36})'
|
_VALID_URL = r'https?://(?:www\.)?bleacherreport\.com/video_embed\?id=(?P<id>[0-9a-f-]{36}|\d{5})'
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'http://bleacherreport.com/video_embed?id=8fd44c2f-3dc5-4821-9118-2c825a98c0e1',
|
'url': 'http://bleacherreport.com/video_embed?id=8fd44c2f-3dc5-4821-9118-2c825a98c0e1&library=video-cms',
|
||||||
'md5': '2e4b0a997f9228ffa31fada5c53d1ed1',
|
'md5': '2e4b0a997f9228ffa31fada5c53d1ed1',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '8fd44c2f-3dc5-4821-9118-2c825a98c0e1',
|
'id': '8fd44c2f-3dc5-4821-9118-2c825a98c0e1',
|
||||||
@ -101,6 +101,6 @@ class BleacherReportCMSIE(AMPIE):
|
|||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
video_id = self._match_id(url)
|
video_id = self._match_id(url)
|
||||||
info = self._extract_feed_info('http://cms.bleacherreport.com/media/items/%s/akamai.json' % video_id)
|
info = self._extract_feed_info('http://vid.bleacherreport.com/videos/%s.akamai' % video_id)
|
||||||
info['id'] = video_id
|
info['id'] = video_id
|
||||||
return info
|
return info
|
||||||
|
@ -220,7 +220,7 @@ class InfoExtractor(object):
|
|||||||
* "preference" (optional, int) - quality of the image
|
* "preference" (optional, int) - quality of the image
|
||||||
* "width" (optional, int)
|
* "width" (optional, int)
|
||||||
* "height" (optional, int)
|
* "height" (optional, int)
|
||||||
* "resolution" (optional, string "{width}x{height"},
|
* "resolution" (optional, string "{width}x{height}",
|
||||||
deprecated)
|
deprecated)
|
||||||
* "filesize" (optional, int)
|
* "filesize" (optional, int)
|
||||||
thumbnail: Full URL to a video thumbnail image.
|
thumbnail: Full URL to a video thumbnail image.
|
||||||
|
@ -3,6 +3,7 @@ from __future__ import unicode_literals
|
|||||||
|
|
||||||
from .common import InfoExtractor
|
from .common import InfoExtractor
|
||||||
from ..utils import unified_timestamp
|
from ..utils import unified_timestamp
|
||||||
|
from .youtube import YoutubeIE
|
||||||
|
|
||||||
|
|
||||||
class CtsNewsIE(InfoExtractor):
|
class CtsNewsIE(InfoExtractor):
|
||||||
@ -14,8 +15,8 @@ class CtsNewsIE(InfoExtractor):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '201501291578109',
|
'id': '201501291578109',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'title': '以色列.真主黨交火 3人死亡',
|
'title': '以色列.真主黨交火 3人死亡 - 華視新聞網',
|
||||||
'description': '以色列和黎巴嫩真主黨,爆發五年最嚴重衝突,雙方砲轟交火,兩名以軍死亡,還有一名西班牙籍的聯合國維和人...',
|
'description': '以色列和黎巴嫩真主黨,爆發五年最嚴重衝突,雙方砲轟交火,兩名以軍死亡,還有一名西班牙籍的聯合國維和人員也不幸罹難。大陸陝西、河南、安徽、江蘇和湖北五個省份出現大暴雪,嚴重影響陸空交通,不過九華山卻出現...',
|
||||||
'timestamp': 1422528540,
|
'timestamp': 1422528540,
|
||||||
'upload_date': '20150129',
|
'upload_date': '20150129',
|
||||||
}
|
}
|
||||||
@ -26,7 +27,7 @@ class CtsNewsIE(InfoExtractor):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '201309031304098',
|
'id': '201309031304098',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'title': '韓國31歲童顏男 貌如十多歲小孩',
|
'title': '韓國31歲童顏男 貌如十多歲小孩 - 華視新聞網',
|
||||||
'description': '越有年紀的人,越希望看起來年輕一點,而南韓卻有一位31歲的男子,看起來像是11、12歲的小孩,身...',
|
'description': '越有年紀的人,越希望看起來年輕一點,而南韓卻有一位31歲的男子,看起來像是11、12歲的小孩,身...',
|
||||||
'thumbnail': r're:^https?://.*\.jpg$',
|
'thumbnail': r're:^https?://.*\.jpg$',
|
||||||
'timestamp': 1378205880,
|
'timestamp': 1378205880,
|
||||||
@ -62,8 +63,7 @@ class CtsNewsIE(InfoExtractor):
|
|||||||
video_url = mp4_feed['source_url']
|
video_url = mp4_feed['source_url']
|
||||||
else:
|
else:
|
||||||
self.to_screen('Not CTSPlayer video, trying Youtube...')
|
self.to_screen('Not CTSPlayer video, trying Youtube...')
|
||||||
youtube_url = self._search_regex(
|
youtube_url = YoutubeIE._extract_url(page)
|
||||||
r'src="(//www\.youtube\.com/embed/[^"]+)"', page, 'youtube url')
|
|
||||||
|
|
||||||
return self.url_result(youtube_url, ie='Youtube')
|
return self.url_result(youtube_url, ie='Youtube')
|
||||||
|
|
||||||
|
@ -7,50 +7,51 @@ from .common import InfoExtractor
|
|||||||
|
|
||||||
|
|
||||||
class DBTVIE(InfoExtractor):
|
class DBTVIE(InfoExtractor):
|
||||||
_VALID_URL = r'https?://(?:www\.)?dbtv\.no/(?:[^/]+/)?(?P<id>[0-9]+)(?:#(?P<display_id>.+))?'
|
_VALID_URL = r'https?://(?:www\.)?dagbladet\.no/video/(?:(?:embed|(?P<display_id>[^/]+))/)?(?P<id>[0-9A-Za-z_-]{11}|[a-zA-Z0-9]{8})'
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'http://dbtv.no/3649835190001#Skulle_teste_ut_fornøyelsespark,_men_kollegaen_var_bare_opptatt_av_bikinikroppen',
|
'url': 'https://www.dagbladet.no/video/PynxJnNWChE/',
|
||||||
'md5': '2e24f67936517b143a234b4cadf792ec',
|
'md5': 'b8f850ba1860adbda668d367f9b77699',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '3649835190001',
|
'id': 'PynxJnNWChE',
|
||||||
'display_id': 'Skulle_teste_ut_fornøyelsespark,_men_kollegaen_var_bare_opptatt_av_bikinikroppen',
|
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'title': 'Skulle teste ut fornøyelsespark, men kollegaen var bare opptatt av bikinikroppen',
|
'title': 'Skulle teste ut fornøyelsespark, men kollegaen var bare opptatt av bikinikroppen',
|
||||||
'description': 'md5:1504a54606c4dde3e4e61fc97aa857e0',
|
'description': 'md5:49cc8370e7d66e8a2ef15c3b4631fd3f',
|
||||||
'thumbnail': r're:https?://.*\.jpg',
|
'thumbnail': r're:https?://.*\.jpg',
|
||||||
'timestamp': 1404039863,
|
'upload_date': '20160916',
|
||||||
'upload_date': '20140629',
|
'duration': 69,
|
||||||
'duration': 69.544,
|
'uploader_id': 'UCk5pvsyZJoYJBd7_oFPTlRQ',
|
||||||
'uploader_id': '1027729757001',
|
'uploader': 'Dagbladet',
|
||||||
},
|
},
|
||||||
'add_ie': ['BrightcoveNew']
|
'add_ie': ['Youtube']
|
||||||
}, {
|
}, {
|
||||||
'url': 'http://dbtv.no/3649835190001',
|
'url': 'https://www.dagbladet.no/video/embed/xlGmyIeN9Jo/?autoplay=false',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}, {
|
}, {
|
||||||
'url': 'http://www.dbtv.no/lazyplayer/4631135248001',
|
'url': 'https://www.dagbladet.no/video/truer-iran-bor-passe-dere/PalfB2Cw',
|
||||||
'only_matching': True,
|
|
||||||
}, {
|
|
||||||
'url': 'http://dbtv.no/vice/5000634109001',
|
|
||||||
'only_matching': True,
|
|
||||||
}, {
|
|
||||||
'url': 'http://dbtv.no/filmtrailer/3359293614001',
|
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}]
|
}]
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _extract_urls(webpage):
|
def _extract_urls(webpage):
|
||||||
return [url for _, url in re.findall(
|
return [url for _, url in re.findall(
|
||||||
r'<iframe[^>]+src=(["\'])((?:https?:)?//(?:www\.)?dbtv\.no/(?:lazy)?player/\d+.*?)\1',
|
r'<iframe[^>]+src=(["\'])((?:https?:)?//(?:www\.)?dagbladet\.no/video/embed/(?:[0-9A-Za-z_-]{11}|[a-zA-Z0-9]{8}).*?)\1',
|
||||||
webpage)]
|
webpage)]
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
video_id, display_id = re.match(self._VALID_URL, url).groups()
|
display_id, video_id = re.match(self._VALID_URL, url).groups()
|
||||||
|
info = {
|
||||||
return {
|
|
||||||
'_type': 'url_transparent',
|
'_type': 'url_transparent',
|
||||||
'url': 'http://players.brightcove.net/1027729757001/default_default/index.html?videoId=%s' % video_id,
|
|
||||||
'id': video_id,
|
'id': video_id,
|
||||||
'display_id': display_id,
|
'display_id': display_id,
|
||||||
'ie_key': 'BrightcoveNew',
|
|
||||||
}
|
}
|
||||||
|
if len(video_id) == 11:
|
||||||
|
info.update({
|
||||||
|
'url': video_id,
|
||||||
|
'ie_key': 'Youtube',
|
||||||
|
})
|
||||||
|
else:
|
||||||
|
info.update({
|
||||||
|
'url': 'jwplatform:' + video_id,
|
||||||
|
'ie_key': 'JWPlatform',
|
||||||
|
})
|
||||||
|
return info
|
||||||
|
@ -19,9 +19,9 @@ from ..compat import compat_HTTPError
|
|||||||
class DiscoveryIE(DiscoveryGoBaseIE):
|
class DiscoveryIE(DiscoveryGoBaseIE):
|
||||||
_VALID_URL = r'''(?x)https?://
|
_VALID_URL = r'''(?x)https?://
|
||||||
(?P<site>
|
(?P<site>
|
||||||
|
(?:(?:www|go)\.)?discovery|
|
||||||
(?:www\.)?
|
(?:www\.)?
|
||||||
(?:
|
(?:
|
||||||
discovery|
|
|
||||||
investigationdiscovery|
|
investigationdiscovery|
|
||||||
discoverylife|
|
discoverylife|
|
||||||
animalplanet|
|
animalplanet|
|
||||||
@ -56,6 +56,9 @@ class DiscoveryIE(DiscoveryGoBaseIE):
|
|||||||
}, {
|
}, {
|
||||||
'url': 'https://www.investigationdiscovery.com/tv-shows/final-vision/full-episodes/final-vision',
|
'url': 'https://www.investigationdiscovery.com/tv-shows/final-vision/full-episodes/final-vision',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://go.discovery.com/tv-shows/alaskan-bush-people/videos/follow-your-own-road',
|
||||||
|
'only_matching': True,
|
||||||
}]
|
}]
|
||||||
_GEO_COUNTRIES = ['US']
|
_GEO_COUNTRIES = ['US']
|
||||||
_GEO_BYPASS = False
|
_GEO_BYPASS = False
|
||||||
|
94
youtube_dl/extractor/dlive.py
Normal file
94
youtube_dl/extractor/dlive.py
Normal file
@ -0,0 +1,94 @@
|
|||||||
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
|
||||||
|
from .common import InfoExtractor
|
||||||
|
from ..utils import int_or_none
|
||||||
|
|
||||||
|
|
||||||
|
class DLiveVODIE(InfoExtractor):
|
||||||
|
IE_NAME = 'dlive:vod'
|
||||||
|
_VALID_URL = r'https?://(?:www\.)?dlive\.tv/p/(?P<uploader_id>.+?)\+(?P<id>[a-zA-Z0-9]+)'
|
||||||
|
_TEST = {
|
||||||
|
'url': 'https://dlive.tv/p/pdp+3mTzOl4WR',
|
||||||
|
'info_dict': {
|
||||||
|
'id': '3mTzOl4WR',
|
||||||
|
'ext': 'mp4',
|
||||||
|
'title': 'Minecraft with james charles epic',
|
||||||
|
'upload_date': '20190701',
|
||||||
|
'timestamp': 1562011015,
|
||||||
|
'uploader_id': 'pdp',
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
def _real_extract(self, url):
|
||||||
|
uploader_id, vod_id = re.match(self._VALID_URL, url).groups()
|
||||||
|
broadcast = self._download_json(
|
||||||
|
'https://graphigo.prd.dlive.tv/', vod_id,
|
||||||
|
data=json.dumps({'query': '''query {
|
||||||
|
pastBroadcast(permlink:"%s+%s") {
|
||||||
|
content
|
||||||
|
createdAt
|
||||||
|
length
|
||||||
|
playbackUrl
|
||||||
|
title
|
||||||
|
thumbnailUrl
|
||||||
|
viewCount
|
||||||
|
}
|
||||||
|
}''' % (uploader_id, vod_id)}).encode())['data']['pastBroadcast']
|
||||||
|
title = broadcast['title']
|
||||||
|
formats = self._extract_m3u8_formats(
|
||||||
|
broadcast['playbackUrl'], vod_id, 'mp4', 'm3u8_native')
|
||||||
|
self._sort_formats(formats)
|
||||||
|
return {
|
||||||
|
'id': vod_id,
|
||||||
|
'title': title,
|
||||||
|
'uploader_id': uploader_id,
|
||||||
|
'formats': formats,
|
||||||
|
'description': broadcast.get('content'),
|
||||||
|
'thumbnail': broadcast.get('thumbnailUrl'),
|
||||||
|
'timestamp': int_or_none(broadcast.get('createdAt'), 1000),
|
||||||
|
'view_count': int_or_none(broadcast.get('viewCount')),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class DLiveStreamIE(InfoExtractor):
|
||||||
|
IE_NAME = 'dlive:stream'
|
||||||
|
_VALID_URL = r'https?://(?:www\.)?dlive\.tv/(?!p/)(?P<id>[\w.-]+)'
|
||||||
|
|
||||||
|
def _real_extract(self, url):
|
||||||
|
display_name = self._match_id(url)
|
||||||
|
user = self._download_json(
|
||||||
|
'https://graphigo.prd.dlive.tv/', display_name,
|
||||||
|
data=json.dumps({'query': '''query {
|
||||||
|
userByDisplayName(displayname:"%s") {
|
||||||
|
livestream {
|
||||||
|
content
|
||||||
|
createdAt
|
||||||
|
title
|
||||||
|
thumbnailUrl
|
||||||
|
watchingCount
|
||||||
|
}
|
||||||
|
username
|
||||||
|
}
|
||||||
|
}''' % display_name}).encode())['data']['userByDisplayName']
|
||||||
|
livestream = user['livestream']
|
||||||
|
title = livestream['title']
|
||||||
|
username = user['username']
|
||||||
|
formats = self._extract_m3u8_formats(
|
||||||
|
'https://live.prd.dlive.tv/hls/live/%s.m3u8' % username,
|
||||||
|
display_name, 'mp4')
|
||||||
|
self._sort_formats(formats)
|
||||||
|
return {
|
||||||
|
'id': display_name,
|
||||||
|
'title': self._live_title(title),
|
||||||
|
'uploader': display_name,
|
||||||
|
'uploader_id': username,
|
||||||
|
'formats': formats,
|
||||||
|
'description': livestream.get('content'),
|
||||||
|
'thumbnail': livestream.get('thumbnailUrl'),
|
||||||
|
'is_live': True,
|
||||||
|
'timestamp': int_or_none(livestream.get('createdAt'), 1000),
|
||||||
|
'view_count': int_or_none(livestream.get('watchingCount')),
|
||||||
|
}
|
@ -2,6 +2,7 @@
|
|||||||
from __future__ import unicode_literals
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
import re
|
||||||
|
|
||||||
from .common import InfoExtractor
|
from .common import InfoExtractor
|
||||||
from ..compat import (
|
from ..compat import (
|
||||||
@ -18,7 +19,7 @@ from ..utils import (
|
|||||||
|
|
||||||
|
|
||||||
class EinthusanIE(InfoExtractor):
|
class EinthusanIE(InfoExtractor):
|
||||||
_VALID_URL = r'https?://einthusan\.tv/movie/watch/(?P<id>[^/?#&]+)'
|
_VALID_URL = r'https?://(?P<host>einthusan\.(?:tv|com))/movie/watch/(?P<id>[^/?#&]+)'
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'https://einthusan.tv/movie/watch/9097/',
|
'url': 'https://einthusan.tv/movie/watch/9097/',
|
||||||
'md5': 'ff0f7f2065031b8a2cf13a933731c035',
|
'md5': 'ff0f7f2065031b8a2cf13a933731c035',
|
||||||
@ -32,6 +33,9 @@ class EinthusanIE(InfoExtractor):
|
|||||||
}, {
|
}, {
|
||||||
'url': 'https://einthusan.tv/movie/watch/51MZ/?lang=hindi',
|
'url': 'https://einthusan.tv/movie/watch/51MZ/?lang=hindi',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://einthusan.com/movie/watch/9097/',
|
||||||
|
'only_matching': True,
|
||||||
}]
|
}]
|
||||||
|
|
||||||
# reversed from jsoncrypto.prototype.decrypt() in einthusan-PGMovieWatcher.js
|
# reversed from jsoncrypto.prototype.decrypt() in einthusan-PGMovieWatcher.js
|
||||||
@ -41,7 +45,9 @@ class EinthusanIE(InfoExtractor):
|
|||||||
)).decode('utf-8'), video_id)
|
)).decode('utf-8'), video_id)
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
video_id = self._match_id(url)
|
mobj = re.match(self._VALID_URL, url)
|
||||||
|
host = mobj.group('host')
|
||||||
|
video_id = mobj.group('id')
|
||||||
|
|
||||||
webpage = self._download_webpage(url, video_id)
|
webpage = self._download_webpage(url, video_id)
|
||||||
|
|
||||||
@ -53,7 +59,7 @@ class EinthusanIE(InfoExtractor):
|
|||||||
page_id = self._html_search_regex(
|
page_id = self._html_search_regex(
|
||||||
'<html[^>]+data-pageid="([^"]+)"', webpage, 'page ID')
|
'<html[^>]+data-pageid="([^"]+)"', webpage, 'page ID')
|
||||||
video_data = self._download_json(
|
video_data = self._download_json(
|
||||||
'https://einthusan.tv/ajax/movie/watch/%s/' % video_id, video_id,
|
'https://%s/ajax/movie/watch/%s/' % (host, video_id), video_id,
|
||||||
data=urlencode_postdata({
|
data=urlencode_postdata({
|
||||||
'xEvent': 'UIVideoPlayer.PingOutcome',
|
'xEvent': 'UIVideoPlayer.PingOutcome',
|
||||||
'xJson': json.dumps({
|
'xJson': json.dumps({
|
||||||
|
@ -216,17 +216,14 @@ class FiveThirtyEightIE(InfoExtractor):
|
|||||||
_TEST = {
|
_TEST = {
|
||||||
'url': 'http://fivethirtyeight.com/features/how-the-6-8-raiders-can-still-make-the-playoffs/',
|
'url': 'http://fivethirtyeight.com/features/how-the-6-8-raiders-can-still-make-the-playoffs/',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '21846851',
|
'id': '56032156',
|
||||||
'ext': 'mp4',
|
'ext': 'flv',
|
||||||
'title': 'FiveThirtyEight: The Raiders can still make the playoffs',
|
'title': 'FiveThirtyEight: The Raiders can still make the playoffs',
|
||||||
'description': 'Neil Paine breaks down the simplest scenario that will put the Raiders into the playoffs at 8-8.',
|
'description': 'Neil Paine breaks down the simplest scenario that will put the Raiders into the playoffs at 8-8.',
|
||||||
'timestamp': 1513960621,
|
|
||||||
'upload_date': '20171222',
|
|
||||||
},
|
},
|
||||||
'params': {
|
'params': {
|
||||||
'skip_download': True,
|
'skip_download': True,
|
||||||
},
|
},
|
||||||
'expected_warnings': ['Unable to download f4m manifest'],
|
|
||||||
}
|
}
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
@ -234,9 +231,8 @@ class FiveThirtyEightIE(InfoExtractor):
|
|||||||
|
|
||||||
webpage = self._download_webpage(url, video_id)
|
webpage = self._download_webpage(url, video_id)
|
||||||
|
|
||||||
video_id = self._search_regex(
|
embed_url = self._search_regex(
|
||||||
r'data-video-id=["\'](?P<id>\d+)',
|
r'<iframe[^>]+src=["\'](https?://fivethirtyeight\.abcnews\.go\.com/video/embed/\d+/\d+)',
|
||||||
webpage, 'video id', group='id')
|
webpage, 'embed url')
|
||||||
|
|
||||||
return self.url_result(
|
return self.url_result(embed_url, 'AbcNewsVideo')
|
||||||
'http://espn.go.com/video/clip?id=%s' % video_id, ESPNIE.ie_key())
|
|
||||||
|
@ -395,11 +395,7 @@ from .frontendmasters import (
|
|||||||
FrontendMastersCourseIE
|
FrontendMastersCourseIE
|
||||||
)
|
)
|
||||||
from .funimation import FunimationIE
|
from .funimation import FunimationIE
|
||||||
from .funk import (
|
from .funk import FunkIE
|
||||||
FunkMixIE,
|
|
||||||
FunkChannelIE,
|
|
||||||
)
|
|
||||||
from .funnyordie import FunnyOrDieIE
|
|
||||||
from .fusion import FusionIE
|
from .fusion import FusionIE
|
||||||
from .fxnetworks import FXNetworksIE
|
from .fxnetworks import FXNetworksIE
|
||||||
from .gaia import GaiaIE
|
from .gaia import GaiaIE
|
||||||
@ -583,6 +579,7 @@ from .linkedin import (
|
|||||||
)
|
)
|
||||||
from .linuxacademy import LinuxAcademyIE
|
from .linuxacademy import LinuxAcademyIE
|
||||||
from .litv import LiTVIE
|
from .litv import LiTVIE
|
||||||
|
from .livejournal import LiveJournalIE
|
||||||
from .liveleak import (
|
from .liveleak import (
|
||||||
LiveLeakIE,
|
LiveLeakIE,
|
||||||
LiveLeakEmbedIE,
|
LiveLeakEmbedIE,
|
||||||
@ -971,7 +968,6 @@ from .rts import RTSIE
|
|||||||
from .rtve import RTVEALaCartaIE, RTVELiveIE, RTVEInfantilIE, RTVELiveIE, RTVETelevisionIE
|
from .rtve import RTVEALaCartaIE, RTVELiveIE, RTVEInfantilIE, RTVELiveIE, RTVETelevisionIE
|
||||||
from .rtvnh import RTVNHIE
|
from .rtvnh import RTVNHIE
|
||||||
from .rtvs import RTVSIE
|
from .rtvs import RTVSIE
|
||||||
from .rudo import RudoIE
|
|
||||||
from .ruhd import RUHDIE
|
from .ruhd import RUHDIE
|
||||||
from .rutube import (
|
from .rutube import (
|
||||||
RutubeIE,
|
RutubeIE,
|
||||||
@ -1259,6 +1255,10 @@ from .udn import UDNEmbedIE
|
|||||||
from .ufctv import UFCTVIE
|
from .ufctv import UFCTVIE
|
||||||
from .uktvplay import UKTVPlayIE
|
from .uktvplay import UKTVPlayIE
|
||||||
from .digiteka import DigitekaIE
|
from .digiteka import DigitekaIE
|
||||||
|
from .dlive import (
|
||||||
|
DLiveVODIE,
|
||||||
|
DLiveStreamIE,
|
||||||
|
)
|
||||||
from .umg import UMGDeIE
|
from .umg import UMGDeIE
|
||||||
from .unistra import UnistraIE
|
from .unistra import UnistraIE
|
||||||
from .unity import UnityIE
|
from .unity import UnityIE
|
||||||
@ -1448,6 +1448,7 @@ from .yahoo import (
|
|||||||
YahooSearchIE,
|
YahooSearchIE,
|
||||||
YahooGyaOPlayerIE,
|
YahooGyaOPlayerIE,
|
||||||
YahooGyaOIE,
|
YahooGyaOIE,
|
||||||
|
YahooJapanNewsIE,
|
||||||
)
|
)
|
||||||
from .yandexdisk import YandexDiskIE
|
from .yandexdisk import YandexDiskIE
|
||||||
from .yandexmusic import (
|
from .yandexmusic import (
|
||||||
|
@ -428,7 +428,7 @@ class FacebookIE(InfoExtractor):
|
|||||||
timestamp = int_or_none(self._search_regex(
|
timestamp = int_or_none(self._search_regex(
|
||||||
r'<abbr[^>]+data-utime=["\'](\d+)', webpage,
|
r'<abbr[^>]+data-utime=["\'](\d+)', webpage,
|
||||||
'timestamp', default=None))
|
'timestamp', default=None))
|
||||||
thumbnail = self._og_search_thumbnail(webpage)
|
thumbnail = self._html_search_meta(['og:image', 'twitter:image'], webpage)
|
||||||
|
|
||||||
view_count = parse_count(self._search_regex(
|
view_count = parse_count(self._search_regex(
|
||||||
r'\bviewCount\s*:\s*["\']([\d,.]+)', webpage, 'view count',
|
r'\bviewCount\s*:\s*["\']([\d,.]+)', webpage, 'view count',
|
||||||
|
@ -9,7 +9,7 @@ from ..utils import int_or_none
|
|||||||
|
|
||||||
class FiveTVIE(InfoExtractor):
|
class FiveTVIE(InfoExtractor):
|
||||||
_VALID_URL = r'''(?x)
|
_VALID_URL = r'''(?x)
|
||||||
http://
|
https?://
|
||||||
(?:www\.)?5-tv\.ru/
|
(?:www\.)?5-tv\.ru/
|
||||||
(?:
|
(?:
|
||||||
(?:[^/]+/)+(?P<id>\d+)|
|
(?:[^/]+/)+(?P<id>\d+)|
|
||||||
@ -39,6 +39,7 @@ class FiveTVIE(InfoExtractor):
|
|||||||
'duration': 180,
|
'duration': 180,
|
||||||
},
|
},
|
||||||
}, {
|
}, {
|
||||||
|
# redirect to https://www.5-tv.ru/projects/1000095/izvestia-glavnoe/
|
||||||
'url': 'http://www.5-tv.ru/glavnoe/#itemDetails',
|
'url': 'http://www.5-tv.ru/glavnoe/#itemDetails',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'glavnoe',
|
'id': 'glavnoe',
|
||||||
@ -46,6 +47,7 @@ class FiveTVIE(InfoExtractor):
|
|||||||
'title': r're:^Итоги недели с \d+ по \d+ \w+ \d{4} года$',
|
'title': r're:^Итоги недели с \d+ по \d+ \w+ \d{4} года$',
|
||||||
'thumbnail': r're:^https?://.*\.jpg$',
|
'thumbnail': r're:^https?://.*\.jpg$',
|
||||||
},
|
},
|
||||||
|
'skip': 'redirect to «Известия. Главное» project page',
|
||||||
}, {
|
}, {
|
||||||
'url': 'http://www.5-tv.ru/glavnoe/broadcasts/508645/',
|
'url': 'http://www.5-tv.ru/glavnoe/broadcasts/508645/',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
@ -70,7 +72,7 @@ class FiveTVIE(InfoExtractor):
|
|||||||
webpage = self._download_webpage(url, video_id)
|
webpage = self._download_webpage(url, video_id)
|
||||||
|
|
||||||
video_url = self._search_regex(
|
video_url = self._search_regex(
|
||||||
[r'<div[^>]+?class="flowplayer[^>]+?data-href="([^"]+)"',
|
[r'<div[^>]+?class="(?:flow)?player[^>]+?data-href="([^"]+)"',
|
||||||
r'<a[^>]+?href="([^"]+)"[^>]+?class="videoplayer"'],
|
r'<a[^>]+?href="([^"]+)"[^>]+?class="videoplayer"'],
|
||||||
webpage, 'video url')
|
webpage, 'video url')
|
||||||
|
|
||||||
|
@ -1,89 +1,21 @@
|
|||||||
# coding: utf-8
|
# coding: utf-8
|
||||||
from __future__ import unicode_literals
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
import itertools
|
|
||||||
import re
|
import re
|
||||||
|
|
||||||
from .common import InfoExtractor
|
from .common import InfoExtractor
|
||||||
from .nexx import NexxIE
|
from .nexx import NexxIE
|
||||||
from ..compat import compat_str
|
|
||||||
from ..utils import (
|
from ..utils import (
|
||||||
int_or_none,
|
int_or_none,
|
||||||
try_get,
|
str_or_none,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
class FunkBaseIE(InfoExtractor):
|
class FunkIE(InfoExtractor):
|
||||||
_HEADERS = {
|
_VALID_URL = r'https?://(?:www\.)?funk\.net/(?:channel|playlist)/[^/]+/(?P<display_id>[0-9a-z-]+)-(?P<id>\d+)'
|
||||||
'Accept': '*/*',
|
|
||||||
'Accept-Language': 'en-US,en;q=0.9,ru;q=0.8',
|
|
||||||
'authorization': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbGllbnROYW1lIjoid2ViYXBwLXYzMSIsInNjb3BlIjoic3RhdGljLWNvbnRlbnQtYXBpLGN1cmF0aW9uLWFwaSxuZXh4LWNvbnRlbnQtYXBpLXYzMSx3ZWJhcHAtYXBpIn0.mbuG9wS9Yf5q6PqgR4fiaRFIagiHk9JhwoKES7ksVX4',
|
|
||||||
}
|
|
||||||
_AUTH = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjbGllbnROYW1lIjoid2ViYXBwLXYzMSIsInNjb3BlIjoic3RhdGljLWNvbnRlbnQtYXBpLGN1cmF0aW9uLWFwaSxuZXh4LWNvbnRlbnQtYXBpLXYzMSx3ZWJhcHAtYXBpIn0.mbuG9wS9Yf5q6PqgR4fiaRFIagiHk9JhwoKES7ksVX4'
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _make_headers(referer):
|
|
||||||
headers = FunkBaseIE._HEADERS.copy()
|
|
||||||
headers['Referer'] = referer
|
|
||||||
return headers
|
|
||||||
|
|
||||||
def _make_url_result(self, video):
|
|
||||||
return {
|
|
||||||
'_type': 'url_transparent',
|
|
||||||
'url': 'nexx:741:%s' % video['sourceId'],
|
|
||||||
'ie_key': NexxIE.ie_key(),
|
|
||||||
'id': video['sourceId'],
|
|
||||||
'title': video.get('title'),
|
|
||||||
'description': video.get('description'),
|
|
||||||
'duration': int_or_none(video.get('duration')),
|
|
||||||
'season_number': int_or_none(video.get('seasonNr')),
|
|
||||||
'episode_number': int_or_none(video.get('episodeNr')),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class FunkMixIE(FunkBaseIE):
|
|
||||||
_VALID_URL = r'https?://(?:www\.)?funk\.net/mix/(?P<id>[^/]+)/(?P<alias>[^/?#&]+)'
|
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'https://www.funk.net/mix/59d65d935f8b160001828b5b/die-realste-kifferdoku-aller-zeiten',
|
'url': 'https://www.funk.net/channel/ba-793/die-lustigsten-instrumente-aus-dem-internet-teil-2-1155821',
|
||||||
'md5': '8edf617c2f2b7c9847dfda313f199009',
|
'md5': '8dd9d9ab59b4aa4173b3197f2ea48e81',
|
||||||
'info_dict': {
|
|
||||||
'id': '123748',
|
|
||||||
'ext': 'mp4',
|
|
||||||
'title': '"Die realste Kifferdoku aller Zeiten"',
|
|
||||||
'description': 'md5:c97160f5bafa8d47ec8e2e461012aa9d',
|
|
||||||
'timestamp': 1490274721,
|
|
||||||
'upload_date': '20170323',
|
|
||||||
},
|
|
||||||
}]
|
|
||||||
|
|
||||||
def _real_extract(self, url):
|
|
||||||
mobj = re.match(self._VALID_URL, url)
|
|
||||||
mix_id = mobj.group('id')
|
|
||||||
alias = mobj.group('alias')
|
|
||||||
|
|
||||||
lists = self._download_json(
|
|
||||||
'https://www.funk.net/api/v3.1/curation/curatedLists/',
|
|
||||||
mix_id, headers=self._make_headers(url), query={
|
|
||||||
'size': 100,
|
|
||||||
})['_embedded']['curatedListList']
|
|
||||||
|
|
||||||
metas = next(
|
|
||||||
l for l in lists
|
|
||||||
if mix_id in (l.get('entityId'), l.get('alias')))['videoMetas']
|
|
||||||
video = next(
|
|
||||||
meta['videoDataDelegate']
|
|
||||||
for meta in metas
|
|
||||||
if try_get(
|
|
||||||
meta, lambda x: x['videoDataDelegate']['alias'],
|
|
||||||
compat_str) == alias)
|
|
||||||
|
|
||||||
return self._make_url_result(video)
|
|
||||||
|
|
||||||
|
|
||||||
class FunkChannelIE(FunkBaseIE):
|
|
||||||
_VALID_URL = r'https?://(?:www\.)?funk\.net/channel/(?P<id>[^/]+)/(?P<alias>[^/?#&]+)'
|
|
||||||
_TESTS = [{
|
|
||||||
'url': 'https://www.funk.net/channel/ba/die-lustigsten-instrumente-aus-dem-internet-teil-2',
|
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '1155821',
|
'id': '1155821',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
@ -92,83 +24,26 @@ class FunkChannelIE(FunkBaseIE):
|
|||||||
'timestamp': 1514507395,
|
'timestamp': 1514507395,
|
||||||
'upload_date': '20171229',
|
'upload_date': '20171229',
|
||||||
},
|
},
|
||||||
'params': {
|
|
||||||
'skip_download': True,
|
|
||||||
},
|
|
||||||
}, {
|
}, {
|
||||||
# only available via byIdList API
|
'url': 'https://www.funk.net/playlist/neuesteVideos/kameras-auf-dem-fusion-festival-1618699',
|
||||||
'url': 'https://www.funk.net/channel/informr/martin-sonneborn-erklaert-die-eu',
|
|
||||||
'info_dict': {
|
|
||||||
'id': '205067',
|
|
||||||
'ext': 'mp4',
|
|
||||||
'title': 'Martin Sonneborn erklärt die EU',
|
|
||||||
'description': 'md5:050f74626e4ed87edf4626d2024210c0',
|
|
||||||
'timestamp': 1494424042,
|
|
||||||
'upload_date': '20170510',
|
|
||||||
},
|
|
||||||
'params': {
|
|
||||||
'skip_download': True,
|
|
||||||
},
|
|
||||||
}, {
|
|
||||||
'url': 'https://www.funk.net/channel/59d5149841dca100012511e3/mein-erster-job-lovemilla-folge-1/lovemilla/',
|
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}]
|
}]
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
mobj = re.match(self._VALID_URL, url)
|
display_id, nexx_id = re.match(self._VALID_URL, url).groups()
|
||||||
channel_id = mobj.group('id')
|
video = self._download_json(
|
||||||
alias = mobj.group('alias')
|
'https://www.funk.net/api/v4.0/videos/' + nexx_id, nexx_id)
|
||||||
|
return {
|
||||||
headers = self._make_headers(url)
|
'_type': 'url_transparent',
|
||||||
|
'url': 'nexx:741:' + nexx_id,
|
||||||
video = None
|
'ie_key': NexxIE.ie_key(),
|
||||||
|
'id': nexx_id,
|
||||||
# Id-based channels are currently broken on their side: webplayer
|
'title': video.get('title'),
|
||||||
# tries to process them via byChannelAlias endpoint and fails
|
'description': video.get('description'),
|
||||||
# predictably.
|
'duration': int_or_none(video.get('duration')),
|
||||||
for page_num in itertools.count():
|
'channel_id': str_or_none(video.get('channelId')),
|
||||||
by_channel_alias = self._download_json(
|
'display_id': display_id,
|
||||||
'https://www.funk.net/api/v3.1/webapp/videos/byChannelAlias/%s'
|
'tags': video.get('tags'),
|
||||||
% channel_id,
|
'thumbnail': video.get('imageUrlLandscape'),
|
||||||
'Downloading byChannelAlias JSON page %d' % (page_num + 1),
|
}
|
||||||
headers=headers, query={
|
|
||||||
'filterFsk': 'false',
|
|
||||||
'sort': 'creationDate,desc',
|
|
||||||
'size': 100,
|
|
||||||
'page': page_num,
|
|
||||||
}, fatal=False)
|
|
||||||
if not by_channel_alias:
|
|
||||||
break
|
|
||||||
video_list = try_get(
|
|
||||||
by_channel_alias, lambda x: x['_embedded']['videoList'], list)
|
|
||||||
if not video_list:
|
|
||||||
break
|
|
||||||
try:
|
|
||||||
video = next(r for r in video_list if r.get('alias') == alias)
|
|
||||||
break
|
|
||||||
except StopIteration:
|
|
||||||
pass
|
|
||||||
if not try_get(
|
|
||||||
by_channel_alias, lambda x: x['_links']['next']):
|
|
||||||
break
|
|
||||||
|
|
||||||
if not video:
|
|
||||||
by_id_list = self._download_json(
|
|
||||||
'https://www.funk.net/api/v3.0/content/videos/byIdList',
|
|
||||||
channel_id, 'Downloading byIdList JSON', headers=headers,
|
|
||||||
query={
|
|
||||||
'ids': alias,
|
|
||||||
}, fatal=False)
|
|
||||||
if by_id_list:
|
|
||||||
video = try_get(by_id_list, lambda x: x['result'][0], dict)
|
|
||||||
|
|
||||||
if not video:
|
|
||||||
results = self._download_json(
|
|
||||||
'https://www.funk.net/api/v3.0/content/videos/filter',
|
|
||||||
channel_id, 'Downloading filter JSON', headers=headers, query={
|
|
||||||
'channelId': channel_id,
|
|
||||||
'size': 100,
|
|
||||||
})['result']
|
|
||||||
video = next(r for r in results if r.get('alias') == alias)
|
|
||||||
|
|
||||||
return self._make_url_result(video)
|
|
||||||
|
@ -1,162 +0,0 @@
|
|||||||
from __future__ import unicode_literals
|
|
||||||
|
|
||||||
import re
|
|
||||||
|
|
||||||
from .common import InfoExtractor
|
|
||||||
from ..utils import (
|
|
||||||
ExtractorError,
|
|
||||||
float_or_none,
|
|
||||||
int_or_none,
|
|
||||||
unified_timestamp,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class FunnyOrDieIE(InfoExtractor):
|
|
||||||
_VALID_URL = r'https?://(?:www\.)?funnyordie\.com/(?P<type>embed|articles|videos)/(?P<id>[0-9a-f]+)(?:$|[?#/])'
|
|
||||||
_TESTS = [{
|
|
||||||
'url': 'http://www.funnyordie.com/videos/0732f586d7/heart-shaped-box-literal-video-version',
|
|
||||||
'md5': 'bcd81e0c4f26189ee09be362ad6e6ba9',
|
|
||||||
'info_dict': {
|
|
||||||
'id': '0732f586d7',
|
|
||||||
'ext': 'mp4',
|
|
||||||
'title': 'Heart-Shaped Box: Literal Video Version',
|
|
||||||
'description': 'md5:ea09a01bc9a1c46d9ab696c01747c338',
|
|
||||||
'thumbnail': r're:^http:.*\.jpg$',
|
|
||||||
'uploader': 'DASjr',
|
|
||||||
'timestamp': 1317904928,
|
|
||||||
'upload_date': '20111006',
|
|
||||||
'duration': 318.3,
|
|
||||||
},
|
|
||||||
}, {
|
|
||||||
'url': 'http://www.funnyordie.com/embed/e402820827',
|
|
||||||
'info_dict': {
|
|
||||||
'id': 'e402820827',
|
|
||||||
'ext': 'mp4',
|
|
||||||
'title': 'Please Use This Song (Jon Lajoie)',
|
|
||||||
'description': 'Please use this to sell something. www.jonlajoie.com',
|
|
||||||
'thumbnail': r're:^http:.*\.jpg$',
|
|
||||||
'timestamp': 1398988800,
|
|
||||||
'upload_date': '20140502',
|
|
||||||
},
|
|
||||||
'params': {
|
|
||||||
'skip_download': True,
|
|
||||||
},
|
|
||||||
}, {
|
|
||||||
'url': 'http://www.funnyordie.com/articles/ebf5e34fc8/10-hours-of-walking-in-nyc-as-a-man',
|
|
||||||
'only_matching': True,
|
|
||||||
}]
|
|
||||||
|
|
||||||
def _real_extract(self, url):
|
|
||||||
mobj = re.match(self._VALID_URL, url)
|
|
||||||
|
|
||||||
video_id = mobj.group('id')
|
|
||||||
webpage = self._download_webpage(url, video_id)
|
|
||||||
|
|
||||||
links = re.findall(r'<source src="([^"]+/v)[^"]+\.([^"]+)" type=\'video', webpage)
|
|
||||||
if not links:
|
|
||||||
raise ExtractorError('No media links available for %s' % video_id)
|
|
||||||
|
|
||||||
links.sort(key=lambda link: 1 if link[1] == 'mp4' else 0)
|
|
||||||
|
|
||||||
m3u8_url = self._search_regex(
|
|
||||||
r'<source[^>]+src=(["\'])(?P<url>.+?/master\.m3u8[^"\']*)\1',
|
|
||||||
webpage, 'm3u8 url', group='url')
|
|
||||||
|
|
||||||
formats = []
|
|
||||||
|
|
||||||
m3u8_formats = self._extract_m3u8_formats(
|
|
||||||
m3u8_url, video_id, 'mp4', 'm3u8_native',
|
|
||||||
m3u8_id='hls', fatal=False)
|
|
||||||
source_formats = list(filter(
|
|
||||||
lambda f: f.get('vcodec') != 'none', m3u8_formats))
|
|
||||||
|
|
||||||
bitrates = [int(bitrate) for bitrate in re.findall(r'[,/]v(\d+)(?=[,/])', m3u8_url)]
|
|
||||||
bitrates.sort()
|
|
||||||
|
|
||||||
if source_formats:
|
|
||||||
self._sort_formats(source_formats)
|
|
||||||
|
|
||||||
for bitrate, f in zip(bitrates, source_formats or [{}] * len(bitrates)):
|
|
||||||
for path, ext in links:
|
|
||||||
ff = f.copy()
|
|
||||||
if ff:
|
|
||||||
if ext != 'mp4':
|
|
||||||
ff = dict(
|
|
||||||
[(k, v) for k, v in ff.items()
|
|
||||||
if k in ('height', 'width', 'format_id')])
|
|
||||||
ff.update({
|
|
||||||
'format_id': ff['format_id'].replace('hls', ext),
|
|
||||||
'ext': ext,
|
|
||||||
'protocol': 'http',
|
|
||||||
})
|
|
||||||
else:
|
|
||||||
ff.update({
|
|
||||||
'format_id': '%s-%d' % (ext, bitrate),
|
|
||||||
'vbr': bitrate,
|
|
||||||
})
|
|
||||||
ff['url'] = self._proto_relative_url(
|
|
||||||
'%s%d.%s' % (path, bitrate, ext))
|
|
||||||
formats.append(ff)
|
|
||||||
self._check_formats(formats, video_id)
|
|
||||||
|
|
||||||
formats.extend(m3u8_formats)
|
|
||||||
self._sort_formats(
|
|
||||||
formats, field_preference=('height', 'width', 'tbr', 'format_id'))
|
|
||||||
|
|
||||||
subtitles = {}
|
|
||||||
for src, src_lang in re.findall(r'<track kind="captions" src="([^"]+)" srclang="([^"]+)"', webpage):
|
|
||||||
subtitles[src_lang] = [{
|
|
||||||
'ext': src.split('/')[-1],
|
|
||||||
'url': 'http://www.funnyordie.com%s' % src,
|
|
||||||
}]
|
|
||||||
|
|
||||||
timestamp = unified_timestamp(self._html_search_meta(
|
|
||||||
'uploadDate', webpage, 'timestamp', default=None))
|
|
||||||
|
|
||||||
uploader = self._html_search_regex(
|
|
||||||
r'<h\d[^>]+\bclass=["\']channel-preview-name[^>]+>(.+?)</h',
|
|
||||||
webpage, 'uploader', default=None)
|
|
||||||
|
|
||||||
title, description, thumbnail, duration = [None] * 4
|
|
||||||
|
|
||||||
medium = self._parse_json(
|
|
||||||
self._search_regex(
|
|
||||||
r'jsonMedium\s*=\s*({.+?});', webpage, 'JSON medium',
|
|
||||||
default='{}'),
|
|
||||||
video_id, fatal=False)
|
|
||||||
if medium:
|
|
||||||
title = medium.get('title')
|
|
||||||
duration = float_or_none(medium.get('duration'))
|
|
||||||
if not timestamp:
|
|
||||||
timestamp = unified_timestamp(medium.get('publishDate'))
|
|
||||||
|
|
||||||
post = self._parse_json(
|
|
||||||
self._search_regex(
|
|
||||||
r'fb_post\s*=\s*(\{.*?\});', webpage, 'post details',
|
|
||||||
default='{}'),
|
|
||||||
video_id, fatal=False)
|
|
||||||
if post:
|
|
||||||
if not title:
|
|
||||||
title = post.get('name')
|
|
||||||
description = post.get('description')
|
|
||||||
thumbnail = post.get('picture')
|
|
||||||
|
|
||||||
if not title:
|
|
||||||
title = self._og_search_title(webpage)
|
|
||||||
if not description:
|
|
||||||
description = self._og_search_description(webpage)
|
|
||||||
if not duration:
|
|
||||||
duration = int_or_none(self._html_search_meta(
|
|
||||||
('video:duration', 'duration'), webpage, 'duration', default=False))
|
|
||||||
|
|
||||||
return {
|
|
||||||
'id': video_id,
|
|
||||||
'title': title,
|
|
||||||
'description': description,
|
|
||||||
'thumbnail': thumbnail,
|
|
||||||
'uploader': uploader,
|
|
||||||
'timestamp': timestamp,
|
|
||||||
'duration': duration,
|
|
||||||
'formats': formats,
|
|
||||||
'subtitles': subtitles,
|
|
||||||
}
|
|
@ -1,12 +1,19 @@
|
|||||||
# coding: utf-8
|
# coding: utf-8
|
||||||
from __future__ import unicode_literals
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
|
from .brightcove import BrightcoveNewIE
|
||||||
from .common import InfoExtractor
|
from .common import InfoExtractor
|
||||||
|
from ..utils import (
|
||||||
|
clean_html,
|
||||||
|
get_element_by_class,
|
||||||
|
get_element_by_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class GameInformerIE(InfoExtractor):
|
class GameInformerIE(InfoExtractor):
|
||||||
_VALID_URL = r'https?://(?:www\.)?gameinformer\.com/(?:[^/]+/)*(?P<id>.+)\.aspx'
|
_VALID_URL = r'https?://(?:www\.)?gameinformer\.com/(?:[^/]+/)*(?P<id>[^.?&#]+)'
|
||||||
_TEST = {
|
_TESTS = [{
|
||||||
|
# normal Brightcove embed code extracted with BrightcoveNewIE._extract_url
|
||||||
'url': 'http://www.gameinformer.com/b/features/archive/2015/09/26/replay-animal-crossing.aspx',
|
'url': 'http://www.gameinformer.com/b/features/archive/2015/09/26/replay-animal-crossing.aspx',
|
||||||
'md5': '292f26da1ab4beb4c9099f1304d2b071',
|
'md5': '292f26da1ab4beb4c9099f1304d2b071',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
@ -18,16 +25,25 @@ class GameInformerIE(InfoExtractor):
|
|||||||
'upload_date': '20150928',
|
'upload_date': '20150928',
|
||||||
'uploader_id': '694940074001',
|
'uploader_id': '694940074001',
|
||||||
},
|
},
|
||||||
}
|
}, {
|
||||||
|
# Brightcove id inside unique element with field--name-field-brightcove-video-id class
|
||||||
|
'url': 'https://www.gameinformer.com/video-feature/new-gameplay-today/2019/07/09/new-gameplay-today-streets-of-rogue',
|
||||||
|
'info_dict': {
|
||||||
|
'id': '6057111913001',
|
||||||
|
'ext': 'mp4',
|
||||||
|
'title': 'New Gameplay Today – Streets Of Rogue',
|
||||||
|
'timestamp': 1562699001,
|
||||||
|
'upload_date': '20190709',
|
||||||
|
'uploader_id': '694940074001',
|
||||||
|
|
||||||
|
},
|
||||||
|
}]
|
||||||
BRIGHTCOVE_URL_TEMPLATE = 'http://players.brightcove.net/694940074001/default_default/index.html?videoId=%s'
|
BRIGHTCOVE_URL_TEMPLATE = 'http://players.brightcove.net/694940074001/default_default/index.html?videoId=%s'
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
display_id = self._match_id(url)
|
display_id = self._match_id(url)
|
||||||
webpage = self._download_webpage(
|
webpage = self._download_webpage(
|
||||||
url, display_id, headers=self.geo_verification_headers())
|
url, display_id, headers=self.geo_verification_headers())
|
||||||
brightcove_id = self._search_regex(
|
brightcove_id = clean_html(get_element_by_class('field--name-field-brightcove-video-id', webpage) or get_element_by_id('video-source-content', webpage))
|
||||||
[r'<[^>]+\bid=["\']bc_(\d+)', r"getVideo\('[^']+video_id=(\d+)"],
|
brightcove_url = self.BRIGHTCOVE_URL_TEMPLATE % brightcove_id if brightcove_id else BrightcoveNewIE._extract_url(self, webpage)
|
||||||
webpage, 'brightcove id')
|
return self.url_result(brightcove_url, 'BrightcoveNew', brightcove_id)
|
||||||
return self.url_result(
|
|
||||||
self.BRIGHTCOVE_URL_TEMPLATE % brightcove_id, 'BrightcoveNew',
|
|
||||||
brightcove_id)
|
|
||||||
|
@ -2226,7 +2226,7 @@ class GenericIE(InfoExtractor):
|
|||||||
default_search = 'fixup_error'
|
default_search = 'fixup_error'
|
||||||
|
|
||||||
if default_search in ('auto', 'auto_warning', 'fixup_error'):
|
if default_search in ('auto', 'auto_warning', 'fixup_error'):
|
||||||
if '/' in url:
|
if re.match(r'^[^\s/]+\.[^\s/]+/', url):
|
||||||
self._downloader.report_warning('The url doesn\'t specify the protocol, trying with http')
|
self._downloader.report_warning('The url doesn\'t specify the protocol, trying with http')
|
||||||
return self.url_result('http://' + url)
|
return self.url_result('http://' + url)
|
||||||
elif default_search != 'fixup_error':
|
elif default_search != 'fixup_error':
|
||||||
|
@ -11,7 +11,7 @@ from ..utils import (
|
|||||||
|
|
||||||
|
|
||||||
class GfycatIE(InfoExtractor):
|
class GfycatIE(InfoExtractor):
|
||||||
_VALID_URL = r'https?://(?:www\.)?gfycat\.com/(?:ifr/|gifs/detail/)?(?P<id>[^-/?#]+)'
|
_VALID_URL = r'https?://(?:www\.)?gfycat\.com/(?:ru/|ifr/|gifs/detail/)?(?P<id>[^-/?#]+)'
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'http://gfycat.com/DeadlyDecisiveGermanpinscher',
|
'url': 'http://gfycat.com/DeadlyDecisiveGermanpinscher',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
@ -44,6 +44,9 @@ class GfycatIE(InfoExtractor):
|
|||||||
'categories': list,
|
'categories': list,
|
||||||
'age_limit': 0,
|
'age_limit': 0,
|
||||||
}
|
}
|
||||||
|
}, {
|
||||||
|
'url': 'https://gfycat.com/ru/RemarkableDrearyAmurstarfish',
|
||||||
|
'only_matching': True
|
||||||
}, {
|
}, {
|
||||||
'url': 'https://gfycat.com/gifs/detail/UnconsciousLankyIvorygull',
|
'url': 'https://gfycat.com/gifs/detail/UnconsciousLankyIvorygull',
|
||||||
'only_matching': True
|
'only_matching': True
|
||||||
|
@ -103,6 +103,11 @@ class KalturaIE(InfoExtractor):
|
|||||||
{
|
{
|
||||||
'url': 'https://www.kaltura.com:443/index.php/extwidget/preview/partner_id/1770401/uiconf_id/37307382/entry_id/0_58u8kme7/embed/iframe?&flashvars[streamerType]=auto',
|
'url': 'https://www.kaltura.com:443/index.php/extwidget/preview/partner_id/1770401/uiconf_id/37307382/entry_id/0_58u8kme7/embed/iframe?&flashvars[streamerType]=auto',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
# unavailable source format
|
||||||
|
'url': 'kaltura:513551:1_66x4rg7o',
|
||||||
|
'only_matching': True,
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
|
|
||||||
@ -306,12 +311,17 @@ class KalturaIE(InfoExtractor):
|
|||||||
f['fileExt'] = 'mp4'
|
f['fileExt'] = 'mp4'
|
||||||
video_url = sign_url(
|
video_url = sign_url(
|
||||||
'%s/flavorId/%s' % (data_url, f['id']))
|
'%s/flavorId/%s' % (data_url, f['id']))
|
||||||
|
format_id = '%(fileExt)s-%(bitrate)s' % f
|
||||||
|
# Source format may not be available (e.g. kaltura:513551:1_66x4rg7o)
|
||||||
|
if f.get('isOriginal') is True and not self._is_valid_url(
|
||||||
|
video_url, entry_id, format_id):
|
||||||
|
continue
|
||||||
# audio-only has no videoCodecId (e.g. kaltura:1926081:0_c03e1b5g
|
# audio-only has no videoCodecId (e.g. kaltura:1926081:0_c03e1b5g
|
||||||
# -f mp4-56)
|
# -f mp4-56)
|
||||||
vcodec = 'none' if 'videoCodecId' not in f and f.get(
|
vcodec = 'none' if 'videoCodecId' not in f and f.get(
|
||||||
'frameRate') == 0 else f.get('videoCodecId')
|
'frameRate') == 0 else f.get('videoCodecId')
|
||||||
formats.append({
|
formats.append({
|
||||||
'format_id': '%(fileExt)s-%(bitrate)s' % f,
|
'format_id': format_id,
|
||||||
'ext': f.get('fileExt'),
|
'ext': f.get('fileExt'),
|
||||||
'tbr': int_or_none(f['bitrate']),
|
'tbr': int_or_none(f['bitrate']),
|
||||||
'fps': int_or_none(f.get('frameRate')),
|
'fps': int_or_none(f.get('frameRate')),
|
||||||
|
@ -326,7 +326,7 @@ class LetvCloudIE(InfoExtractor):
|
|||||||
elif play_json.get('code'):
|
elif play_json.get('code'):
|
||||||
raise ExtractorError('Letv cloud returned error %d' % play_json['code'], expected=True)
|
raise ExtractorError('Letv cloud returned error %d' % play_json['code'], expected=True)
|
||||||
else:
|
else:
|
||||||
raise ExtractorError('Letv cloud returned an unknwon error')
|
raise ExtractorError('Letv cloud returned an unknown error')
|
||||||
|
|
||||||
def b64decode(s):
|
def b64decode(s):
|
||||||
return compat_b64decode(s).decode('utf-8')
|
return compat_b64decode(s).decode('utf-8')
|
||||||
|
42
youtube_dl/extractor/livejournal.py
Normal file
42
youtube_dl/extractor/livejournal.py
Normal file
@ -0,0 +1,42 @@
|
|||||||
|
# coding: utf-8
|
||||||
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
|
from .common import InfoExtractor
|
||||||
|
from ..compat import compat_str
|
||||||
|
from ..utils import int_or_none
|
||||||
|
|
||||||
|
|
||||||
|
class LiveJournalIE(InfoExtractor):
|
||||||
|
_VALID_URL = r'https?://(?:[^.]+\.)?livejournal\.com/video/album/\d+.+?\bid=(?P<id>\d+)'
|
||||||
|
_TEST = {
|
||||||
|
'url': 'https://andrei-bt.livejournal.com/video/album/407/?mode=view&id=51272',
|
||||||
|
'md5': 'adaf018388572ced8a6f301ace49d4b2',
|
||||||
|
'info_dict': {
|
||||||
|
'id': '1263729',
|
||||||
|
'ext': 'mp4',
|
||||||
|
'title': 'Истребители против БПЛА',
|
||||||
|
'upload_date': '20190624',
|
||||||
|
'timestamp': 1561406715,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
def _real_extract(self, url):
|
||||||
|
video_id = self._match_id(url)
|
||||||
|
webpage = self._download_webpage(url, video_id)
|
||||||
|
record = self._parse_json(self._search_regex(
|
||||||
|
r'Site\.page\s*=\s*({.+?});', webpage,
|
||||||
|
'page data'), video_id)['video']['record']
|
||||||
|
storage_id = compat_str(record['storageid'])
|
||||||
|
title = record.get('name')
|
||||||
|
if title:
|
||||||
|
# remove filename extension(.mp4, .mov, etc...)
|
||||||
|
title = title.rsplit('.', 1)[0]
|
||||||
|
return {
|
||||||
|
'_type': 'url_transparent',
|
||||||
|
'id': video_id,
|
||||||
|
'title': title,
|
||||||
|
'thumbnail': record.get('thumbnail'),
|
||||||
|
'timestamp': int_or_none(record.get('timecreate')),
|
||||||
|
'url': 'eagleplatform:vc.videos.livejournal.com:' + storage_id,
|
||||||
|
'ie_key': 'EaglePlatform',
|
||||||
|
}
|
@ -117,6 +117,10 @@ class LyndaIE(LyndaBaseIE):
|
|||||||
}, {
|
}, {
|
||||||
'url': 'https://www.lynda.com/de/Graphic-Design-tutorials/Willkommen-Grundlagen-guten-Gestaltung/393570/393572-4.html',
|
'url': 'https://www.lynda.com/de/Graphic-Design-tutorials/Willkommen-Grundlagen-guten-Gestaltung/393570/393572-4.html',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
# Status="NotFound", Message="Transcript not found"
|
||||||
|
'url': 'https://www.lynda.com/ASP-NET-tutorials/What-you-should-know/5034180/2811512-4.html',
|
||||||
|
'only_matching': True,
|
||||||
}]
|
}]
|
||||||
|
|
||||||
def _raise_unavailable(self, video_id):
|
def _raise_unavailable(self, video_id):
|
||||||
@ -247,12 +251,17 @@ class LyndaIE(LyndaBaseIE):
|
|||||||
|
|
||||||
def _get_subtitles(self, video_id):
|
def _get_subtitles(self, video_id):
|
||||||
url = 'https://www.lynda.com/ajax/player?videoId=%s&type=transcript' % video_id
|
url = 'https://www.lynda.com/ajax/player?videoId=%s&type=transcript' % video_id
|
||||||
subs = self._download_json(url, None, False)
|
subs = self._download_webpage(
|
||||||
|
url, video_id, 'Downloading subtitles JSON', fatal=False)
|
||||||
|
if not subs or 'Status="NotFound"' in subs:
|
||||||
|
return {}
|
||||||
|
subs = self._parse_json(subs, video_id, fatal=False)
|
||||||
|
if not subs:
|
||||||
|
return {}
|
||||||
fixed_subs = self._fix_subtitles(subs)
|
fixed_subs = self._fix_subtitles(subs)
|
||||||
if fixed_subs:
|
if fixed_subs:
|
||||||
return {'en': [{'ext': 'srt', 'data': fixed_subs}]}
|
return {'en': [{'ext': 'srt', 'data': fixed_subs}]}
|
||||||
else:
|
return {}
|
||||||
return {}
|
|
||||||
|
|
||||||
|
|
||||||
class LyndaCourseIE(LyndaBaseIE):
|
class LyndaCourseIE(LyndaBaseIE):
|
||||||
|
@ -79,6 +79,9 @@ class MGTVIE(InfoExtractor):
|
|||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'tbr': tbr,
|
'tbr': tbr,
|
||||||
'protocol': 'm3u8_native',
|
'protocol': 'm3u8_native',
|
||||||
|
'http_headers': {
|
||||||
|
'Referer': url,
|
||||||
|
},
|
||||||
})
|
})
|
||||||
self._sort_formats(formats)
|
self._sort_formats(formats)
|
||||||
|
|
||||||
|
@ -39,7 +39,12 @@ class Porn91IE(InfoExtractor):
|
|||||||
r'<div id="viewvideo-title">([^<]+)</div>', webpage, 'title')
|
r'<div id="viewvideo-title">([^<]+)</div>', webpage, 'title')
|
||||||
title = title.replace('\n', '')
|
title = title.replace('\n', '')
|
||||||
|
|
||||||
info_dict = self._parse_html5_media_entries(url, webpage, video_id)[0]
|
video_link_url = self._search_regex(
|
||||||
|
r'<textarea[^>]+id=["\']fm-video_link[^>]+>([^<]+)</textarea>',
|
||||||
|
webpage, 'video link')
|
||||||
|
videopage = self._download_webpage(video_link_url, video_id)
|
||||||
|
|
||||||
|
info_dict = self._parse_html5_media_entries(url, videopage, video_id)[0]
|
||||||
|
|
||||||
duration = parse_duration(self._search_regex(
|
duration = parse_duration(self._search_regex(
|
||||||
r'时长:\s*</span>\s*(\d+:\d+)', webpage, 'duration', fatal=False))
|
r'时长:\s*</span>\s*(\d+:\d+)', webpage, 'duration', fatal=False))
|
||||||
|
@ -4,11 +4,14 @@ from __future__ import unicode_literals
|
|||||||
import re
|
import re
|
||||||
|
|
||||||
from .common import InfoExtractor
|
from .common import InfoExtractor
|
||||||
|
from ..compat import (
|
||||||
|
compat_HTTPError,
|
||||||
|
compat_str,
|
||||||
|
)
|
||||||
from ..utils import (
|
from ..utils import (
|
||||||
ExtractorError,
|
ExtractorError,
|
||||||
int_or_none,
|
int_or_none,
|
||||||
strip_or_none,
|
str_or_none,
|
||||||
unescapeHTML,
|
|
||||||
urlencode_postdata,
|
urlencode_postdata,
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -21,15 +24,14 @@ class RoosterTeethIE(InfoExtractor):
|
|||||||
'url': 'http://roosterteeth.com/episode/million-dollars-but-season-2-million-dollars-but-the-game-announcement',
|
'url': 'http://roosterteeth.com/episode/million-dollars-but-season-2-million-dollars-but-the-game-announcement',
|
||||||
'md5': 'e2bd7764732d785ef797700a2489f212',
|
'md5': 'e2bd7764732d785ef797700a2489f212',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '26576',
|
'id': '9156',
|
||||||
'display_id': 'million-dollars-but-season-2-million-dollars-but-the-game-announcement',
|
'display_id': 'million-dollars-but-season-2-million-dollars-but-the-game-announcement',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'title': 'Million Dollars, But...: Million Dollars, But... The Game Announcement',
|
'title': 'Million Dollars, But... The Game Announcement',
|
||||||
'description': 'md5:0cc3b21986d54ed815f5faeccd9a9ca5',
|
'description': 'md5:168a54b40e228e79f4ddb141e89fe4f5',
|
||||||
'thumbnail': r're:^https?://.*\.png$',
|
'thumbnail': r're:^https?://.*\.png$',
|
||||||
'series': 'Million Dollars, But...',
|
'series': 'Million Dollars, But...',
|
||||||
'episode': 'Million Dollars, But... The Game Announcement',
|
'episode': 'Million Dollars, But... The Game Announcement',
|
||||||
'comment_count': int,
|
|
||||||
},
|
},
|
||||||
}, {
|
}, {
|
||||||
'url': 'http://achievementhunter.roosterteeth.com/episode/off-topic-the-achievement-hunter-podcast-2016-i-didn-t-think-it-would-pass-31',
|
'url': 'http://achievementhunter.roosterteeth.com/episode/off-topic-the-achievement-hunter-podcast-2016-i-didn-t-think-it-would-pass-31',
|
||||||
@ -89,60 +91,55 @@ class RoosterTeethIE(InfoExtractor):
|
|||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
display_id = self._match_id(url)
|
display_id = self._match_id(url)
|
||||||
|
api_episode_url = 'https://svod-be.roosterteeth.com/api/v1/episodes/%s' % display_id
|
||||||
|
|
||||||
webpage = self._download_webpage(url, display_id)
|
try:
|
||||||
|
m3u8_url = self._download_json(
|
||||||
episode = strip_or_none(unescapeHTML(self._search_regex(
|
api_episode_url + '/videos', display_id,
|
||||||
(r'videoTitle\s*=\s*(["\'])(?P<title>(?:(?!\1).)+)\1',
|
'Downloading video JSON metadata')['data'][0]['attributes']['url']
|
||||||
r'<title>(?P<title>[^<]+)</title>'), webpage, 'title',
|
except ExtractorError as e:
|
||||||
default=None, group='title')))
|
if isinstance(e.cause, compat_HTTPError) and e.cause.code == 403:
|
||||||
|
if self._parse_json(e.cause.read().decode(), display_id).get('access') is False:
|
||||||
title = strip_or_none(self._og_search_title(
|
self.raise_login_required(
|
||||||
webpage, default=None)) or episode
|
'%s is only available for FIRST members' % display_id)
|
||||||
|
raise
|
||||||
m3u8_url = self._search_regex(
|
|
||||||
r'file\s*:\s*(["\'])(?P<url>http.+?\.m3u8.*?)\1',
|
|
||||||
webpage, 'm3u8 url', default=None, group='url')
|
|
||||||
|
|
||||||
if not m3u8_url:
|
|
||||||
if re.search(r'<div[^>]+class=["\']non-sponsor', webpage):
|
|
||||||
self.raise_login_required(
|
|
||||||
'%s is only available for FIRST members' % display_id)
|
|
||||||
|
|
||||||
if re.search(r'<div[^>]+class=["\']golive-gate', webpage):
|
|
||||||
self.raise_login_required('%s is not available yet' % display_id)
|
|
||||||
|
|
||||||
raise ExtractorError('Unable to extract m3u8 URL')
|
|
||||||
|
|
||||||
formats = self._extract_m3u8_formats(
|
formats = self._extract_m3u8_formats(
|
||||||
m3u8_url, display_id, ext='mp4',
|
m3u8_url, display_id, 'mp4', 'm3u8_native', m3u8_id='hls')
|
||||||
entry_protocol='m3u8_native', m3u8_id='hls')
|
|
||||||
self._sort_formats(formats)
|
self._sort_formats(formats)
|
||||||
|
|
||||||
description = strip_or_none(self._og_search_description(webpage))
|
episode = self._download_json(
|
||||||
thumbnail = self._proto_relative_url(self._og_search_thumbnail(webpage))
|
api_episode_url, display_id,
|
||||||
|
'Downloading episode JSON metadata')['data'][0]
|
||||||
|
attributes = episode['attributes']
|
||||||
|
title = attributes.get('title') or attributes['display_title']
|
||||||
|
video_id = compat_str(episode['id'])
|
||||||
|
|
||||||
series = self._search_regex(
|
thumbnails = []
|
||||||
(r'<h2>More ([^<]+)</h2>', r'<a[^>]+>See All ([^<]+) Videos<'),
|
for image in episode.get('included', {}).get('images', []):
|
||||||
webpage, 'series', fatal=False)
|
if image.get('type') == 'episode_image':
|
||||||
|
img_attributes = image.get('attributes') or {}
|
||||||
comment_count = int_or_none(self._search_regex(
|
for k in ('thumb', 'small', 'medium', 'large'):
|
||||||
r'>Comments \((\d+)\)<', webpage,
|
img_url = img_attributes.get(k)
|
||||||
'comment count', fatal=False))
|
if img_url:
|
||||||
|
thumbnails.append({
|
||||||
video_id = self._search_regex(
|
'id': k,
|
||||||
(r'containerId\s*=\s*["\']episode-(\d+)\1',
|
'url': img_url,
|
||||||
r'<div[^<]+id=["\']episode-(\d+)'), webpage,
|
})
|
||||||
'video id', default=display_id)
|
|
||||||
|
|
||||||
return {
|
return {
|
||||||
'id': video_id,
|
'id': video_id,
|
||||||
'display_id': display_id,
|
'display_id': display_id,
|
||||||
'title': title,
|
'title': title,
|
||||||
'description': description,
|
'description': attributes.get('description') or attributes.get('caption'),
|
||||||
'thumbnail': thumbnail,
|
'thumbnails': thumbnails,
|
||||||
'series': series,
|
'series': attributes.get('show_title'),
|
||||||
'episode': episode,
|
'season_number': int_or_none(attributes.get('season_number')),
|
||||||
'comment_count': comment_count,
|
'season_id': attributes.get('season_id'),
|
||||||
|
'episode': title,
|
||||||
|
'episode_number': int_or_none(attributes.get('number')),
|
||||||
|
'episode_id': str_or_none(episode.get('uuid')),
|
||||||
'formats': formats,
|
'formats': formats,
|
||||||
|
'channel_id': attributes.get('channel_id'),
|
||||||
|
'duration': int_or_none(attributes.get('length')),
|
||||||
}
|
}
|
||||||
|
@ -32,7 +32,7 @@ class RtlNlIE(InfoExtractor):
|
|||||||
'duration': 1167.96,
|
'duration': 1167.96,
|
||||||
},
|
},
|
||||||
}, {
|
}, {
|
||||||
# best format avaialble a3t
|
# best format available a3t
|
||||||
'url': 'http://www.rtl.nl/system/videoplayer/derden/rtlnieuws/video_embed.html#uuid=84ae5571-ac25-4225-ae0c-ef8d9efb2aed/autoplay=false',
|
'url': 'http://www.rtl.nl/system/videoplayer/derden/rtlnieuws/video_embed.html#uuid=84ae5571-ac25-4225-ae0c-ef8d9efb2aed/autoplay=false',
|
||||||
'md5': 'dea7474214af1271d91ef332fb8be7ea',
|
'md5': 'dea7474214af1271d91ef332fb8be7ea',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
|
@ -1,53 +0,0 @@
|
|||||||
# coding: utf-8
|
|
||||||
from __future__ import unicode_literals
|
|
||||||
|
|
||||||
import re
|
|
||||||
|
|
||||||
from .common import InfoExtractor
|
|
||||||
from ..utils import (
|
|
||||||
js_to_json,
|
|
||||||
get_element_by_class,
|
|
||||||
unified_strdate,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class RudoIE(InfoExtractor):
|
|
||||||
_VALID_URL = r'https?://rudo\.video/vod/(?P<id>[0-9a-zA-Z]+)'
|
|
||||||
|
|
||||||
_TEST = {
|
|
||||||
'url': 'http://rudo.video/vod/oTzw0MGnyG',
|
|
||||||
'md5': '2a03a5b32dd90a04c83b6d391cf7b415',
|
|
||||||
'info_dict': {
|
|
||||||
'id': 'oTzw0MGnyG',
|
|
||||||
'ext': 'mp4',
|
|
||||||
'title': 'Comentario Tomás Mosciatti',
|
|
||||||
'upload_date': '20160617',
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _extract_url(cls, webpage):
|
|
||||||
mobj = re.search(
|
|
||||||
r'<iframe[^>]+src=(?P<q1>[\'"])(?P<url>(?:https?:)?//rudo\.video/vod/[0-9a-zA-Z]+)(?P=q1)',
|
|
||||||
webpage)
|
|
||||||
if mobj:
|
|
||||||
return mobj.group('url')
|
|
||||||
|
|
||||||
def _real_extract(self, url):
|
|
||||||
video_id = self._match_id(url)
|
|
||||||
|
|
||||||
webpage = self._download_webpage(url, video_id, encoding='iso-8859-1')
|
|
||||||
|
|
||||||
jwplayer_data = self._parse_json(self._search_regex(
|
|
||||||
r'(?s)playerInstance\.setup\(({.+?})\)', webpage, 'jwplayer data'), video_id,
|
|
||||||
transform_source=lambda s: js_to_json(re.sub(r'encodeURI\([^)]+\)', '""', s)))
|
|
||||||
|
|
||||||
info_dict = self._parse_jwplayer_data(
|
|
||||||
jwplayer_data, video_id, require_title=False, m3u8_id='hls', mpd_id='dash')
|
|
||||||
|
|
||||||
info_dict.update({
|
|
||||||
'title': self._og_search_title(webpage),
|
|
||||||
'upload_date': unified_strdate(get_element_by_class('date', webpage)),
|
|
||||||
})
|
|
||||||
|
|
||||||
return info_dict
|
|
@ -197,7 +197,7 @@ class SoundcloudIE(InfoExtractor):
|
|||||||
'skip_download': True,
|
'skip_download': True,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
# not avaialble via api.soundcloud.com/i1/tracks/id/streams
|
# not available via api.soundcloud.com/i1/tracks/id/streams
|
||||||
{
|
{
|
||||||
'url': 'https://soundcloud.com/giovannisarani/mezzo-valzer',
|
'url': 'https://soundcloud.com/giovannisarani/mezzo-valzer',
|
||||||
'md5': 'e22aecd2bc88e0e4e432d7dcc0a1abf7',
|
'md5': 'e22aecd2bc88e0e4e432d7dcc0a1abf7',
|
||||||
|
@ -5,6 +5,7 @@ import re
|
|||||||
from .common import InfoExtractor
|
from .common import InfoExtractor
|
||||||
from ..utils import (
|
from ..utils import (
|
||||||
ExtractorError,
|
ExtractorError,
|
||||||
|
merge_dicts,
|
||||||
orderedSet,
|
orderedSet,
|
||||||
parse_duration,
|
parse_duration,
|
||||||
parse_resolution,
|
parse_resolution,
|
||||||
@ -26,6 +27,8 @@ class SpankBangIE(InfoExtractor):
|
|||||||
'description': 'dillion harper masturbates on a bed',
|
'description': 'dillion harper masturbates on a bed',
|
||||||
'thumbnail': r're:^https?://.*\.jpg$',
|
'thumbnail': r're:^https?://.*\.jpg$',
|
||||||
'uploader': 'silly2587',
|
'uploader': 'silly2587',
|
||||||
|
'timestamp': 1422571989,
|
||||||
|
'upload_date': '20150129',
|
||||||
'age_limit': 18,
|
'age_limit': 18,
|
||||||
}
|
}
|
||||||
}, {
|
}, {
|
||||||
@ -106,31 +109,36 @@ class SpankBangIE(InfoExtractor):
|
|||||||
|
|
||||||
for format_id, format_url in stream.items():
|
for format_id, format_url in stream.items():
|
||||||
if format_id.startswith(STREAM_URL_PREFIX):
|
if format_id.startswith(STREAM_URL_PREFIX):
|
||||||
|
if format_url and isinstance(format_url, list):
|
||||||
|
format_url = format_url[0]
|
||||||
extract_format(
|
extract_format(
|
||||||
format_id[len(STREAM_URL_PREFIX):], format_url)
|
format_id[len(STREAM_URL_PREFIX):], format_url)
|
||||||
|
|
||||||
self._sort_formats(formats)
|
self._sort_formats(formats)
|
||||||
|
|
||||||
|
info = self._search_json_ld(webpage, video_id, default={})
|
||||||
|
|
||||||
title = self._html_search_regex(
|
title = self._html_search_regex(
|
||||||
r'(?s)<h1[^>]*>(.+?)</h1>', webpage, 'title')
|
r'(?s)<h1[^>]*>(.+?)</h1>', webpage, 'title', default=None)
|
||||||
description = self._search_regex(
|
description = self._search_regex(
|
||||||
r'<div[^>]+\bclass=["\']bottom[^>]+>\s*<p>[^<]*</p>\s*<p>([^<]+)',
|
r'<div[^>]+\bclass=["\']bottom[^>]+>\s*<p>[^<]*</p>\s*<p>([^<]+)',
|
||||||
webpage, 'description', fatal=False)
|
webpage, 'description', default=None)
|
||||||
thumbnail = self._og_search_thumbnail(webpage)
|
thumbnail = self._og_search_thumbnail(webpage, default=None)
|
||||||
uploader = self._search_regex(
|
uploader = self._html_search_regex(
|
||||||
r'class="user"[^>]*><img[^>]+>([^<]+)',
|
(r'(?s)<li[^>]+class=["\']profile[^>]+>(.+?)</a>',
|
||||||
|
r'class="user"[^>]*><img[^>]+>([^<]+)'),
|
||||||
webpage, 'uploader', default=None)
|
webpage, 'uploader', default=None)
|
||||||
duration = parse_duration(self._search_regex(
|
duration = parse_duration(self._search_regex(
|
||||||
r'<div[^>]+\bclass=["\']right_side[^>]+>\s*<span>([^<]+)',
|
r'<div[^>]+\bclass=["\']right_side[^>]+>\s*<span>([^<]+)',
|
||||||
webpage, 'duration', fatal=False))
|
webpage, 'duration', default=None))
|
||||||
view_count = str_to_int(self._search_regex(
|
view_count = str_to_int(self._search_regex(
|
||||||
r'([\d,.]+)\s+plays', webpage, 'view count', fatal=False))
|
r'([\d,.]+)\s+plays', webpage, 'view count', default=None))
|
||||||
|
|
||||||
age_limit = self._rta_search(webpage)
|
age_limit = self._rta_search(webpage)
|
||||||
|
|
||||||
return {
|
return merge_dicts({
|
||||||
'id': video_id,
|
'id': video_id,
|
||||||
'title': title,
|
'title': title or video_id,
|
||||||
'description': description,
|
'description': description,
|
||||||
'thumbnail': thumbnail,
|
'thumbnail': thumbnail,
|
||||||
'uploader': uploader,
|
'uploader': uploader,
|
||||||
@ -138,7 +146,8 @@ class SpankBangIE(InfoExtractor):
|
|||||||
'view_count': view_count,
|
'view_count': view_count,
|
||||||
'formats': formats,
|
'formats': formats,
|
||||||
'age_limit': age_limit,
|
'age_limit': age_limit,
|
||||||
}
|
}, info
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class SpankBangPlaylistIE(InfoExtractor):
|
class SpankBangPlaylistIE(InfoExtractor):
|
||||||
|
@ -22,7 +22,7 @@ class BellatorIE(MTVServicesInfoExtractor):
|
|||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
}]
|
}]
|
||||||
|
|
||||||
_FEED_URL = 'http://www.spike.com/feeds/mrss/'
|
_FEED_URL = 'http://www.bellator.com/feeds/mrss/'
|
||||||
_GEO_COUNTRIES = ['US']
|
_GEO_COUNTRIES = ['US']
|
||||||
|
|
||||||
|
|
||||||
|
@ -428,11 +428,22 @@ class TwitterIE(InfoExtractor):
|
|||||||
'params': {
|
'params': {
|
||||||
'skip_download': True, # requires ffmpeg
|
'skip_download': True, # requires ffmpeg
|
||||||
},
|
},
|
||||||
|
}, {
|
||||||
|
'url': 'https://twitter.com/foobar/status/1087791357756956680',
|
||||||
|
'info_dict': {
|
||||||
|
'id': '1087791357756956680',
|
||||||
|
'ext': 'mp4',
|
||||||
|
'title': 'Twitter - A new is coming. Some of you got an opt-in to try it now. Check out the emoji button, quick keyboard shortcuts, upgraded trends, advanced search, and more. Let us know your thoughts!',
|
||||||
|
'thumbnail': r're:^https?://.*\.jpg',
|
||||||
|
'description': 'md5:66d493500c013e3e2d434195746a7f78',
|
||||||
|
'uploader': 'Twitter',
|
||||||
|
'uploader_id': 'Twitter',
|
||||||
|
'duration': 61.567,
|
||||||
|
},
|
||||||
}]
|
}]
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
mobj = re.match(self._VALID_URL, url)
|
mobj = re.match(self._VALID_URL, url)
|
||||||
user_id = mobj.group('user_id')
|
|
||||||
twid = mobj.group('id')
|
twid = mobj.group('id')
|
||||||
|
|
||||||
webpage, urlh = self._download_webpage_handle(
|
webpage, urlh = self._download_webpage_handle(
|
||||||
@ -441,8 +452,13 @@ class TwitterIE(InfoExtractor):
|
|||||||
if 'twitter.com/account/suspended' in urlh.geturl():
|
if 'twitter.com/account/suspended' in urlh.geturl():
|
||||||
raise ExtractorError('Account suspended by Twitter.', expected=True)
|
raise ExtractorError('Account suspended by Twitter.', expected=True)
|
||||||
|
|
||||||
if user_id is None:
|
user_id = None
|
||||||
mobj = re.match(self._VALID_URL, urlh.geturl())
|
|
||||||
|
redirect_mobj = re.match(self._VALID_URL, urlh.geturl())
|
||||||
|
if redirect_mobj:
|
||||||
|
user_id = redirect_mobj.group('user_id')
|
||||||
|
|
||||||
|
if not user_id:
|
||||||
user_id = mobj.group('user_id')
|
user_id = mobj.group('user_id')
|
||||||
|
|
||||||
username = remove_end(self._og_search_title(webpage), ' on Twitter')
|
username = remove_end(self._og_search_title(webpage), ' on Twitter')
|
||||||
|
@ -4,7 +4,10 @@ from __future__ import unicode_literals
|
|||||||
from .common import InfoExtractor
|
from .common import InfoExtractor
|
||||||
from .once import OnceIE
|
from .once import OnceIE
|
||||||
from ..compat import compat_urllib_parse_unquote
|
from ..compat import compat_urllib_parse_unquote
|
||||||
from ..utils import ExtractorError
|
from ..utils import (
|
||||||
|
ExtractorError,
|
||||||
|
int_or_none,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class VoxMediaVolumeIE(OnceIE):
|
class VoxMediaVolumeIE(OnceIE):
|
||||||
@ -13,18 +16,43 @@ class VoxMediaVolumeIE(OnceIE):
|
|||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
video_id = self._match_id(url)
|
video_id = self._match_id(url)
|
||||||
webpage = self._download_webpage(url, video_id)
|
webpage = self._download_webpage(url, video_id)
|
||||||
video_data = self._parse_json(self._search_regex(
|
|
||||||
r'Volume\.createVideo\(({.+})\s*,\s*{.*}\s*,\s*\[.*\]\s*,\s*{.*}\);', webpage, 'video data'), video_id)
|
setup = self._parse_json(self._search_regex(
|
||||||
|
r'setup\s*=\s*({.+});', webpage, 'setup'), video_id)
|
||||||
|
video_data = setup.get('video') or {}
|
||||||
|
info = {
|
||||||
|
'id': video_id,
|
||||||
|
'title': video_data.get('title_short'),
|
||||||
|
'description': video_data.get('description_long') or video_data.get('description_short'),
|
||||||
|
'thumbnail': video_data.get('brightcove_thumbnail')
|
||||||
|
}
|
||||||
|
asset = setup.get('asset') or setup.get('params') or {}
|
||||||
|
|
||||||
|
formats = []
|
||||||
|
hls_url = asset.get('hls_url')
|
||||||
|
if hls_url:
|
||||||
|
formats.extend(self._extract_m3u8_formats(
|
||||||
|
hls_url, video_id, 'mp4', 'm3u8_native', m3u8_id='hls', fatal=False))
|
||||||
|
mp4_url = asset.get('mp4_url')
|
||||||
|
if mp4_url:
|
||||||
|
tbr = self._search_regex(r'-(\d+)k\.', mp4_url, 'bitrate', default=None)
|
||||||
|
format_id = 'http'
|
||||||
|
if tbr:
|
||||||
|
format_id += '-' + tbr
|
||||||
|
formats.append({
|
||||||
|
'format_id': format_id,
|
||||||
|
'url': mp4_url,
|
||||||
|
'tbr': int_or_none(tbr),
|
||||||
|
})
|
||||||
|
if formats:
|
||||||
|
self._sort_formats(formats)
|
||||||
|
info['formats'] = formats
|
||||||
|
return info
|
||||||
|
|
||||||
for provider_video_type in ('ooyala', 'youtube', 'brightcove'):
|
for provider_video_type in ('ooyala', 'youtube', 'brightcove'):
|
||||||
provider_video_id = video_data.get('%s_id' % provider_video_type)
|
provider_video_id = video_data.get('%s_id' % provider_video_type)
|
||||||
if not provider_video_id:
|
if not provider_video_id:
|
||||||
continue
|
continue
|
||||||
info = {
|
|
||||||
'id': video_id,
|
|
||||||
'title': video_data.get('title_short'),
|
|
||||||
'description': video_data.get('description_long') or video_data.get('description_short'),
|
|
||||||
'thumbnail': video_data.get('brightcove_thumbnail')
|
|
||||||
}
|
|
||||||
if provider_video_type == 'brightcove':
|
if provider_video_type == 'brightcove':
|
||||||
info['formats'] = self._extract_once_formats(provider_video_id)
|
info['formats'] = self._extract_once_formats(provider_video_id)
|
||||||
self._sort_formats(info['formats'])
|
self._sort_formats(info['formats'])
|
||||||
@ -39,46 +67,49 @@ class VoxMediaVolumeIE(OnceIE):
|
|||||||
|
|
||||||
|
|
||||||
class VoxMediaIE(InfoExtractor):
|
class VoxMediaIE(InfoExtractor):
|
||||||
_VALID_URL = r'https?://(?:www\.)?(?:(?:theverge|vox|sbnation|eater|polygon|curbed|racked)\.com|recode\.net)/(?:[^/]+/)*(?P<id>[^/?]+)'
|
_VALID_URL = r'https?://(?:www\.)?(?:(?:theverge|vox|sbnation|eater|polygon|curbed|racked|funnyordie)\.com|recode\.net)/(?:[^/]+/)*(?P<id>[^/?]+)'
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
|
# Volume embed, Youtube
|
||||||
'url': 'http://www.theverge.com/2014/6/27/5849272/material-world-how-google-discovered-what-software-is-made-of',
|
'url': 'http://www.theverge.com/2014/6/27/5849272/material-world-how-google-discovered-what-software-is-made-of',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '11eXZobjrG8DCSTgrNjVinU-YmmdYjhe',
|
'id': 'j4mLW6x17VM',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'title': 'Google\'s new material design direction',
|
'title': 'Material world: how Google discovered what software is made of',
|
||||||
'description': 'md5:2f44f74c4d14a1f800ea73e1c6832ad2',
|
'description': 'md5:dfc17e7715e3b542d66e33a109861382',
|
||||||
|
'upload_date': '20190710',
|
||||||
|
'uploader_id': 'TheVerge',
|
||||||
|
'uploader': 'The Verge',
|
||||||
},
|
},
|
||||||
'params': {
|
'add_ie': ['Youtube'],
|
||||||
# m3u8 download
|
|
||||||
'skip_download': True,
|
|
||||||
},
|
|
||||||
'add_ie': ['Ooyala'],
|
|
||||||
}, {
|
}, {
|
||||||
# data-ooyala-id
|
# Volume embed, Youtube
|
||||||
'url': 'http://www.theverge.com/2014/10/21/7025853/google-nexus-6-hands-on-photos-video-android-phablet',
|
'url': 'http://www.theverge.com/2014/10/21/7025853/google-nexus-6-hands-on-photos-video-android-phablet',
|
||||||
'md5': 'd744484ff127884cd2ba09e3fa604e4b',
|
'md5': '4c8f4a0937752b437c3ebc0ed24802b5',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'RkZXU4cTphOCPDMZg5oEounJyoFI0g-B',
|
'id': 'Gy8Md3Eky38',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'title': 'The Nexus 6: hands-on with Google\'s phablet',
|
'title': 'The Nexus 6: hands-on with Google\'s phablet',
|
||||||
'description': 'md5:87a51fe95ff8cea8b5bdb9ac7ae6a6af',
|
'description': 'md5:d9f0216e5fb932dd2033d6db37ac3f1d',
|
||||||
|
'uploader_id': 'TheVerge',
|
||||||
|
'upload_date': '20141021',
|
||||||
|
'uploader': 'The Verge',
|
||||||
},
|
},
|
||||||
'add_ie': ['Ooyala'],
|
'add_ie': ['Youtube'],
|
||||||
'skip': 'Video Not Found',
|
'skip': 'similar to the previous test',
|
||||||
}, {
|
}, {
|
||||||
# volume embed
|
# Volume embed, Youtube
|
||||||
'url': 'http://www.vox.com/2016/3/31/11336640/mississippi-lgbt-religious-freedom-bill',
|
'url': 'http://www.vox.com/2016/3/31/11336640/mississippi-lgbt-religious-freedom-bill',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'wydzk3dDpmRz7PQoXRsTIX6XTkPjYL0b',
|
'id': 'YCjDnX-Xzhg',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
'title': 'The new frontier of LGBTQ civil rights, explained',
|
'title': "Mississippi's laws are so bad that its anti-LGBTQ law isn't needed to allow discrimination",
|
||||||
'description': 'md5:0dc58e94a465cbe91d02950f770eb93f',
|
'description': 'md5:fc1317922057de31cd74bce91eb1c66c',
|
||||||
|
'uploader_id': 'voxdotcom',
|
||||||
|
'upload_date': '20150915',
|
||||||
|
'uploader': 'Vox',
|
||||||
},
|
},
|
||||||
'params': {
|
'add_ie': ['Youtube'],
|
||||||
# m3u8 download
|
'skip': 'similar to the previous test',
|
||||||
'skip_download': True,
|
|
||||||
},
|
|
||||||
'add_ie': ['Ooyala'],
|
|
||||||
}, {
|
}, {
|
||||||
# youtube embed
|
# youtube embed
|
||||||
'url': 'http://www.vox.com/2016/3/24/11291692/robot-dance',
|
'url': 'http://www.vox.com/2016/3/24/11291692/robot-dance',
|
||||||
@ -93,6 +124,7 @@ class VoxMediaIE(InfoExtractor):
|
|||||||
'uploader': 'Vox',
|
'uploader': 'Vox',
|
||||||
},
|
},
|
||||||
'add_ie': ['Youtube'],
|
'add_ie': ['Youtube'],
|
||||||
|
'skip': 'Page no longer contain videos',
|
||||||
}, {
|
}, {
|
||||||
# SBN.VideoLinkset.entryGroup multiple ooyala embeds
|
# SBN.VideoLinkset.entryGroup multiple ooyala embeds
|
||||||
'url': 'http://www.sbnation.com/college-football-recruiting/2015/2/3/7970291/national-signing-day-rationalizations-itll-be-ok-itll-be-ok',
|
'url': 'http://www.sbnation.com/college-football-recruiting/2015/2/3/7970291/national-signing-day-rationalizations-itll-be-ok-itll-be-ok',
|
||||||
@ -118,10 +150,11 @@ class VoxMediaIE(InfoExtractor):
|
|||||||
'description': 'md5:e02d56b026d51aa32c010676765a690d',
|
'description': 'md5:e02d56b026d51aa32c010676765a690d',
|
||||||
},
|
},
|
||||||
}],
|
}],
|
||||||
|
'skip': 'Page no longer contain videos',
|
||||||
}, {
|
}, {
|
||||||
# volume embed, Brightcove Once
|
# volume embed, Brightcove Once
|
||||||
'url': 'https://www.recode.net/2014/6/17/11628066/post-post-pc-ceo-the-full-code-conference-video-of-microsofts-satya',
|
'url': 'https://www.recode.net/2014/6/17/11628066/post-post-pc-ceo-the-full-code-conference-video-of-microsofts-satya',
|
||||||
'md5': '01571a896281f77dc06e084138987ea2',
|
'md5': '2dbc77b8b0bff1894c2fce16eded637d',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '1231c973d',
|
'id': '1231c973d',
|
||||||
'ext': 'mp4',
|
'ext': 'mp4',
|
||||||
|
@ -64,7 +64,15 @@ class VRVBaseIE(InfoExtractor):
|
|||||||
|
|
||||||
def _call_cms(self, path, video_id, note):
|
def _call_cms(self, path, video_id, note):
|
||||||
if not self._CMS_SIGNING:
|
if not self._CMS_SIGNING:
|
||||||
self._CMS_SIGNING = self._call_api('index', video_id, 'CMS Signing')['cms_signing']
|
index = self._call_api('index', video_id, 'CMS Signing')
|
||||||
|
self._CMS_SIGNING = index.get('cms_signing') or {}
|
||||||
|
if not self._CMS_SIGNING:
|
||||||
|
for signing_policy in index.get('signing_policies', []):
|
||||||
|
signing_path = signing_policy.get('path')
|
||||||
|
if signing_path and signing_path.startswith('/cms/'):
|
||||||
|
name, value = signing_policy.get('name'), signing_policy.get('value')
|
||||||
|
if name and value:
|
||||||
|
self._CMS_SIGNING[name] = value
|
||||||
return self._download_json(
|
return self._download_json(
|
||||||
self._API_DOMAIN + path, video_id, query=self._CMS_SIGNING,
|
self._API_DOMAIN + path, video_id, query=self._CMS_SIGNING,
|
||||||
note='Downloading %s JSON metadata' % note, headers=self.geo_verification_headers())
|
note='Downloading %s JSON metadata' % note, headers=self.geo_verification_headers())
|
||||||
|
@ -1,12 +1,14 @@
|
|||||||
# coding: utf-8
|
# coding: utf-8
|
||||||
from __future__ import unicode_literals
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
|
import hashlib
|
||||||
import itertools
|
import itertools
|
||||||
import json
|
import json
|
||||||
import re
|
import re
|
||||||
|
|
||||||
from .common import InfoExtractor, SearchInfoExtractor
|
from .common import InfoExtractor, SearchInfoExtractor
|
||||||
from ..compat import (
|
from ..compat import (
|
||||||
|
compat_str,
|
||||||
compat_urllib_parse,
|
compat_urllib_parse,
|
||||||
compat_urlparse,
|
compat_urlparse,
|
||||||
)
|
)
|
||||||
@ -18,7 +20,9 @@ from ..utils import (
|
|||||||
int_or_none,
|
int_or_none,
|
||||||
mimetype2ext,
|
mimetype2ext,
|
||||||
smuggle_url,
|
smuggle_url,
|
||||||
|
try_get,
|
||||||
unescapeHTML,
|
unescapeHTML,
|
||||||
|
url_or_none,
|
||||||
)
|
)
|
||||||
|
|
||||||
from .brightcove import (
|
from .brightcove import (
|
||||||
@ -556,3 +560,130 @@ class YahooGyaOIE(InfoExtractor):
|
|||||||
'https://gyao.yahoo.co.jp/player/%s/' % video_id.replace(':', '/'),
|
'https://gyao.yahoo.co.jp/player/%s/' % video_id.replace(':', '/'),
|
||||||
YahooGyaOPlayerIE.ie_key(), video_id))
|
YahooGyaOPlayerIE.ie_key(), video_id))
|
||||||
return self.playlist_result(entries, program_id)
|
return self.playlist_result(entries, program_id)
|
||||||
|
|
||||||
|
|
||||||
|
class YahooJapanNewsIE(InfoExtractor):
|
||||||
|
IE_NAME = 'yahoo:japannews'
|
||||||
|
IE_DESC = 'Yahoo! Japan News'
|
||||||
|
_VALID_URL = r'https?://(?P<host>(?:news|headlines)\.yahoo\.co\.jp)[^\d]*(?P<id>\d[\d-]*\d)?'
|
||||||
|
_GEO_COUNTRIES = ['JP']
|
||||||
|
_TESTS = [{
|
||||||
|
'url': 'https://headlines.yahoo.co.jp/videonews/ann?a=20190716-00000071-ann-int',
|
||||||
|
'info_dict': {
|
||||||
|
'id': '1736242',
|
||||||
|
'ext': 'mp4',
|
||||||
|
'title': 'ムン大統領が対日批判を強化“現金化”効果は?(テレビ朝日系(ANN)) - Yahoo!ニュース',
|
||||||
|
'description': '韓国の元徴用工らを巡る裁判の原告が弁護士が差し押さえた三菱重工業の資産を売却して - Yahoo!ニュース(テレビ朝日系(ANN))',
|
||||||
|
'thumbnail': r're:^https?://.*\.[a-zA-Z\d]{3,4}$',
|
||||||
|
},
|
||||||
|
'params': {
|
||||||
|
'skip_download': True,
|
||||||
|
},
|
||||||
|
}, {
|
||||||
|
# geo restricted
|
||||||
|
'url': 'https://headlines.yahoo.co.jp/hl?a=20190721-00000001-oxv-l04',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://headlines.yahoo.co.jp/videonews/',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://news.yahoo.co.jp',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://news.yahoo.co.jp/byline/hashimotojunji/20190628-00131977/',
|
||||||
|
'only_matching': True,
|
||||||
|
}, {
|
||||||
|
'url': 'https://news.yahoo.co.jp/feature/1356',
|
||||||
|
'only_matching': True
|
||||||
|
}]
|
||||||
|
|
||||||
|
def _extract_formats(self, json_data, content_id):
|
||||||
|
formats = []
|
||||||
|
|
||||||
|
video_data = try_get(
|
||||||
|
json_data,
|
||||||
|
lambda x: x['ResultSet']['Result'][0]['VideoUrlSet']['VideoUrl'],
|
||||||
|
list)
|
||||||
|
for vid in video_data or []:
|
||||||
|
delivery = vid.get('delivery')
|
||||||
|
url = url_or_none(vid.get('Url'))
|
||||||
|
if not delivery or not url:
|
||||||
|
continue
|
||||||
|
elif delivery == 'hls':
|
||||||
|
formats.extend(
|
||||||
|
self._extract_m3u8_formats(
|
||||||
|
url, content_id, 'mp4', 'm3u8_native',
|
||||||
|
m3u8_id='hls', fatal=False))
|
||||||
|
else:
|
||||||
|
formats.append({
|
||||||
|
'url': url,
|
||||||
|
'format_id': 'http-%s' % compat_str(vid.get('bitrate', '')),
|
||||||
|
'height': int_or_none(vid.get('height')),
|
||||||
|
'width': int_or_none(vid.get('width')),
|
||||||
|
'tbr': int_or_none(vid.get('bitrate')),
|
||||||
|
})
|
||||||
|
self._remove_duplicate_formats(formats)
|
||||||
|
self._sort_formats(formats)
|
||||||
|
|
||||||
|
return formats
|
||||||
|
|
||||||
|
def _real_extract(self, url):
|
||||||
|
mobj = re.match(self._VALID_URL, url)
|
||||||
|
host = mobj.group('host')
|
||||||
|
display_id = mobj.group('id') or host
|
||||||
|
|
||||||
|
webpage = self._download_webpage(url, display_id)
|
||||||
|
|
||||||
|
title = self._html_search_meta(
|
||||||
|
['og:title', 'twitter:title'], webpage, 'title', default=None
|
||||||
|
) or self._html_search_regex('<title>([^<]+)</title>', webpage, 'title')
|
||||||
|
|
||||||
|
if display_id == host:
|
||||||
|
# Headline page (w/ multiple BC playlists) ('news.yahoo.co.jp', 'headlines.yahoo.co.jp/videonews/', ...)
|
||||||
|
stream_plists = re.findall(r'plist=(\d+)', webpage) or re.findall(r'plist["\']:\s*["\']([^"\']+)', webpage)
|
||||||
|
entries = [
|
||||||
|
self.url_result(
|
||||||
|
smuggle_url(
|
||||||
|
'http://players.brightcove.net/5690807595001/HyZNerRl7_default/index.html?playlistId=%s' % plist_id,
|
||||||
|
{'geo_countries': ['JP']}),
|
||||||
|
ie='BrightcoveNew', video_id=plist_id)
|
||||||
|
for plist_id in stream_plists]
|
||||||
|
return self.playlist_result(entries, playlist_title=title)
|
||||||
|
|
||||||
|
# Article page
|
||||||
|
description = self._html_search_meta(
|
||||||
|
['og:description', 'description', 'twitter:description'],
|
||||||
|
webpage, 'description', default=None)
|
||||||
|
thumbnail = self._og_search_thumbnail(
|
||||||
|
webpage, default=None) or self._html_search_meta(
|
||||||
|
'twitter:image', webpage, 'thumbnail', default=None)
|
||||||
|
space_id = self._search_regex([
|
||||||
|
r'<script[^>]+class=["\']yvpub-player["\'][^>]+spaceid=([^&"\']+)',
|
||||||
|
r'YAHOO\.JP\.srch\.\w+link\.onLoad[^;]+spaceID["\' ]*:["\' ]+([^"\']+)',
|
||||||
|
r'<!--\s+SpaceID=(\d+)'
|
||||||
|
], webpage, 'spaceid')
|
||||||
|
|
||||||
|
content_id = self._search_regex(
|
||||||
|
r'<script[^>]+class=["\']yvpub-player["\'][^>]+contentid=(?P<contentid>[^&"\']+)',
|
||||||
|
webpage, 'contentid', group='contentid')
|
||||||
|
|
||||||
|
json_data = self._download_json(
|
||||||
|
'https://feapi-yvpub.yahooapis.jp/v1/content/%s' % content_id,
|
||||||
|
content_id,
|
||||||
|
query={
|
||||||
|
'appid': 'dj0zaiZpPVZMTVFJR0FwZWpiMyZzPWNvbnN1bWVyc2VjcmV0Jng9YjU-',
|
||||||
|
'output': 'json',
|
||||||
|
'space_id': space_id,
|
||||||
|
'domain': host,
|
||||||
|
'ak': hashlib.md5('_'.join((space_id, host)).encode()).hexdigest(),
|
||||||
|
'device_type': '1100',
|
||||||
|
})
|
||||||
|
formats = self._extract_formats(json_data, content_id)
|
||||||
|
|
||||||
|
return {
|
||||||
|
'id': content_id,
|
||||||
|
'title': title,
|
||||||
|
'description': description,
|
||||||
|
'thumbnail': thumbnail,
|
||||||
|
'formats': formats,
|
||||||
|
}
|
||||||
|
@ -10,6 +10,7 @@ from ..utils import (
|
|||||||
ExtractorError,
|
ExtractorError,
|
||||||
int_or_none,
|
int_or_none,
|
||||||
float_or_none,
|
float_or_none,
|
||||||
|
try_get,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@ -51,23 +52,43 @@ class YandexMusicTrackIE(YandexMusicBaseIE):
|
|||||||
IE_DESC = 'Яндекс.Музыка - Трек'
|
IE_DESC = 'Яндекс.Музыка - Трек'
|
||||||
_VALID_URL = r'https?://music\.yandex\.(?:ru|kz|ua|by)/album/(?P<album_id>\d+)/track/(?P<id>\d+)'
|
_VALID_URL = r'https?://music\.yandex\.(?:ru|kz|ua|by)/album/(?P<album_id>\d+)/track/(?P<id>\d+)'
|
||||||
|
|
||||||
_TEST = {
|
_TESTS = [{
|
||||||
'url': 'http://music.yandex.ru/album/540508/track/4878838',
|
'url': 'http://music.yandex.ru/album/540508/track/4878838',
|
||||||
'md5': 'f496818aa2f60b6c0062980d2e00dc20',
|
'md5': 'f496818aa2f60b6c0062980d2e00dc20',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '4878838',
|
'id': '4878838',
|
||||||
'ext': 'mp3',
|
'ext': 'mp3',
|
||||||
'title': 'Carlo Ambrosio, Carlo Ambrosio & Fabio Di Bari - Gypsy Eyes 1',
|
'title': 'Carlo Ambrosio & Fabio Di Bari - Gypsy Eyes 1',
|
||||||
'filesize': 4628061,
|
'filesize': 4628061,
|
||||||
'duration': 193.04,
|
'duration': 193.04,
|
||||||
'track': 'Gypsy Eyes 1',
|
'track': 'Gypsy Eyes 1',
|
||||||
'album': 'Gypsy Soul',
|
'album': 'Gypsy Soul',
|
||||||
'album_artist': 'Carlo Ambrosio',
|
'album_artist': 'Carlo Ambrosio',
|
||||||
'artist': 'Carlo Ambrosio, Carlo Ambrosio & Fabio Di Bari',
|
'artist': 'Carlo Ambrosio & Fabio Di Bari',
|
||||||
'release_year': 2009,
|
'release_year': 2009,
|
||||||
},
|
},
|
||||||
'skip': 'Travis CI servers blocked by YandexMusic',
|
'skip': 'Travis CI servers blocked by YandexMusic',
|
||||||
}
|
}, {
|
||||||
|
# multiple disks
|
||||||
|
'url': 'http://music.yandex.ru/album/3840501/track/705105',
|
||||||
|
'md5': 'ebe7b4e2ac7ac03fe11c19727ca6153e',
|
||||||
|
'info_dict': {
|
||||||
|
'id': '705105',
|
||||||
|
'ext': 'mp3',
|
||||||
|
'title': 'Hooverphonic - Sometimes',
|
||||||
|
'filesize': 5743386,
|
||||||
|
'duration': 239.27,
|
||||||
|
'track': 'Sometimes',
|
||||||
|
'album': 'The Best of Hooverphonic',
|
||||||
|
'album_artist': 'Hooverphonic',
|
||||||
|
'artist': 'Hooverphonic',
|
||||||
|
'release_year': 2016,
|
||||||
|
'genre': 'pop',
|
||||||
|
'disc_number': 2,
|
||||||
|
'track_number': 9,
|
||||||
|
},
|
||||||
|
'skip': 'Travis CI servers blocked by YandexMusic',
|
||||||
|
}]
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
mobj = re.match(self._VALID_URL, url)
|
mobj = re.match(self._VALID_URL, url)
|
||||||
@ -110,9 +131,21 @@ class YandexMusicTrackIE(YandexMusicBaseIE):
|
|||||||
'abr': int_or_none(download_data.get('bitrate')),
|
'abr': int_or_none(download_data.get('bitrate')),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def extract_artist_name(artist):
|
||||||
|
decomposed = artist.get('decomposed')
|
||||||
|
if not isinstance(decomposed, list):
|
||||||
|
return artist['name']
|
||||||
|
parts = [artist['name']]
|
||||||
|
for element in decomposed:
|
||||||
|
if isinstance(element, dict) and element.get('name'):
|
||||||
|
parts.append(element['name'])
|
||||||
|
elif isinstance(element, compat_str):
|
||||||
|
parts.append(element)
|
||||||
|
return ''.join(parts)
|
||||||
|
|
||||||
def extract_artist(artist_list):
|
def extract_artist(artist_list):
|
||||||
if artist_list and isinstance(artist_list, list):
|
if artist_list and isinstance(artist_list, list):
|
||||||
artists_names = [a['name'] for a in artist_list if a.get('name')]
|
artists_names = [extract_artist_name(a) for a in artist_list if a.get('name')]
|
||||||
if artists_names:
|
if artists_names:
|
||||||
return ', '.join(artists_names)
|
return ', '.join(artists_names)
|
||||||
|
|
||||||
@ -121,10 +154,17 @@ class YandexMusicTrackIE(YandexMusicBaseIE):
|
|||||||
album = albums[0]
|
album = albums[0]
|
||||||
if isinstance(album, dict):
|
if isinstance(album, dict):
|
||||||
year = album.get('year')
|
year = album.get('year')
|
||||||
|
disc_number = int_or_none(try_get(
|
||||||
|
album, lambda x: x['trackPosition']['volume']))
|
||||||
|
track_number = int_or_none(try_get(
|
||||||
|
album, lambda x: x['trackPosition']['index']))
|
||||||
track_info.update({
|
track_info.update({
|
||||||
'album': album.get('title'),
|
'album': album.get('title'),
|
||||||
'album_artist': extract_artist(album.get('artists')),
|
'album_artist': extract_artist(album.get('artists')),
|
||||||
'release_year': int_or_none(year),
|
'release_year': int_or_none(year),
|
||||||
|
'genre': album.get('genre'),
|
||||||
|
'disc_number': disc_number,
|
||||||
|
'track_number': track_number,
|
||||||
})
|
})
|
||||||
|
|
||||||
track_artist = extract_artist(track.get('artists'))
|
track_artist = extract_artist(track.get('artists'))
|
||||||
@ -152,7 +192,7 @@ class YandexMusicAlbumIE(YandexMusicPlaylistBaseIE):
|
|||||||
IE_DESC = 'Яндекс.Музыка - Альбом'
|
IE_DESC = 'Яндекс.Музыка - Альбом'
|
||||||
_VALID_URL = r'https?://music\.yandex\.(?:ru|kz|ua|by)/album/(?P<id>\d+)/?(\?|$)'
|
_VALID_URL = r'https?://music\.yandex\.(?:ru|kz|ua|by)/album/(?P<id>\d+)/?(\?|$)'
|
||||||
|
|
||||||
_TEST = {
|
_TESTS = [{
|
||||||
'url': 'http://music.yandex.ru/album/540508',
|
'url': 'http://music.yandex.ru/album/540508',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': '540508',
|
'id': '540508',
|
||||||
@ -160,7 +200,15 @@ class YandexMusicAlbumIE(YandexMusicPlaylistBaseIE):
|
|||||||
},
|
},
|
||||||
'playlist_count': 50,
|
'playlist_count': 50,
|
||||||
'skip': 'Travis CI servers blocked by YandexMusic',
|
'skip': 'Travis CI servers blocked by YandexMusic',
|
||||||
}
|
}, {
|
||||||
|
'url': 'https://music.yandex.ru/album/3840501',
|
||||||
|
'info_dict': {
|
||||||
|
'id': '3840501',
|
||||||
|
'title': 'Hooverphonic - The Best of Hooverphonic (2016)',
|
||||||
|
},
|
||||||
|
'playlist_count': 33,
|
||||||
|
'skip': 'Travis CI servers blocked by YandexMusic',
|
||||||
|
}]
|
||||||
|
|
||||||
def _real_extract(self, url):
|
def _real_extract(self, url):
|
||||||
album_id = self._match_id(url)
|
album_id = self._match_id(url)
|
||||||
@ -169,7 +217,7 @@ class YandexMusicAlbumIE(YandexMusicPlaylistBaseIE):
|
|||||||
'http://music.yandex.ru/handlers/album.jsx?album=%s' % album_id,
|
'http://music.yandex.ru/handlers/album.jsx?album=%s' % album_id,
|
||||||
album_id, 'Downloading album JSON')
|
album_id, 'Downloading album JSON')
|
||||||
|
|
||||||
entries = self._build_playlist(album['volumes'][0])
|
entries = self._build_playlist([track for volume in album['volumes'] for track in volume])
|
||||||
|
|
||||||
title = '%s - %s' % (album['artists'][0]['name'], album['title'])
|
title = '%s - %s' % (album['artists'][0]['name'], album['title'])
|
||||||
year = album.get('year')
|
year = album.get('year')
|
||||||
|
@ -27,6 +27,7 @@ from ..compat import (
|
|||||||
compat_str,
|
compat_str,
|
||||||
)
|
)
|
||||||
from ..utils import (
|
from ..utils import (
|
||||||
|
bool_or_none,
|
||||||
clean_html,
|
clean_html,
|
||||||
dict_get,
|
dict_get,
|
||||||
error_to_compat_str,
|
error_to_compat_str,
|
||||||
@ -116,6 +117,8 @@ class YoutubeBaseInfoExtractor(InfoExtractor):
|
|||||||
'f.req': json.dumps(f_req),
|
'f.req': json.dumps(f_req),
|
||||||
'flowName': 'GlifWebSignIn',
|
'flowName': 'GlifWebSignIn',
|
||||||
'flowEntry': 'ServiceLogin',
|
'flowEntry': 'ServiceLogin',
|
||||||
|
# TODO: reverse actual botguard identifier generation algo
|
||||||
|
'bgRequest': '["identifier",""]',
|
||||||
})
|
})
|
||||||
return self._download_json(
|
return self._download_json(
|
||||||
url, None, note=note, errnote=errnote,
|
url, None, note=note, errnote=errnote,
|
||||||
@ -368,10 +371,15 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
(?:www\.)?hooktube\.com/|
|
(?:www\.)?hooktube\.com/|
|
||||||
(?:www\.)?yourepeat\.com/|
|
(?:www\.)?yourepeat\.com/|
|
||||||
tube\.majestyc\.net/|
|
tube\.majestyc\.net/|
|
||||||
|
# Invidious instances taken from https://github.com/omarroth/invidious/wiki/Invidious-Instances
|
||||||
(?:(?:www|dev)\.)?invidio\.us/|
|
(?:(?:www|dev)\.)?invidio\.us/|
|
||||||
(?:www\.)?invidiou\.sh/|
|
(?:(?:www|no)\.)?invidiou\.sh/|
|
||||||
(?:www\.)?invidious\.snopyta\.org/|
|
(?:(?:www|fi|de)\.)?invidious\.snopyta\.org/|
|
||||||
(?:www\.)?invidious\.kabi\.tk/|
|
(?:www\.)?invidious\.kabi\.tk/|
|
||||||
|
(?:www\.)?invidious\.enkirton\.net/|
|
||||||
|
(?:www\.)?invidious\.13ad\.de/|
|
||||||
|
(?:www\.)?invidious\.mastodon\.host/|
|
||||||
|
(?:www\.)?tube\.poal\.co/|
|
||||||
(?:www\.)?vid\.wxzm\.sx/|
|
(?:www\.)?vid\.wxzm\.sx/|
|
||||||
youtube\.googleapis\.com/) # the various hostnames, with wildcard subdomains
|
youtube\.googleapis\.com/) # the various hostnames, with wildcard subdomains
|
||||||
(?:.*?\#/)? # handle anchor (#/) redirect urls
|
(?:.*?\#/)? # handle anchor (#/) redirect urls
|
||||||
@ -1812,16 +1820,11 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
video_details = try_get(
|
video_details = try_get(
|
||||||
player_response, lambda x: x['videoDetails'], dict) or {}
|
player_response, lambda x: x['videoDetails'], dict) or {}
|
||||||
|
|
||||||
# title
|
video_title = video_info.get('title', [None])[0] or video_details.get('title')
|
||||||
if 'title' in video_info:
|
if not video_title:
|
||||||
video_title = video_info['title'][0]
|
|
||||||
elif 'title' in player_response:
|
|
||||||
video_title = video_details['title']
|
|
||||||
else:
|
|
||||||
self._downloader.report_warning('Unable to extract video title')
|
self._downloader.report_warning('Unable to extract video title')
|
||||||
video_title = '_'
|
video_title = '_'
|
||||||
|
|
||||||
# description
|
|
||||||
description_original = video_description = get_element_by_id("eow-description", video_webpage)
|
description_original = video_description = get_element_by_id("eow-description", video_webpage)
|
||||||
if video_description:
|
if video_description:
|
||||||
|
|
||||||
@ -1846,11 +1849,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
''', replace_url, video_description)
|
''', replace_url, video_description)
|
||||||
video_description = clean_html(video_description)
|
video_description = clean_html(video_description)
|
||||||
else:
|
else:
|
||||||
fd_mobj = re.search(r'<meta name="description" content="([^"]+)"', video_webpage)
|
video_description = self._html_search_meta('description', video_webpage) or video_details.get('shortDescription')
|
||||||
if fd_mobj:
|
|
||||||
video_description = unescapeHTML(fd_mobj.group(1))
|
|
||||||
else:
|
|
||||||
video_description = ''
|
|
||||||
|
|
||||||
if not smuggled_data.get('force_singlefeed', False):
|
if not smuggled_data.get('force_singlefeed', False):
|
||||||
if not self._downloader.params.get('noplaylist'):
|
if not self._downloader.params.get('noplaylist'):
|
||||||
@ -1888,6 +1887,9 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
|
|||||||
if view_count is None and video_details:
|
if view_count is None and video_details:
|
||||||
view_count = int_or_none(video_details.get('viewCount'))
|
view_count = int_or_none(video_details.get('viewCount'))
|
||||||
|
|
||||||
|
if is_live is None:
|
||||||
|
is_live = bool_or_none(video_details.get('isLive'))
|
||||||
|
|
||||||
# Check for "rental" videos
|
# Check for "rental" videos
|
||||||
if 'ypc_video_rental_bar_text' in video_info and 'author' not in video_info:
|
if 'ypc_video_rental_bar_text' in video_info and 'author' not in video_info:
|
||||||
raise ExtractorError('"rental" videos not supported. See https://github.com/ytdl-org/youtube-dl/issues/359 for more information.', expected=True)
|
raise ExtractorError('"rental" videos not supported. See https://github.com/ytdl-org/youtube-dl/issues/359 for more information.', expected=True)
|
||||||
@ -2421,7 +2423,7 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
(%(playlist_id)s)
|
(%(playlist_id)s)
|
||||||
)""" % {'playlist_id': YoutubeBaseInfoExtractor._PLAYLIST_ID_RE}
|
)""" % {'playlist_id': YoutubeBaseInfoExtractor._PLAYLIST_ID_RE}
|
||||||
_TEMPLATE_URL = 'https://www.youtube.com/playlist?list=%s'
|
_TEMPLATE_URL = 'https://www.youtube.com/playlist?list=%s'
|
||||||
_VIDEO_RE = r'href="\s*/watch\?v=(?P<id>[0-9A-Za-z_-]{11})&[^"]*?index=(?P<index>\d+)(?:[^>]+>(?P<title>[^<]+))?'
|
_VIDEO_RE = r'href="\s*/watch\?v=(?P<id>[0-9A-Za-z_-]{11})(?:&(?:[^"]*?index=(?P<index>\d+))?(?:[^>]+>(?P<title>[^<]+))?)?'
|
||||||
IE_NAME = 'youtube:playlist'
|
IE_NAME = 'youtube:playlist'
|
||||||
_TESTS = [{
|
_TESTS = [{
|
||||||
'url': 'https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re',
|
'url': 'https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re',
|
||||||
@ -2444,6 +2446,8 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'title': '29C3: Not my department',
|
'title': '29C3: Not my department',
|
||||||
'id': 'PLwP_SiAcdui0KVebT0mU9Apz359a4ubsC',
|
'id': 'PLwP_SiAcdui0KVebT0mU9Apz359a4ubsC',
|
||||||
|
'uploader': 'Christiaan008',
|
||||||
|
'uploader_id': 'ChRiStIaAn008',
|
||||||
},
|
},
|
||||||
'playlist_count': 95,
|
'playlist_count': 95,
|
||||||
}, {
|
}, {
|
||||||
@ -2452,6 +2456,8 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'title': '[OLD]Team Fortress 2 (Class-based LP)',
|
'title': '[OLD]Team Fortress 2 (Class-based LP)',
|
||||||
'id': 'PLBB231211A4F62143',
|
'id': 'PLBB231211A4F62143',
|
||||||
|
'uploader': 'Wickydoo',
|
||||||
|
'uploader_id': 'Wickydoo',
|
||||||
},
|
},
|
||||||
'playlist_mincount': 26,
|
'playlist_mincount': 26,
|
||||||
}, {
|
}, {
|
||||||
@ -2460,6 +2466,8 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'title': 'Uploads from Cauchemar',
|
'title': 'Uploads from Cauchemar',
|
||||||
'id': 'UUBABnxM4Ar9ten8Mdjj1j0Q',
|
'id': 'UUBABnxM4Ar9ten8Mdjj1j0Q',
|
||||||
|
'uploader': 'Cauchemar',
|
||||||
|
'uploader_id': 'Cauchemar89',
|
||||||
},
|
},
|
||||||
'playlist_mincount': 799,
|
'playlist_mincount': 799,
|
||||||
}, {
|
}, {
|
||||||
@ -2477,13 +2485,17 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'title': 'JODA15',
|
'title': 'JODA15',
|
||||||
'id': 'PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu',
|
'id': 'PL6IaIsEjSbf96XFRuNccS_RuEXwNdsoEu',
|
||||||
|
'uploader': 'milan',
|
||||||
|
'uploader_id': 'UCEI1-PVPcYXjB73Hfelbmaw',
|
||||||
}
|
}
|
||||||
}, {
|
}, {
|
||||||
'url': 'http://www.youtube.com/embed/_xDOZElKyNU?list=PLsyOSbh5bs16vubvKePAQ1x3PhKavfBIl',
|
'url': 'http://www.youtube.com/embed/_xDOZElKyNU?list=PLsyOSbh5bs16vubvKePAQ1x3PhKavfBIl',
|
||||||
'playlist_mincount': 485,
|
'playlist_mincount': 485,
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'title': '2017 華語最新單曲 (2/24更新)',
|
'title': '2018 Chinese New Singles (11/6 updated)',
|
||||||
'id': 'PLsyOSbh5bs16vubvKePAQ1x3PhKavfBIl',
|
'id': 'PLsyOSbh5bs16vubvKePAQ1x3PhKavfBIl',
|
||||||
|
'uploader': 'LBK',
|
||||||
|
'uploader_id': 'sdragonfang',
|
||||||
}
|
}
|
||||||
}, {
|
}, {
|
||||||
'note': 'Embedded SWF player',
|
'note': 'Embedded SWF player',
|
||||||
@ -2492,13 +2504,16 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'title': 'JODA7',
|
'title': 'JODA7',
|
||||||
'id': 'YN5VISEtHet5D4NEvfTd0zcgFk84NqFZ',
|
'id': 'YN5VISEtHet5D4NEvfTd0zcgFk84NqFZ',
|
||||||
}
|
},
|
||||||
|
'skip': 'This playlist does not exist',
|
||||||
}, {
|
}, {
|
||||||
'note': 'Buggy playlist: the webpage has a "Load more" button but it doesn\'t have more videos',
|
'note': 'Buggy playlist: the webpage has a "Load more" button but it doesn\'t have more videos',
|
||||||
'url': 'https://www.youtube.com/playlist?list=UUXw-G3eDE9trcvY2sBMM_aA',
|
'url': 'https://www.youtube.com/playlist?list=UUXw-G3eDE9trcvY2sBMM_aA',
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'title': 'Uploads from Interstellar Movie',
|
'title': 'Uploads from Interstellar Movie',
|
||||||
'id': 'UUXw-G3eDE9trcvY2sBMM_aA',
|
'id': 'UUXw-G3eDE9trcvY2sBMM_aA',
|
||||||
|
'uploader': 'Interstellar Movie',
|
||||||
|
'uploader_id': 'InterstellarMovie1',
|
||||||
},
|
},
|
||||||
'playlist_mincount': 21,
|
'playlist_mincount': 21,
|
||||||
}, {
|
}, {
|
||||||
@ -2523,6 +2538,7 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
'params': {
|
'params': {
|
||||||
'skip_download': True,
|
'skip_download': True,
|
||||||
},
|
},
|
||||||
|
'skip': 'This video is not available.',
|
||||||
'add_ie': [YoutubeIE.ie_key()],
|
'add_ie': [YoutubeIE.ie_key()],
|
||||||
}, {
|
}, {
|
||||||
'url': 'https://youtu.be/yeWKywCrFtk?list=PL2qgrgXsNUG5ig9cat4ohreBjYLAPC0J5',
|
'url': 'https://youtu.be/yeWKywCrFtk?list=PL2qgrgXsNUG5ig9cat4ohreBjYLAPC0J5',
|
||||||
@ -2534,7 +2550,6 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
'uploader_id': 'backuspagemuseum',
|
'uploader_id': 'backuspagemuseum',
|
||||||
'uploader_url': r're:https?://(?:www\.)?youtube\.com/user/backuspagemuseum',
|
'uploader_url': r're:https?://(?:www\.)?youtube\.com/user/backuspagemuseum',
|
||||||
'upload_date': '20161008',
|
'upload_date': '20161008',
|
||||||
'license': 'Standard YouTube License',
|
|
||||||
'description': 'md5:800c0c78d5eb128500bffd4f0b4f2e8a',
|
'description': 'md5:800c0c78d5eb128500bffd4f0b4f2e8a',
|
||||||
'categories': ['Nonprofits & Activism'],
|
'categories': ['Nonprofits & Activism'],
|
||||||
'tags': list,
|
'tags': list,
|
||||||
@ -2545,6 +2560,16 @@ class YoutubePlaylistIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
'noplaylist': True,
|
'noplaylist': True,
|
||||||
'skip_download': True,
|
'skip_download': True,
|
||||||
},
|
},
|
||||||
|
}, {
|
||||||
|
# https://github.com/ytdl-org/youtube-dl/issues/21844
|
||||||
|
'url': 'https://www.youtube.com/playlist?list=PLzH6n4zXuckpfMu_4Ff8E7Z1behQks5ba',
|
||||||
|
'info_dict': {
|
||||||
|
'title': 'Data Analysis with Dr Mike Pound',
|
||||||
|
'id': 'PLzH6n4zXuckpfMu_4Ff8E7Z1behQks5ba',
|
||||||
|
'uploader_id': 'Computerphile',
|
||||||
|
'uploader': 'Computerphile',
|
||||||
|
},
|
||||||
|
'playlist_mincount': 11,
|
||||||
}, {
|
}, {
|
||||||
'url': 'https://youtu.be/uWyaPkt-VOI?list=PL9D9FC436B881BA21',
|
'url': 'https://youtu.be/uWyaPkt-VOI?list=PL9D9FC436B881BA21',
|
||||||
'only_matching': True,
|
'only_matching': True,
|
||||||
@ -2711,6 +2736,8 @@ class YoutubeChannelIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UUKfVa3S1e4PHvxWcwyMMg8w',
|
'id': 'UUKfVa3S1e4PHvxWcwyMMg8w',
|
||||||
'title': 'Uploads from lex will',
|
'title': 'Uploads from lex will',
|
||||||
|
'uploader': 'lex will',
|
||||||
|
'uploader_id': 'UCKfVa3S1e4PHvxWcwyMMg8w',
|
||||||
}
|
}
|
||||||
}, {
|
}, {
|
||||||
'note': 'Age restricted channel',
|
'note': 'Age restricted channel',
|
||||||
@ -2720,6 +2747,8 @@ class YoutubeChannelIE(YoutubePlaylistBaseInfoExtractor):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UUs0ifCMCm1icqRbqhUINa0w',
|
'id': 'UUs0ifCMCm1icqRbqhUINa0w',
|
||||||
'title': 'Uploads from Deus Ex',
|
'title': 'Uploads from Deus Ex',
|
||||||
|
'uploader': 'Deus Ex',
|
||||||
|
'uploader_id': 'DeusExOfficial',
|
||||||
},
|
},
|
||||||
}, {
|
}, {
|
||||||
'url': 'https://invidio.us/channel/UC23qupoDRn9YOAVzeoxjOQA',
|
'url': 'https://invidio.us/channel/UC23qupoDRn9YOAVzeoxjOQA',
|
||||||
@ -2804,6 +2833,8 @@ class YoutubeUserIE(YoutubeChannelIE):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UUfX55Sx5hEFjoC3cNs6mCUQ',
|
'id': 'UUfX55Sx5hEFjoC3cNs6mCUQ',
|
||||||
'title': 'Uploads from The Linux Foundation',
|
'title': 'Uploads from The Linux Foundation',
|
||||||
|
'uploader': 'The Linux Foundation',
|
||||||
|
'uploader_id': 'TheLinuxFoundation',
|
||||||
}
|
}
|
||||||
}, {
|
}, {
|
||||||
# Only available via https://www.youtube.com/c/12minuteathlete/videos
|
# Only available via https://www.youtube.com/c/12minuteathlete/videos
|
||||||
@ -2813,6 +2844,8 @@ class YoutubeUserIE(YoutubeChannelIE):
|
|||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'UUVjM-zV6_opMDx7WYxnjZiQ',
|
'id': 'UUVjM-zV6_opMDx7WYxnjZiQ',
|
||||||
'title': 'Uploads from 12 Minute Athlete',
|
'title': 'Uploads from 12 Minute Athlete',
|
||||||
|
'uploader': '12 Minute Athlete',
|
||||||
|
'uploader_id': 'the12minuteathlete',
|
||||||
}
|
}
|
||||||
}, {
|
}, {
|
||||||
'url': 'ytuser:phihag',
|
'url': 'ytuser:phihag',
|
||||||
@ -2906,7 +2939,7 @@ class YoutubePlaylistsIE(YoutubePlaylistsBaseInfoExtractor):
|
|||||||
'playlist_mincount': 4,
|
'playlist_mincount': 4,
|
||||||
'info_dict': {
|
'info_dict': {
|
||||||
'id': 'ThirstForScience',
|
'id': 'ThirstForScience',
|
||||||
'title': 'Thirst for Science',
|
'title': 'ThirstForScience',
|
||||||
},
|
},
|
||||||
}, {
|
}, {
|
||||||
# with "Load more" button
|
# with "Load more" button
|
||||||
@ -2923,6 +2956,7 @@ class YoutubePlaylistsIE(YoutubePlaylistsBaseInfoExtractor):
|
|||||||
'id': 'UCiU1dHvZObB2iP6xkJ__Icw',
|
'id': 'UCiU1dHvZObB2iP6xkJ__Icw',
|
||||||
'title': 'Chem Player',
|
'title': 'Chem Player',
|
||||||
},
|
},
|
||||||
|
'skip': 'Blocked',
|
||||||
}]
|
}]
|
||||||
|
|
||||||
|
|
||||||
|
@ -1,3 +1,3 @@
|
|||||||
from __future__ import unicode_literals
|
from __future__ import unicode_literals
|
||||||
|
|
||||||
__version__ = '2019.07.02'
|
__version__ = '2019.07.30'
|
||||||
|
Loading…
Reference in New Issue
Block a user