Marcin Cieślak a8eeb0597b Fix AssertionError when og property not found ...

On tvp.pl some webpages contain OpenGraph
metadata and some don't.

If og property is not found, _og_search_description
fails with

WARNING: unable to extract OpenGraph description; please report this issue on http://yt-dl.org/bug
Traceback (most recent call last):
  File "/usr/home/saper/bin/youtube-dl", line 18, in <module>
    youtube_dl.main()
  File "/usr/home/saper/sw/youtube-dl/youtube_dl/__init__.py", line 766, in main
    _real_main(argv)
  File "/usr/home/saper/sw/youtube-dl/youtube_dl/__init__.py", line 719, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/home/saper/sw/youtube-dl/youtube_dl/YoutubeDL.py", line 715, in download
    videos = self.extract_info(url)
  File "/usr/home/saper/sw/youtube-dl/youtube_dl/YoutubeDL.py", line 348, in extract_info
    ie_result = ie.extract(url)
  File "/usr/home/saper/sw/youtube-dl/youtube_dl/extractor/common.py", line 125, in extract
    return self._real_extract(url)
  File "/usr/home/saper/sw/youtube-dl/youtube_dl/extractor/tvp.py", line 56, in _real_extract
    info['description'] = self._og_search_description(webpage)
  File "/usr/home/saper/sw/youtube-dl/youtube_dl/extractor/common.py", line 331, in _og_search_description
    return self._og_search_property('description', html, fatal=False, **kargs)
  File "/usr/home/saper/sw/youtube-dl/youtube_dl/extractor/common.py", line 325, in _og_search_property
    return unescapeHTML(escaped)
  File "/usr/home/saper/sw/youtube-dl/youtube_dl/utils.py", line 494, in unescapeHTML
    assert type(s) == type(u'')
AssertionError

The patch allows me to use:

  try:
    info['description'] = self._og_search_description(webpage)
    info['thumbnail'] = self._og_search_thumbnail(webpage)
  except RegexNotFoundError:
    pass

2013-11-05 23:19:29 +01:00

..

__init__.py

Merge remote-tracking branch 'CBGoodBuddy/ytsearchtime'

2013-11-03 15:56:55 +01:00

addanime.py

Let extractors omit ext in formats

2013-10-28 11:28:02 +01:00

appletrailers.py

[appletrailers] The request for the settings must have the trailer name in lower case (fixes #1329)

2013-09-29 20:59:19 +02:00

archiveorg.py

Set the ext field for each format

2013-09-14 14:45:04 +02:00

ard.py

Do not redefine variables in list comprehensions

2013-07-08 02:12:20 +02:00

arte.py

[arte] Download the 'Originalversion' version if it's the only one available (fixes #1682)

2013-10-29 20:58:49 +01:00

auengine.py

[auengine] Add tests (Fixes #985)

2013-07-03 16:36:36 +02:00

bambuser.py

[bambuser] Add an extractor for channels (closes #1702)

2013-11-02 19:50:57 +01:00

bandcamp.py

Move tests to the IE definitions

2013-06-27 20:46:46 +02:00

bliptv.py

[bliptv] Make sure video ID is a string

2013-10-05 16:12:29 +02:00

bloomberg.py

Add an extractor for Bloomberg (closes #1436)

2013-09-16 20:38:48 +02:00

breakcom.py

Fix BreakIE

2013-07-23 11:41:05 +02:00

brightcove.py

[brightcove] Don't set the extension

2013-11-02 21:20:46 +01:00

c56.py

fixed tests for c56 and dailymotion

2013-08-27 01:00:17 +05:00

canalc2.py

[canal2c] Accept more urls (fixes #1723)

2013-11-04 22:26:19 +01:00

canalplus.py

[canalplust] accept urls that don't include the video id (fixes #1415), extract more info and update test

2013-09-12 21:56:36 +02:00

cinemassacre.py

[CinemassacreIE] Support more embed urls

2013-11-01 01:24:23 +01:00

cnn.py

[cnn] remove debug print call.

2013-08-28 10:20:37 +02:00

collegehumor.py

[collegehumor] Allow old-style videos (Fixes #1285)

2013-08-21 03:50:56 +02:00

comedycentral.py

[comedycentral] Prepare for generic video extraction (#980)

2013-10-04 11:14:10 +02:00

common.py

Fix AssertionError when og property not found

2013-11-05 23:19:29 +01:00

condenast.py

Add CondeNastIE

2013-07-17 14:39:02 +02:00

criterion.py

CriterionIE: simplify some parts and use _html_search_regex

2013-07-13 12:26:05 +02:00

cspan.py

InfoExtractor: add some helper methods to extract OpenGraph info

2013-07-12 22:12:04 +02:00

dailymotion.py

[dailymotion] Fix support for age-restricted videos (Fixes #1688)

2013-10-31 00:20:49 +01:00

daum.py

[daum] Get the video page from a canonical url to extract the full id (fixes #1373) and extract description.

2013-09-05 10:08:17 +02:00

defense.py

Suggested by @phihag

2013-09-03 12:03:19 +02:00

depositfiles.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

dotsub.py

DotsubIE: simplify and extract the upload date

2013-07-08 22:40:42 +02:00

dreisat.py

Set the ext field for each format

2013-09-14 14:45:04 +02:00

ebaumsworld.py

Add an extractor for ebaumsworld.com (closes #1462)

2013-09-20 16:55:50 +02:00

ehow.py

InfoExtractor: add some helper methods to extract OpenGraph info

2013-07-12 22:12:04 +02:00

eighttracks.py

[8tracks] Use track count instead of looking at at_last_track property

2013-10-25 23:46:19 +02:00

escapist.py

InfoExtractor: add some helper methods to extract OpenGraph info

2013-07-12 22:12:04 +02:00

exfm.py

[exfm] skip tests

2013-11-02 20:51:09 +01:00

extremetube.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

facebook.py

[facebook] Don't recommend to report the issue if the video is private.

2013-10-27 12:13:55 +01:00

faz.py

[faz] Use a regex for getting the description

2013-10-27 14:18:55 +01:00

fktv.py

[fktv] support videos splitted in any number of parts and some style changes

2013-09-18 23:32:37 +02:00

flickr.py

[FlickrIE] Fix HTTPS url

2013-10-04 07:47:40 +02:00

francetv.py

[francetv] Use common format for ID of generation-quoi subextractor

2013-10-04 11:30:47 +02:00

freesound.py

[freesound] Minor improvements

2013-07-15 21:33:45 +02:00

funnyordie.py

[funnyordie] Fix video url extraction

2013-09-20 13:05:34 +02:00

gamespot.py

[gamespot] Fix video extraction (fixes #1587)

2013-10-14 16:46:07 +02:00

gametrailers.py

GametrailersIE: make it a subclass of MTVIE to reuse most of the extraction process

2013-07-14 14:29:15 +02:00

generic.py

[GenericIE] Also detect youtube if src url of iframe is embedded in ' instaed of "

2013-11-05 02:08:02 +01:00

googleplus.py

[googleplus] Fix upload_date extraction

2013-10-21 15:00:21 +02:00

googlesearch.py

Add --list-extractor-descriptions (human-readable list of IEs)

2013-07-01 18:52:19 +02:00

hark.py

[hark] get the song info in JSON and extract more information.

2013-08-27 10:25:38 +02:00

hotnewhiphop.py

[hotnewhiphop] Fix test case title

2013-09-17 21:10:57 +02:00

howcast.py

[howcast] Do not download from http://www.howcast.com/videos/{video_id}

2013-09-11 17:36:23 +02:00

hypem.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

ign.py

[ign] Support more urls and detect multiple videos in articles (fixes #1543)

2013-10-02 20:59:34 +02:00

ina.py

[ina] Allow I at start of video IDs

2013-07-25 09:52:58 +02:00

infoq.py

Tests: Add coding to files

2013-06-27 20:32:02 +02:00

instagram.py

[Instagram] get the non-https link, as they are serving Akamai cert from a instagram.com domain

2013-10-28 02:34:29 -04:00

internetvideoarchive.py

Fix the duration field for the VideoDetective and InternetVideoArchive tests

2013-10-21 15:07:33 +02:00

jeuxvideo.py

[jeuxvideo] Improve code quality (fixes #1567)

2013-10-07 14:33:23 +02:00

jukebox.py

JukeboxIE: support more countries

2013-06-25 22:04:44 +02:00

justintv.py

Move tests to the IE definitions

2013-06-27 20:46:46 +02:00

kankan.py

fix bug: kankan extractor not support http://vod.kankan.com/v/70/70309.shtml

2013-08-02 15:26:11 +08:00

keek.py

[keek] Fix testcase (Broken by accident in 6625f82940b2701e2ccc9a6cf5808056b075e16b)

2013-07-25 10:10:37 +02:00

keezmovies.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

kickstarter.py

Merge pull request #1409 from JohnyMoSwag/master (closes #1404)

2013-09-13 19:52:56 +02:00

liveleak.py

InfoExtractor: add some helper methods to extract OpenGraph info

2013-07-12 22:12:04 +02:00

livestream.py

[livestream] fix the extraction of events

2013-10-31 08:07:26 +01:00

metacafe.py

[metacafe] Fix support for age-restricted videos (fixes #1696)

2013-11-01 11:56:15 +01:00

metacritic.py

Add an extractor for Metacritic

2013-09-06 18:08:07 +02:00

mit.py

Fix MIT extractor for Python 2.6

2013-08-28 14:24:42 -05:00

mixcloud.py

[mixcloud] Rewrite extractor (fixes #278)

2013-09-14 14:26:42 +02:00

mofosex.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

mtv.py

[mtv] Remove the templates from the mediagen url

2013-10-28 23:37:01 +01:00

muzu.py

Add an extractor from muzu.tv (closes #1177)

2013-08-04 11:10:57 +02:00

myspace.py

Add an extractor for MySpace (closes #1666)

2013-10-28 22:02:17 +01:00

myspass.py

Move tests to the IE definitions

2013-06-27 20:46:46 +02:00

myvideo.py

[myvideo] add support for videos that place the video info inside www.myvideo.de/service/data/video/{id}/config (fixes #616)

2013-08-02 21:09:17 +02:00

naver.py

Add extractor for tvcast.naver.com (closes #1331)

2013-09-05 10:53:40 +02:00

nba.py

InfoExtractor: add some helper methods to extract OpenGraph info

2013-07-12 22:12:04 +02:00

nbc.py

Add an extractor for NBC news (closes #1320)

2013-08-27 12:38:57 +02:00

newgrounds.py

[newgrounds] simplify

2013-09-18 22:14:43 +02:00

nhl.py

[nhl:videocenter] Fix playlist title extraction

2013-10-22 21:01:16 +02:00

nowvideo.py

[nowvideo] Fix key extraction

2013-10-23 17:00:33 +02:00

ooyala.py

Add an extractor for vice.com (closes #1051)

2013-09-16 20:58:36 +02:00

orf.py

[orf] Remove the test video, they seem to expire in one week

2013-09-03 11:51:01 +02:00

pbs.py

Add an extractor for PBS (closes #870 and #873)

2013-08-22 11:57:21 +02:00

photobucket.py

Move tests to the IE definitions

2013-06-27 20:46:46 +02:00

pornhub.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

pornotube.py

Add the missing age_limit tags; added a devscript to do a superficial check for porn sites without the age_limit tag in the test

2013-10-28 01:50:17 -04:00

rbmaradio.py

Move tests to the IE definitions

2013-06-27 20:46:46 +02:00

redtube.py

extractor: Set age limit on some adult-related extractors.

2013-10-19 14:19:25 -03:00

ringtv.py

Add RingTVIE (Thanks @yasoob)

2013-06-28 18:51:00 +02:00

ro220.py

[ro220] Add support for 220.ro

2013-08-24 21:10:03 +02:00

rottentomatoes.py

Add an extractor for rottentomatoes.com and improve InternetVideoArchiveIE to get the best quality

2013-10-12 22:22:31 +02:00

roxwel.py

Add an extractor for roxwell.com (closes #1044)

2013-07-26 13:00:59 +02:00

rtlnow.py

[rtlnow] Change the test for rtlnitronow

2013-10-27 14:26:19 +01:00

rutube.py

Add an extractor for rutube.ru (closes #1136)

2013-10-16 16:57:40 +02:00

sina.py

Add SinaIE (related #1039): extractor for video.sina.com.cn

2013-07-18 15:31:50 +02:00

slashdot.py

Add an extractor for tv.slashdot.org (closes #1192)

2013-08-21 21:51:58 +02:00

slideshare.py

Add an extractor for Slideshare (closes #1400)

2013-09-10 11:19:58 +02:00

sohu.py

[sohu] add support for my.tv.sohu.com urls (fixes #1398)

2013-09-09 19:56:16 +02:00

soundcloud.py

[soundcloud] Add an extractor for users (closes #1426)

2013-09-14 21:41:49 +02:00

southparkstudios.py

Revert "[southparkstudios] Fix mgid extraction"

2013-09-24 21:39:38 +02:00

spankwire.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

spiegel.py

Move tests to the IE definitions

2013-06-27 20:46:46 +02:00

stanfordoc.py

Add --list-extractor-descriptions (human-readable list of IEs)

2013-07-01 18:52:19 +02:00

statigram.py

[statigram] change test video

2013-08-21 19:20:27 +02:00

steam.py

Set the playlist_index and playlist fields for already resolved video results.

2013-07-10 23:36:30 +02:00

subtitles.py

Now --all-sub is a modifier to --write-sub and --write-auto-sub (closes #1412)

2013-09-14 11:14:40 +02:00

sztvhu.py

[sztvhu] Fix the title extraction

2013-10-15 08:22:59 +02:00

teamcoco.py

[teamcoco] Parse the xml file and extract all the formats

2013-11-03 17:48:12 +01:00

techtalks.py

Add an extractor for techtalks.tv (closes #1606)

2013-10-17 08:20:58 +02:00

ted.py

[ted] Prepare #980 merge

2013-10-04 10:32:34 +02:00

tf1.py

Disable way and tf1 tests, the whole videos are served sometimes, so the md5 sum doesn't match.

2013-07-30 11:19:07 +02:00

thisav.py

[thisav] add a test for video MD5

2013-07-17 11:18:14 +02:00

traileraddict.py

Merge pull request #1114 from alexvh/traileraddict_hd

2013-07-24 10:52:24 +02:00

trilulilu.py

Set the ext field for each format

2013-09-14 14:45:04 +02:00

tube8.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

tudou.py

[tudou] Fix title regex (Fixes #1614)

2013-10-18 11:16:20 +02:00

tumblr.py

TumblrIE: update test

2013-06-30 13:52:20 +02:00

tutv.py

InfoExtractor: add some helper methods to extract OpenGraph info

2013-07-12 22:12:04 +02:00

unistra.py

Update descriptions checksum for some test for Unistra and Youtube

2013-08-29 13:41:59 +02:00

ustream.py

[ustream] Simplify channel extraction

2013-09-13 22:05:29 +02:00

vbox7.py

Move tests to the IE definitions

2013-06-27 20:46:46 +02:00

veehd.py

Add an extractor for VeeHD (closes #1359)

2013-09-02 11:54:09 +02:00

veoh.py

VeohIE: remove debug logging

2013-07-09 11:11:55 +02:00

vevo.py

[vevo] Readd support for SMIL (Fixes #1683)

2013-10-30 01:14:17 +01:00

vice.py

Add an extractor for vice.com (closes #1051)

2013-09-16 20:58:36 +02:00

viddler.py

[viddler] Support non-digit IDs (Fixes #1714)

2013-11-03 15:49:59 +01:00

videodetective.py

Fix the duration field for the VideoDetective and InternetVideoArchive tests

2013-10-21 15:07:33 +02:00

videofyme.py

[videofyme] Add an additional quality (they change between downloads of the info) and update md5 sum of the test video

2013-08-22 23:23:52 +02:00

videopremium.py

add VideoPremium.tv RTMP support

2013-10-14 01:32:47 -04:00

vimeo.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

vine.py

InfoExtractor: add some helper methods to extract OpenGraph info

2013-07-12 22:12:04 +02:00

vk.py

Add an extractor for vk.com (closes #1635)

2013-11-01 22:34:18 +01:00

wat.py

Remove unused imports

2013-08-28 12:47:38 +02:00

websurg.py

[websurg] Skipt the test

2013-10-15 08:12:30 +02:00

weibo.py

WeiboIE: extract the player urls from a json webpage

2013-07-19 20:43:44 +02:00

wimp.py

Move tests to the IE definitions

2013-06-27 20:46:46 +02:00

worldstarhiphop.py

[worldstarhiphop] Small cleanup

2013-07-30 11:10:17 +02:00

xhamster.py

[XHamsterIE] Extract SD and HD video

2013-10-26 20:38:54 +02:00

xnxx.py

[xnxx] Add age_limit

2013-10-18 23:35:17 +02:00

xtube.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

xvideos.py

extractor: Set age limit on some adult-related extractors.

2013-10-18 19:32:37 -03:00

yahoo.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

youjizz.py

Add the missing age_limit tags; added a devscript to do a superficial check for porn sites without the age_limit tag in the test

2013-10-28 01:50:17 -04:00

youku.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

youporn.py

Style fixes for extractors: remove spaces around (,),{ and }

2013-11-03 14:06:47 +01:00

youtube.py

[youtube] Add description for YoutubeSearchDateIE (#1710)

2013-11-03 15:59:10 +01:00

zdf.py

[zdf] Add support for hash URLs and simplify (#1518)

2013-09-28 15:43:34 +02:00