CLIVE(1) User Commands CLIVE(1) SYNOPSIS clive [ OPTION ]... [ URL ]... DESCRIPTION clive is an open source command line tool to extract videos and to bypass the need to use Adobe Flash in order to view user-generated con- tent available on video-sharing websites. OPTIONS --version Print version to stdout and exit. -h, --help Print out a verbose message describing the program options and exit. -q, --quiet Causes clive to be less verbose. config: enable_verbose=["YES"|"NO"] -u, --check-update Check for update and prompt if a new version is available. -n, --no-extract Do not actually download any videos. Simulate only. config: enable_extract=["YES"|"NO"] -d, --clear-cache Clear URL cache. See FILES. -D, --clear-last Clear last recorded URL (batch). See FILES. -w, --write-conf Write configuration file which will contain all supported com- mands. Attempts to autodetect ffmpeg(1) xine(1) xclip(1) and some other player commands. See FILES. -L, --low-quality Explicitly extract low-quality video. clive extracts better quality by default but switches to low quality if a high-qual- ity link is not found. See HOSTS. config: enable_low_quality=["YES"|"NO"] -l, --recall Recall the URL (batch) from the last runtime. clive saves the batch by default so it can be recalled with this option without having to re-type it. Note that the data gets overwrittenon every runtime unless the --no-recall option is toggled. See FILES. See also: --cache -R, --no-recall Do not save URL (batch) for the --recall option. config: enable_recall=["YES"|"NO"] -e, --emit-csv Emit video details in CSV format without extracting the videos. -F, --rss Causes clive to treat URLs as RSS/Atom feed URLs. The feeds are fetched, parsed and displayed with found video items. For exam- ple: % clive --rss http://youtube.com/rss/user/lisanova/videos.rss Feed URLs can also be saved to the configuration file. For example: ... feed_url="http://youtube.com/rss/user/lisanova/videos.rss" feed_url="http://youtube.com/rss/user/spricket24/videos.rss" feed_url="http://youtube.com/rss/user/theonion/videos.rss" feed_url="http://youtube.com/rss/user/clipcritics/videos.rss" ... Once saved, run clive with: % clive --rss -c, --cache Displays URL cache and re-extract videos using the collected cache data. See FILES. See also: --no-cache --clear-cache -C, --no-cache Turn off all URL caching features. clive will not read or write cache. URL caching speeds up re-extracting videos. Instead of re-fetching the video page data from a host, clive reuses the cache data. For some hosts, e.g. Youtube, the page needs to be re-fetched as the saved extraction URL expires eventually. If you are getting HTTP/410 or HTTP/403 errors, try disabling the caching. This is often an indication of using an expired extraction URL. Note that clive disables the caching automatically if the Python SQLite3 module is not found. config: enable_cache=["YES"|"NO"] -S, --scan Causes clive to scan the URLs for video links. These video links may be either embedded or normal video page links. For example: % clive --scan http://fakesteve.blogspot.com % clive --scan http://youtube.com The found links are then visited, parsed and displayed for extraction. Note that clive currently scans for Youtube and GoogleVideo links only. -Q, --no-confirm Do not prompt for approval before each action. For example: % clive --scan --no-confirm http://youtube.com The above example would cause clive to scan the front page of Youtube for videos and then extract all of them without select- ing them first. Output Options -s, --savedir=PATH Specifies the save directory for the extracted videos. config: output_savedir=["PATH"] -a, --exists=ACTION By default, clive will attempt to continue partially downloaded videos (local_file < remote_file).If this is not possible or not preferable, this option can be used to define the action if a file exists already. rename ( default ) causes the program to skip the already existing file and re-download the file with a slightly differ- ent name. Typically the new file will contain the original filename and a random string appended to it. overwrite causes the program to overwrite the existing file. config: output_if_file_exists=["RENAME"|"OVERWRITE"] -o, --output=FILENAME Specifies the output filename for the video. NOTE: Do not use this option when extracting a batch of videos. -m, --filter=ACTION Specifies the preferred method for filtering video page title information which is used to construct the video output file- name. clive extracts the video page title of each the URL that it visits. After this the program strips any host specific strings from the title (e.g. site name) and proceeds to one of the three options: 1: Default filter is very strict about the used characters, accepting only characters of A..Z, a..z and 0..9 from the orig- inal video title to be used in the filename. This option can be used to customize the regular expression used by default ("A-Za-z0-9"), write a custom method that handles the filtering instead of clive or disable the filter altogether to use the whole video page title as a video filename. If regular expression is used for action, clive calls Python's re.sub method with it. For example, the default regexp is: "A-Za-z0-9". The method is called in the following manner: re.sub('[^%s]' % regexp, '', title) You may want to refer to the re.sub documentation at: http://docs.python.org/lib/node46.html 2: Custom Python method that clive calls instead of the re.sub method is also possible. This requires some basic knowledge of the Python programming language, however. Below is a very basic example that does nothing more but returns the original page title in lower-case: def custom_output_filter(title): return title.lower() The method must be saved to ~/.clive/custom.py and when you run the program you must set the filter to "custom", e.g.: % clive --filter=custom URL clive which uses it for naming the video (original video title in lower-case). 3: Disabling the filtering is the quickest way to keep the original video title. This can, however, result in clive crash- ing if any of the characters used in the video page title can- not be used in a filename. Set the action to "no" if you wish to disable the filter, e.g.: % clive --filter=no URL config: output_title_filter=["regexp"|"NO"|"CUSTOM"] See also: --format -f, --format=FORMAT Specifies the output filename format. Used to define how clive should construct the video output filename from all the avail- able elements. The default format is "%t.%e". Available format specifiers are: "%t" (title), "%e" (extension), "%i" (video id) and "%h" (host). config: output_filename_format=["FORMAT"] Connection Options -y, --proxy=PROXY Use HTTP proxy. clive reads the http_proxy environment setting by default. This option overrides the http_proxy setting (if any). Authentication information may be provided using normal URL constructs, e.g.: % clive --proxy=http://user:host@foo:1234 config: http_proxy=["PROXY"] -N, --no-proxy Disable use of HTTP proxy altogether. -t, --throttle=RATE Causes clive to throttle the bandwidth to the specified rate (KB/s). Setting this option to 0 equals to unlimited. config: http_throttle=["RATE"] -g, --agent=STRING Use HTTP user-agent string. By default, the program generates a random string each runtime. config: http_agent=["STRING"] --youtube=LOGINFO Username and password for Youtube in format "username:pass- word". Used to login and extract flagged content from the host. To use encrypted passwords stored with clive-passwd(1) set this option to "passwd:entry". For example: % clive-passwd add yt % clive --youtube=passwd:yt config: youtube_login=["USERNAME|passwd:PASSWORD|ENTRY"] See also: --passwd --dmotion=LOGINFO Like --youtube above but for Dailymotion. config: dmotion_login=["USERNAME|passwd:PASSWORD|ENTRY"] See also: --passwd -P, --passwd=PASSPHRASE Use passphrase for ~/.clive/passwd without prompting for it. -G, --no-login Do not attempt to login with provided username and password before extraction. Playback Options --player=PATH Specifies path to a player command with any options. The format specifier "%i" must be used in the path. For example: % clive --player="/usr/local/bin/xine %i" config: path_player=["PATH"] -p, --play=FORMAT Specifies the video format to play. The format can be anything as long as ffmpeg(1) recognizes the format. The only exception is "src" which tells clive to play the extracted video without re-encoding it to another format. Examples: "src", "mpg", "avi", "mp4". The --ffmpeg option must be used if the format is anything else than "src". % clive --play=src URL % clive --play=mpg --ffmpeg="/usr/bin/ffmpeg -y -i %i %o" URL config: play_format=["FORMAT"] See also: --ffmpeg --reencode Re-encoding Options --ffmpeg=PATH Specifies the path to the ffmpeg(1) command with any options. For example: % clive --ffmpeg="/usr/bin/ffmpeg -y -i %i %o" URL Note that the format specifiers "%i" and "%o" must be used in the path. Using the "-y" option is recommended to prevent any subprocess lockups which will happen if the output file exists already and ffmpeg(1) prompts to confirm to overwrite it. config: path_ffmpeg=["PATH"] -r, --reencode=FORMAT Causes clive to call ffmpeg(1) to re-encode the extracted video to another format. The format can be anything as long as ffm- peg(1) recognizes it. Examples: "mpg", "avi", "mp4". The --ffm- peg option must be used together with this option. Note that if the --play option is being used with anything else than "src", you don't need to use this option. For example: % clive --ffmpeg="/usr/bin/ffmpeg -y -i %i %o" --play=mpg URL This would extract the video, re-encode it to mpg format and then play the mpg. config: reencode_format=["FORMAT"] Clipboard Options --xclip=PATH Specifies the path to xclip(1) command with any options. Note that the "-o" option must be used in the path. For example: % clive --xclip="/usr/bin/xclip -o" --paste config: path_xclip=["PATH"] -x, --paste Causes clive to paste the URLs from X clipboard. config: enable_paste=["YES"|"NO"] EXAMPLES Using a configuration file is recommended if you are going to use ffm- peg(1) xine(1) xclip(1) commands or any other commands. The easiest way to create the file is to run: % clive --write-conf And then edit the ~/.clive/config file manually. Note that clive will attempt to locate the commands automatically and save the ffmpeg, xine and xclip paths with the min. req. options. So in a perfect world you could ignore the --ffmpeg --player --xclip options used in the examples below. Extract a video from Youtube: % clive http://youtube.com/watch?v=dr3qPRAAnOg Extract multiple videos from Youtube using UNIX pipes: % cat url.lst http://youtube.com/watch?v=dr3qPRAAnOg http://youtube.com/watch?v=VlFGTtU65Xo % cat url.lst | clive Same as above but paste the URL batch from X clipboard: % xclip -o http://youtube.com/watch?v=dr3qPRAAnOg http://youtube.com/watch?v=VlFGTtU65Xo % clive --xclip="/usr/bin/xclip -o" --paste Extract and play a video from Youtube: % clive --player="/usr/bin/xine %i" --play=src http://youtube.com/watch?v=dr3qPRAAnOg Extract, re-encode and play the re-encoded video: % clive --ffmpeg="/usr/bin/ffmpeg -y -i %i %o" --player="/usr/bin/xine %i" --play=mpg http://youtube.com/watch?v=dr3qPRAAnOg Parse a Youtube RSS feed: % clive --rss http://youtube.com/rss/user/clipcritics/videos.rss Scan Youtube front page for videos: % clive --scan http://youtube.com Browse existing URL cache: % clive --cache Advanced example - paste (-x) an URL from X clipboard, scan (-S) the URL for videos, extract as low-quality (-L) and disable URL caching (-C): % clive -xSLC HOSTS Youtube.com format:flv/mp4 resume:mp4 quality:low/high video.google.(com|au|ca|de|es|fr|it|nl|pl|uk|cn) format:flv/mp4 resume:mp4 quality:low/high dailymotion.com format:varies resume:yes quality:varies guba.com format:flv resume:yes quality:low metacafe.com format:flv resume:yes quality:low last.fm Jzalae: "Last.fm recently added a video feature which appears to be a simple wrapping of YouTube videos, and the YouTube id appears in the Last.fm URL. The attached patch finds it from URLs like http://www.last.fm/music/Sylver/+videos/+1-fVD2TfSGkaw" sevenload.com format:flv resume:yes quality:low myvideo.(de|ch|at|nl|be) format:flv resume:yes quality:low break.com format:flv/wmv resume:yes quality:low/high FILES ~/.clive/config User configuration file which can be created with the --write-conf option. ~/.clive/cache The URL cache file. See the --cache --no-cache options. This is a normal SQLite3 database. ~/.clive/recall Contains the last saved URL (batch). See also the --recall --no-recall program options. ~/.clive/passwd Stores encrypted password information for host specific logins. This file is created and maintained using the clive-passwd(1) tool. ENVIRONMENT http_proxy HTTP proxy setting. Authentication information may be provided using normal URL constructs, e.g.: http://user:host@foo:1234 BUGS Dailymotion: login verification clive cannot currently verify if the login succeeded. The only way to know for sure, is to login and then try extract a video that requires logging in will result in "error: extraction url not found". Youtube: partially downloaded FLV videos If the host is requested to continue a transfer, it appears to ignore the request and begins sending the file from the first byte rather than from the requested range. This seems to cause urlgrab- ber, which is used to transfer files, to append the new data incor- rectly at the end of the existing file causing the file to become useless. Because known workarounds do not exist, clive will always skip the existing file creating either a new file (default) or overwriting the existing file ( --exists option). GoogleVideo: partially downloaded FLV videos See above. AUTHOR Toni Gundogdu . Bugs should be reported to the clive bug tracking system: https://gna.org/bugs/?func=additem&group=clive SEE ALSO clive-passwd(1) COPYRIGHT Copyright (C) 2007-2008 Toni Gundogdu This program comes with ABSOLUTELY NO WARRANTY. You may redistribute copies of clive under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. You should have received a copy of of the General Public License along with this pro- gram. If not, see http://www.gnu.org/licenses/. http://home.gna.org/clive August 2008 CLIVE(1)