How to install and run youtube-dl from actual source code, so that you can apply patches as needed

If you use the program youtube-dl, you may have noticed that from time to time things stop working.  Usually it’s because something changes on a site and youtube-dl needs to be patched to deal with it.  But what sometimes happens is that someone will submit a patch and then the youtube-dl developers will ignore that patch for many months before they finally get around to merging it into the program.  The speed at which the youtube-dl developers fix issues with sites can seem glacial at times.

What’s frustrating is that if you browse through the issues section of the youtube-dl support area using an appropriate search term, you may find the fix for your problem, and maybe it’s just been sitting there for quite some time. But, since youtube-dl is distributed as a compiled binary you may think you have no way to apply it.  But that’s not really true – at its core, the entire program is simply a multitude of Python scripts, and it can be run without first being compiled.  And if you use it that way, then you can apply patches as needed to any of the python modules, whether they are patches you have written or patches that you have found in the issues area, or elsewhere on the Internet.

So here is how you can install and use youtube-dl using actual source code.  Please don’t confuse this with other sites that show you how to “install from source” because in most cases what they are showing you how to do is install the binary from the developer’s source site rather than using an old version from your Linux distro’s repository, which is never a good idea since youtube-dl gets updated a lot.  I should mention here that since youtube-dl runs under Python, in theory it will run on any system that has a Python 2.6, 2.7, or 3.2+ interpreter available.  So although it is not Linux-specific, these instructions are aimed primarily at Linux users, and they will probably also work in MacOS. They should work in Windows, too, but we don’t do much in Windows so can’t say what additional steps might be involved.  And, many Linux distributions, as well as MacOS, have Python already included in their base installation, so we’re not going to tell you how to install Python if you don’t already have it.  Just type python --version from a command prompt and if python is already installed, it should tell you which version of Python you have.

So here’s the procedure: First, create a directory under your home directory, maybe call it ytdl, and change to that directory:

cd ~
mkdir ytdl
cd ytdl

There are two versions of the youtube-dl source code that can be obtained.  One is the source code only, and the other is the Full source + docs + binary, which is a much larger download but can be verified using a SHA256 checksum. You can use either one.

If you want the (larger) Full source + docs + binary version, do this: In a web browser go to the youtube-dl Download Page and look for the link that says Full source + docs + binary tarball:

Full source + docs + binary tarball link
That link is updated when new versions are released. Click it and download the resulting file, or right click on it and copy the link, and then in a terminal window use wget to get the file.  The file should be downloaded to or placed in the directory you just created.  For example:

cd ~/ytdl (only necessary if you have changed directories since the previous step)
wget https​://yt-dl.org/downloads/2019.04.17/youtube-dl-2019.04.17.tar.gz (but use the current link from the download page)

Optional, but highly recommended:  Check the SHA256 checksum to make sure it matches the one shown on the download page:

shasum -a 256 youtube-dl-2019.04.17.tar.gz

Then uncompress the file using tar xvzf:

tar xvzf youtube-dl-2019.04.17.tar.gz

The next step is to find the file __main__.py – this is the file you will actually run when using the source code (note that where you see __ in this article, that’s really two underscore characters side by side, even if the gap between them appears nonexistent). If you have followed the directions so far, it will probably be in the ~/ytdl/youtube-dl/youtube_dl directory, unless they change the directory tree in a future version. So try this:

~/ytdl/youtube-dl/youtube_dl/__main__.py --version

That should show you the version number of youtube-dl that you are running.

Now you should be able to use the same string to invoke the source code version of youtube-dl. The format is:
~/ytdl/youtube-dl/youtube_dl/__main__.py [options] [url]

Now scroll down to “Using aliases to avoid excessive typing” and continue from there. Where you see [path to __main__.py], substitute that with /home/username/ytdl/youtube-dl/youtube_dl/__main__.py but replace username with the name of your user directory (also, MacOS users should replace “home” with “Users”).

If you want the (smaller) source code only version, do this: In a web browser go to the youtube-dl GitHub Page and click the green “Clone or Download” button, then right click on the “Download Zip” button in the dropdown that appears and copy the link:

Location of Download Zip button

Then in a terminal window use wget to get the file. The file should be downloaded to or placed in the directory you just created. For example:

cd ~/ytdl (only necessary if you have changed directories since the previous step)
wget https://github.com/ytdl-org/youtube-dl/archive/master.zip

Then uncompress the file using unzip:

unzip master.zip

The next step is to find the file __main__.py – this is the file you will actually run when using the source code (note that where you see __ in this article, that’s really two underscore characters side by side, even if the gap between them appears nonexistent). If you have followed the directions so far, it will probably be in the ~/ytdl/youtube-dl-master/youtube_dl directory, unless they change the directory tree in a future version. So try this:

~/ytdl/youtube-dl-master/youtube_dl/__main__.py --version

That should show you the version number of youtube-dl that you are running.

Now you should be able to use the same string to invoke the source code version of youtube-dl. The format is:
~/ytdl/youtube-dl-master/youtube_dl/__main__.py [options] [url]

Where you see [path to __main__.py] below, substitute that with /home/username/ytdl/youtube-dl-master/youtube_dl/__main__.py but replace username with the name of your user directory (also, MacOS users should replace “home” with “Users”).

Using aliases to avoid excessive typing

Perhaps you don’t want to have to remember that long string, or type it out. In that case, open your .bashrc file in your home directory (assuming you use bash as your shell) and add a line like this:

alias ytdl='[path to __main__.py]'

Remember to make the correct substitution for [path to __main__.py], depending on which version you downloaded. After logging out and back in, you should be able to just use ytdl in place of ~/ytdl/youtube-dl/youtube_dl/__main__.py or ~/ytdl/youtube-dl-master/youtube_dl/__main__.py.

You can also make alaiases that invoke frequently-used options.  For example, if in some cases a file seems to be coming in normally and then all of a sudden you get a series of “Non-monotonous DTS in output stream” errors, you may find that you need to use the --hls-prefer-native option to get a good file. If you find you are doing this a lot, you could add an alias such as this:

alias ytdln='[path to __main__.py] --hls-prefer-native'

The above is all on one line, even if it shows as two or more lines in your display.  Then you would use ytdln to include that option. You can create one or more aliases that include just about any frequently used option(s) in this way.

Please keep in mind that any time you upgrade to a newer version of youtube-dl, you will need to follow this procedure again – you can’t just use youtube -U because that would probably get the binary version. And be careful; if you make any changes and then later on install a new version over the top of them, those changes will be lost. Finally, please note that if you are ever trying to download patches or even entire modules from GitHub, make sure you get the “Raw” versionGitHub "Raw" button(use the button at the top right of the code block) or you will get the html version, which is almost certainly not what you want.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.