Definitive Guide to Setup Newznab Plus on Ubuntu Server 12.10

Due to struggling myself, I created this guide with all the updates to enable an easy install of Newznab + on Ubuntu Server 12.10 as quite a few things have changed over the official guide.

If you're interested (and I hope you are!) I am using a virtual install of Ubuntu using 1GB memory & 1 core on a Mac Mini, things running great and about to do a backfill....

The install as below has been running for about a week and filling up nicely, please comment if you need any help!

**Revision 8** updated ref some helpful comments

3.1 Newznab on Ubuntu 12.10

This guide covers installing Newznab+ on a fresh copy of Ubuntu 12.10 64-bit server and assumes a basic knowledge of using Linux via the command line.

3.1.1 Install Ubuntu

Always login as the root user, any other user will get permissions denied left, right & center!!!

Grab a copy of Ubuntu from their website and install it. Once the install is complete, log in, fire up a terminal window, do a full upgrade and reboot (entering your password when prompted):

sudo apt-get update

sudo apt-get -y upgrade

sudo apt-get -y dist-upgrade

sudo reboot

You’ll probably want to install SSH if you don’t have it already:

sudo apt-get install -y ssh

Additionally, we’re going to be compiling some code from source, so we’ll need the tools to do so:

sudo apt-get install build-essential checkinstall

3.1.2 Install Newznab

Once Ubuntu is installed and updated, we can begin installing Newznab. Before we get started though, we need to decide where to install Newznab. Typically, web-related stuff goes in /var/www/, so lets put Newznab in /var/www/newznab. Let’s create the directory and set it writeable by our user for now:

sudo mkdir -p /var/www/newznab

sudo chmod 777 /var/www/newznab

Now we can begin installing the prerequisites.

Prerequisites

Newznab as a few dependencies, so let’s start with installing the required software. According to the install docs, Newznab needs the following software:

1. PHP 5.2+

2. GD Imaging Library w/PHP integration

3. PEAR

4. Database: MySQL (or Percona)

5. Web Server: Apache (or Nginx)

Its worth mentioning a few things at this point. The first is that the default version of MySQL that comes with Ubuntu is kind of old (version 5.1). Therefore, we’ll also cover installing a higher-performance version (Percona version 5.5) instead. Additionally, Apache is listed as a requirement; however it is possible to use a different web server instead. Therefore, we’ll go over how to use the blazing-fast Nginx as well for those who don’t want to use Apache.

The following software is optional, but we’ll also cover installing it and setting it up:

1. Unrar

2. FFmpeg

3. Lame

4. MediaInfo

5. Sphinx

By the end of this guide we’ll have a fully working Newznab+ install with all optional components working as well.

PHP

Let’s start by installing PHP and required PHP extensions:

sudo apt-get install -y php5 php5-dev php-pear php5-gd php5-mysql php5-curl

The install docs also say that PHP needs a few tweaks. Using your favorite text-editor (I’m using nano) open up php.ini:

sudo nano /etc/php/cli/php.ini

Look for max_execution_time, and set it to (2 minutes):

max_execution_time = 120

Next, look for memory_limit, and either set it to 256MB, or -1 for no limit:

memory_limit = 256M

While you’re here, you should also set PHP’s date.timezone (a list of available timezone settings can be found here):

date.timezone = Europe/London (or whatever your timezome!)

That should do it for PHP for the time being (we’ll come back to it when we configure the web server).

Database

You have a few options here, we’ll cover two:

1. Use MySQL 5.1

2. Use Percona 5.5

I’d recommend that you use Percona, but the choice is yours. Pick one and follow one of the sections below (do not install MySQL and Percona).

MySQL 5.1

If you decided to use MySQL, simply install it via aptitude:

sudo apt-get install mysql-server-5.1 mysql-client-5.1 libmysqlclient-dev

Percona 5.5

Installing Percona requires a little extra work upfront, but in the long run it is generally worth it as Percona is a high-tuned fork of MySQL. The first step is to add Percona into aptitude:

gpg --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A

gpg -a --export CD2EFD2A | sudo apt-key add –

That retrieves and installs the GPG keys needed to add Percona’s aptitude repository. Now we need to tell aptitude about the repositories:

sudo sh -c "echo \"\n#Percona\" >> /etc/apt/sources.list"

sudo sh -c "echo \"deb http://repo.percona.com/apt lenny main\" >> /etc/apt/sources.list"

sudo sh -c "echo \"deb-src http://repo.percona.com/apt lenny main\" >> /etc/apt/sources.list"

Now update and install it:

sudo apt-get update

sudo apt-get install -y percona-server-client-5.5 \

percona-server-server-5.5 \

libmysqlclient-dev

It’ll ask for a password to use for the MySQL root user, so pick one and remember it.

Web Server

OK, now for the web server. Once again, you have multiple choices and we’ll cover two:

1. Apache

2. Nginx

Apache is “easier” and generally more flexible, but it also tends to be a system hog. So, if you can’t make up your mind it is probably best to take the easy route and install Apache. If you feel like squeezing the most out of your machine and you are a little more skilled with configuring software, Nginx might be a good choice. Again, pick one and follow the steps below (don’t install both).

****Please note that if Apache is installed by default you will have to remove before installing Nginx, due to port 80 conflicts****

Apache

First install it:

sudo apt-get install -y apache2

You’ll also need to configure the php.ini for Apache, so follow the section above about configuring PHP (from above), but this time, edit the file /etc/php5/apache2/php.ini:

sudo nano /etc/php5/apache2/php.ini

Let’s create the site configuration file for Apache. Open up a new file:

sudo nano /etc/apache2/sites-available/newznab

Here is a template you can use for this file. You should change settings where appropriate for your setup:

ServerAdmin webmaster@localhost

ServerName localhost # You might want to change this to point to your server IP

# These paths should be fine

DocumentRoot /var/www/newznab/www

ErrorLog /var/log/apache2/error.log

LogLevel warn

Now we need to disable the default Apache settings, enable the one we just created for Newznab and enable modrewrite:

sudo a2dissite default

sudo a2ensite newznab

sudo a2enmod rewrite

sudo service apache2 restart

Nginx

First install it:

sudo apt-get install -y nginx

Now, for Nginx there are multiple ways to serve PHP files. Probably the best is php-fpm, which is basically a daemon that runs and serves PHP to Nginx. So, let’s go ahead and install that now:

sudo apt-get install -y php5-fpm

You’ll also need to configure the php.ini for FPM, so follow the section above about configuring PHP (from above), but this time, edit the file /etc/php5/fpm/php.ini:

sudo nano /etc/php5/fpm/php.ini

Just for good measure, restart the daemon:

sudo /etc/init.d/php5-fpm restart

Let’s create the configuration file for Nginx. Open a new file:

sudo nano /etc/nginx/sites-available/newznab

Here is a template you can use for this file. You should change settings where appropriate for your setup:

server {

# Change these settings to match your machine

listen 80 default_server;

server_name localhost;

# Everything below here doesn’t need to be changed

access_log /var/log/nginx/access.log;

error_log /var/log/nginx/error.log;

root /var/www/newznab/www/;

index index.html index.htm index.php;

location ~* \.(?:ico|css|js|gif|inc|txt|gz|xml|png|jpe?g) {

expires max;

add_header Pragma public;

add_header Cache-Control "public, must-revalidate, proxy-revalidate";

}

location / { try_files $uri $uri/ @rewrites; }

location @rewrites {

rewrite ^/([^/\.]+)/([^/]+)/([^/]+)/? /index.php?page=$1&id=$2&subpage=$3 last;

rewrite ^/([^/\.]+)/([^/]+)/?$ /index.php?page=$1&id=$2 last;

rewrite ^/([^/\.]+)/?$ /index.php?page=$1 last;

}

location /admin { }

location /install { }

location ~ \.php$ {

include /etc/nginx/fastcgi_params;

fastcgi_pass 127.0.0.1:9000;

# The next two lines should go in your fastcgi_params

fastcgi_index index.php;

fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;

}

}

Let’s make sure that the Nginx logs directory exists and is writeable:

sudo mkdir -p /var/log/nginx

sudo chmod 755 /var/log/nginx

Now, let’s disable the default Nginx site handler and enable our newznab configuration:

sudo unlink /etc/nginx/sites-enabled/default

sudo ln -s /etc/nginx/sites-available/newznab /etc/nginx/sites-enabled/newznab

Finally, let’s restart Nginx:

sudo service nginx restart

Extras

Lame is used for processing audio samples

Unrar is used to extract files from releases
MediaInfo is used to gain information about various types of media files found in releases

sudo apt-get install software-properties-common

apt-get install python-software-properties

add-apt-repository ppa:jon-severinsson/ffmpeg

add-apt-repository ppa:shiki/mediainfo && apt-get update && apt-get install ffmpeg x264 mediainfo unrar lame

Sphinx

Sphinx is used for full-text searching. It is insanely fast and if you really want your Newznab+ install to fly, it is highly recommended. Unfortunately the version in Ubuntu’s aptitude repository is horribly old, so we’ll need to build a newer version. Let’s download the source and extract it:

wget http://sphinxsearch.com/files/sphinx-2.0.6-release.tar.gz

tar xvfz sphinx-2.0.6-release.tar.gz

cd sphinx-2.0.6-release

Download and extract libstemmer_c:

wet http://snowball.tartarus.org/dist/libstemmer_c.tgz

tar --strip-components=1 -zxf libstemmer_c.tgz -C libstemmer_c

Configure it:

./configure --prefix=/usr/local --with-libstemmer

Now we’re ready to compile Sphinx. For this step you can speed up the compilation on a multi-core system. If you have a 4-core system, for example, you can do (replace j4 with the number of cores your machine has):

make -j4

Once that finally finished, install it:

sudo checkinstall --pkgname=sphinx --pkgversion="2.0.6-release" --backup=no --deldoc=yes --fstrans=no –default

Now we have a nice new version of Sphinx installed in /usr/local. Binaries are installed in /usr/local/bin.

Newznab Source

Finally, we can now begin installing Newznab! We’ll be grabbing the latest and greatest version, so we’ll need Subversion installed first:

sudo apt-get install -y subversion

Now we can check out the Newznab code:

svn co svn://svn.newznab.com/nn/branches/nnplus /var/www/newznab

(you will need your newznab plus login details here!)

At this point we might as well set the permissions on a couple of directories as well:

sudo chmod 777 /var/www/newznab/www/lib/smarty/templates_c

sudo chmod 777 /var/www/newznab/www/covers/movies

sudo chmod 777 /var/www/newznab/www/covers/anime

sudo chmod 777 /var/www/newznab/www/covers/music

sudo chmod 777 /var/www/newznab/www

sudo chmod 777 /var/www/newznab/www/install

sudo chmod 777 /var/www/newznab/nzbfiles/

Now that Newznab is installed, we need to configure it. Configuration is done via a web browser.

Run Installer

It is now time to configure Newznab. This is done via a web-based installer. Open up

http://localhost/install in a web browser (or whatever the address/IP address is of your server)

and follow the guided steps.

Enable Groups

Head over to /admin/group-list.php in your web browser and pick some groups to index by clicking “activate” on a few groups.

Set Paths and Options

Make sure to set your newznab ID.

We need to let Newznab know where all the extra software is that we installed earlier, so head over to

/admin/site-edit.php in your browser, scroll down to the “3rd Party Application Paths” section and update the fields:

Unrar Path /usr/bin/unrar

Mediainfo Path /usr/bin/mediainfo

Ffmpeg Path /usr//bin/ffmpeg

Lame Path /usr/bin/lame

If you’d like to enable audio previews, check Save audio preview. It is worthwhile to do rar checking, so set Check For Passworded Releases to Deep.

That’s it for configuration right now, but don’t close your browser yet as we’ll be coming back to the configuration page when configuring Sphinx.

Back at the command line its time to fire up the binaries and releases update scripts:

cd /var/www/newznab/misc/update_scripts

The update_scripts folder contains a lot of scripts. The most important ones are update_binaries.php

and update_releases.php. If you have any experience with screen or tmux, it is highly recommended that you use one of these to run the update scripts, as it will allow you to monitor the update process, observe and resolve issues; this is especially important for newcomers to Newznab. With that said, Newznab also ships with an init-sytle script that can be installed to make Newznab run more or less as a daemon that will start and stop with startup and shutdown, respectively.

Screen or tmux

If you want to go the screen or tmux route, you’ll need to pick one and install it:

# Install screen...

sudo apt-get install -y screen

# ...or tmux

sudo apt-get install -y tmux

In the nix_scripts directory there is a useful script called newznab_screen.sh that runs

update_binaries.php and update_releases.php, in addition to a few other scripts, continuously

and automatically. First, we need to modify it however, so lets change dir and make a copy:

cd /var/www/newznab/misc/update_scripts/nix_scripts

cp newznab_screen.sh newznab_screen_local.sh

Now open newznab_screen_local.sh in a text editor and modify NEWZNAB_PATH near the top to point to our installation path:

nano newznab_screen_local.sh

Set NEWZNAB_PATH:

export NEWZNAB_PATH="/var/www/newznab/misc/update_scripts"

Ensure correct permissions:

chmod +x newznab_screen_local.sh

Now we can run the script via screen:

screen bash

./newznab_screen_local.sh

You should see the script download headers for the groups that you have enabled and then run various stages that will attempt to group and catalogue the headers. For now, just leave the script running and detach from screen by typing cntl a d.

Sphinx

As mentioned previously, Sphinx is a fast full-text indexer. By default, it is disabled in Newznab, so go ahead and enable it by visiting /admin/site-edit.php and setting “Use Sphinx” to “Yes”. While there, you’ll also notice that there are a few other configuration options for Sphinx. By default, Sphinx will index all of the release information, however, there are three other optional indexes: NZB contents, NFO contents and release files. Enabling these optional indexes will add increased processing time, so you will likely want to experiment to see what combination works best for your hardware. For now, you don’t have to enable any of the optional indexes.

To get Sphinx running, we need to generate a sphinx.conf file. To do this we’ll use the nnindexer.php script in misc/sphinx:

cd /var/www/newznab/misc/sphinx

./nnindexer generate

The script will print out the location of the sphinx.conf file, which by default will be

/var/www/newznab/sphinx/sphinx.conf. This path needs to be entered into the “Sphinx Configuration

Path” setting located at /admin/site-edit.php.

Now we need to start the search daemon and create the indexes and restart the daemon:

./nnindexer.php daemon

./nnindexer.php index full all

./nnindexer.php index delta all

./nnindexer.php daemon --stop

./nnindexer.php daemon

Sphinx in Newznab admin site (my recommended) settings:

Set use sphinx to yes
index nzb contents to no
index nfo contents yes
index releasefiles yes
index predb no

3.1.6 Summary of Installed Software

• PHP v5.3.6

• Pear v1.9.2

• MySQL v5.5 or

• Percona v5.5.17

• Apache v2.2.20 or

• Nginx 1.0.5

• FFmpeg v0.8.7

• MediaInfo v0.7.50

• Lame v3.98.4

• unrar v4.00-beta3

• Sphinx 2.0.4-release

To Update your Newznab installation:

1. Run svn update (making sure you are in the correct folder /var/www/newznab)
2. Run any new patches since your last update located in /db/patch/0.2.3/date_patchname.sql
3. Delete contents of cached smarty files in /www/lib/smarty/templates_c/*