Thursday 24 April 2014

ffmpeg not creating previews in Newznab

Had an issue on my system where ffmpeg latest version installed via brew (2.2.1) yet I couldn't get any previews on media.

Much head scratching later, it turns out that because my ffmpeg install is so new newznab wasn't configured to use it.

So, you need to edit postprocess.php & add the following line (//2.x):


Make sure you comment out the other lines as above, its near the bottom of the file! :-)

Wednesday 23 April 2014

Running Newznab on OSX Mac Mini Mavericks Server using XAMPP

After using an UBUNTU VM I switched to running newznab on a pure OSX install (Mavericks Server), due to the issues I had with the VM crashing on occasion...plus I thought the extra resources it consumed made sense to have it running direct.

Installation was very straightforward, using the below:

1. I decided to use an XAMPP installation while testing this setup but am soon switching over to using the Mavericks Server setup (simply as it one less point of failure...and why not!)

2. Get the newznab install - plus edition is a no-brainer as regular regexes mean lots of content :-)
http://www.newznab.com/

3. Install XAMPP http://www.apachefriends.org/en/xampp-macosx.html, its a simple drag to Applications

4. Install newznab, svn install with site details from newznab...the usual! Place the install into the www folder of XAMPP

5. In XAMPP control panel start:

MySQL
Apache web server

Once they have a green light next to them you can open Safari web browser and navigate to:

localhost or 127.0.0.1

And you should see the welcome screen:


Congrats, so far so good! now shutdown both the MySQL & Apache by clicking stop & wait till they turn red.

6. Configure XAMPP:

Open Applications/XAMPP/etc/httpd.conf

Best to edit using some like Textwranger

You need to edit around line 180, this is to tell your web server to send any web requests to newznab by default....you can use Apache to host multiple sites quite easily but I will discuss this in another post.

DocumentRoot "/Applications/XAMPP/xamppfiles/htdocs/nnplus/www/"

Make sure your changes are exactly as above!

Close & save the file, then restart MySQL & Apache in XAMPP control panel.

Open Safari and type localhost into address bar - you should now see newznab but with an error...don't worry you're doing good!

7. Configure PHP

Open Applications/XAMPP/etc/php.ini


The max_execution_time needs to be increased – suggest 120
The memory_limit needs to be increased – suggest 1024M

Save the file.

8. Complete Newznab install, set correct folder permissions:

Open a terminal session and copy/paste each line below, pressing enter after each paste to set permissions

chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www/lib/smarty/templates_c
chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www/covers/movies
chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www/covers/anime
chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www/covers/music
chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www
chmod 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/www/install
chmod -R 777 /Applications/XAMPP/xamppfiles/htdocs/nnplus/nzbfiles/


In the XAMPP control panel, ensure MySQL & Apache lights are green.

Open Safari & goto localhost (127.0.0.1) which should now show the newznab install screen with no errors, click the button "Go to step: Pre Flight Check"

Long  as you followed steps 6 & 7 you will then be error free and click "Go to step two: Set up the database"

You will see


Default user is root and password blank unless you have changed it in XAMPP

Click "setup database"

It will setup the database and present you with a screen telling you the database setup is correct

Click the button showing “Step three: Setup News Server connection”

You should have a newsgroup provider setup and have the login/connection details for it – type them in here and click test connection.

You should receive a message telling you “The news server setup is correct…..”

Click the button stating ”Step four: Cache Settings”

Click “Save configuration file”

Click the button marked “Step 5: setup admin user”

Fill in this section and then click the button.

Then click the button marked “Step Seven: Set NZB File Path”

Use the default path and so long as you have correctly entered all the chmod commands presented earlier in step 8 you should get no errors – if you get an error follow the instructions in the error.

Otherwise click the button marked “Set NZB File Path”

Install complete, now click on the admin home page link to begin the admin setup.

9. Admin Setup Newznab

Click on the site options link.

Paste your newznab ID from your welcome email into the last field of the first section.

If your newsgroup provide supports compressed headers (not many do) in the section marked Usenet Settings select the Use compressed headers toggle – do not select if not supported!

Change Max Messages to 60,000 I find this runs pretty smooth but you can experiment with higher or lower figures (I did try @ 100,000 but my system kept barfing...could need a MySQL tweak on memory settings as my mac has 16GB sweeeet :-))

In the Where to start new groups field I change the toggle box to days and enter 1 day (initially it will go back one day to get all the headers – that is enough to start off your site) – if you leave the 20000 posts in there then you will not have enough posts to complete an episode or MP3.

Then scroll to the bottom and click “Save site settings”

10. Select Newsgroups to index

I personally use these, they seem to grab enough releases and things move fast enough..pick or leave em at your leisure!

a.b.x264
a.b.tvseries
a.b.tv
a.b.town
a.b.teevee
a.b.multimedia
a.b.movies.xvid
a.b.movies.divx
a.b.movies
a.b.moovee
a.b.mom
a.b.hdtv.x264
a.b.hdtv
a.b.ghosts
a.b.ebook
a.b.e-book.technical
a.b.e-book.magazines
a.b.e-book.flood
a.b.e-book
a.b.comics.dcp
a.b.astronomy
a.b.0day.stuffz

11. Get things running!

So there are two files that need to be run, 1 will grab releases & the other will preform all the updates that turn the files into the nice content we are looking for!

Open a terminal session

cd /Applications/xampp/xamppfiles/htdocs/nnplus/misc/update_scripts
/applications/xampp/xamppfiles/bin/php-5.3.1 update_binaries.php

The above looks for headers and starts to process them based on regexes, once this has completed you need to run the following

cd /Applications/xampp/xamppfiles/htdocs/nnplus/misc/update_scripts
/applications/xampp/xamppfiles/bin/php-5.3.1 update_releases.php

This turns the millions of headers into releases and populates all the sections in your Newznab (Movies, TV, Books etc....)

12. Making Life Easy!

The best way to run the above though is through an automated script such as the following:

In Applications/XAMPP/xampp/htdocs/nnplus/misc/update_scripts/nix_scripts

You should have a file newznab_local.sh - copy this file copy newznab.local.sh newznab_update.sh

Then edit the newznab_update.sh file in Textwranger to look like:

#!/bin/sh
# call this script from within screen to get binaries, processes releases and 
# every half day get tv/theatre info and optimise the database

set -e

export NEWZNAB_PATH="/Applications/XAMPP/xamppfiles/htdocs/nnplus/misc/update_scripts"
export NEWZNAB_SLEEP_TIME="600" # in seconds
export PHP_PATH="/Applications/XAMPP/xamppfiles/bin"
LASTOPTIMIZE=`date +%s`

while :

 do
CURRTIME=`date +%s`
cd ${NEWZNAB_PATH}
${PHP_PATH}/php-5.5.3 ${NEWZNAB_PATH}/update_binaries.php
${PHP_PATH}/php-5.5.3 ${NEWZNAB_PATH}/update_releases.php

DIFF=$(($CURRTIME-$LASTOPTIMIZE))
if [ "$DIFF" -gt 43200 ] || [ "$DIFF" -lt 1 ]
then
   LASTOPTIMIZE=`date +%s`
   ${PHP_PATH}/php-5.5.3 ${NEWZNAB_PATH}/optimise_db.php
   ${PHP_PATH}/php-5.5.3 ${NEWZNAB_PATH}/update_tvschedule.php
   ${PHP_PATH}/php-5.5.3 ${NEWZNAB_PATH}/update_theaters.php
fi

echo "waiting ${NEWZNAB_SLEEP_TIME} seconds..."
sleep ${NEWZNAB_SLEEP_TIME}

done

Save the file

Run the file ./newznab_local.sh

This will then run quite happily and process releases, you do need to either leave the terminal window open so it runs or use something like screen to have it running as a process - something I will discuss in another post.