Finding Disk Usage of Linux Sub Directories Sorted By Size

 Hosting, Linux, On The Front Line  No Responses »
Sep 012011
 

Cpanel configuration enables you to receive warning when disk space is low. You can instruct cPanel to handle house-keeping duties such as log rotation but this is not a guarantee disk space will not be filled by other activity logs.

I have used the following bash script for years but as disk space is less of a problem I have not needed it for a while so I am posting it here as a reminder for the next time I need it.

To use it paste into a file called duf.sh and change the permissions to allow it to execute.

#!/bin/bash
echo sample usage: /root/duf.sh /usr/\*
du -sk "$@" | sort -n | while read size fname; do for unit in k M G T P E Z Y; do if [ $size -lt 1024 ]; then echo -e "${size}${unit}\t${fname}"; break; fi; size=$((size/1024)); done; done

Run it like this:-

duf.sh /usr/*

And it would return a listing like this:-
how to list linux directories sorted by their size Finding Disk Usage of Linux Sub Directories Sorted By Size

I have the largest folders at the end of the listing to make them easy to find.

 Posted by Martyn at 1:12 pm  Tagged with: , , , ,

Invalid argument supplied for foreach in mysqli.php on line 391

 Hosting, Joomla, PHP  No Responses »
Aug 212011
 

This is a note to self.  When you see Warning: Invalid argument supplied for foreach() in /home/yourdom/public_html/libraries/joomla/database/database/mysqli.php on line 391 it means the host has run out of disk space on the tmp folder used for sessions and other temporary files.  Specifically Joomla is trying to create a temporary table and it appears it gets as far as creating the pointer and php/mysql errors at the point of reading the data rather than at the point of adding data to the table.

So if you are not me and you have this error then the solution is to ask your host to clear out the /tmp folder and increase the amount of disk space allocated to it so it doesn’t happen again.

 Posted by Martyn at 10:44 am  Tagged with: , , ,

Are you happy with your backup strategy?

 Hosting, On The Front Line, Programming, Software  No Responses »
Aug 152010
 

whs 300x211 Are you happy with your backup strategy?As a software developer with hosting development and management experience its not unusual for me to be asked to help someone who has lost a file. There are three places a backup can sometimes be found:-

1. There is a proper backup strategy in place.
2. An adhoc backup may include a copy, to find these think outside the box, did you zip it up and email it to someone?
3. A deleted file can be restored or pieced together from file cache or fragments not yet overwritten.

I am surprised at the number of hosts that do not perform backups. Some will offer extra disk space so you can do it yourself for a fee, some will handle backups for you for a fee, but few do it automatically. The rule has to be If you do not maintain backups then its unlikely anyone else will.

Relying on your programmers has some merit but you should not rely on it. Programmers don’t like to write their code twice and so we find a way to automate the process to ensure we have the best possible solution.

The best backup strategy
Any backup strategy is better than none but if you want an easy to use and cost effective solution to backup several PCs/Macs/Mobiles as well as your hosted applications then check out Microsofts Home Server. It requires a separate PC to operate but nothing special. This is one of those applications you hear little about. It’s not expensive and is really good at what it claims to do (see this link for the sales pitch).

Expanding the amount of disk space available is as simple as plugging in a new external USB drive. So you can dig out those discarded drives and use them as additional space. Home server not only keeps a backup but you can use it to store important items such as photos and videos and it will make duplicate copies, so if a hard disk fails all you need to do is throw in another and it will take care of duplicating the files again for you.

Access to TV, music and all you media is available 24/7. Connect it to your wirelss router and the process of sharing your media around the house really does become straightforward.

Backup for business
I have a few offsite Linux servers, some more than 4,000 miles away. Home server backs these up every day and retains as many versions of those files as you think is convenient. By default you can recover any file up to 3 months old but I have extended this to 12.

 Posted by Martyn at 2:40 pm  Tagged with:

WHMCS Support Monitoring System Tray Application

 CSS, Hosting, Javascript, MySQL, On The Front Line, PHP, Programming, Tech  1 Response »
Dec 302009
 

Sales and support monitoring tool for WHMCS

A growing sales and support requirement inevitably leads to more hands on keyboards which is also the most expensive side of technology, so anything that can help better utilise the human resources already available can be a godsend.

whmcs monitoring tool WHMCS Support Monitoring System Tray Application

WHMCS support module has matured to something very usable, with two caveats.  First it seems a shame the authors have not heeded the calls for the ability to send rich media replies and thus incorporate videos and images in predefined replies.  Their excuse that you can send a link to those if you wish is nothing short of lame, even worse they site “security issues” as the reason when the real reason is surely they just cannot be bothered (after all, if they have customers that agree with them, how difficult would it be to supply the solution as an option).

But I’m not here to rip them a new one, their latest release is truly fantastic and better than any other host billing system I have used, it makes Modern Billing look tedious and infezible (which indeed it is).  Utilising their API and the includes/hooks folder enables unlimited customisation, indeed I have bent it to supplying electronic products and license management with great results.

The second caveat is the lack of an alert system.  Up until recently I used a very old Kayako support system but the balance of power is back in WHMCS court except that Kayako had a simple “flashing beeping” thingy that would notify you when a new ticket or customer reply had been received.  It was a basic product and rather featureless yet it did an important job and as I missed it so much decided to write one for WHMCS.

download WHMCS Support Monitoring System Tray Application

Ticket Review (setup.msi) Windows Setup MSI, download and double click.

Ticket Review (manual) Zip file contains program EXE, a php file and readme.txt with instructions for manual installation.

Installation Instructions

Ticket Review monitors up to two support departments in WHMCS and alerts you of a new ticket by changing the color of the system tray icon and optionally playing a wav file too.  Ticket Review can be used by an unlimited number of sales and support staff.

1.  Unzip and place files in a directory.
2.  Upload ticketreview.php to your WHMCS install direcory on your host webserver.

Configuration

To configure your application double click the Ticket Review.exe  When you first run the application you need to complete some of the fields.  The Tickets Link is the URL to your WHMCS and will look something like this:-

http://yourdomain.com/client/ticketreview.php

If you only have one support ticket department you need only complete the Red Department Id.  We operate two departments and allocate the Red to Sales and the Green to Support.

How to find the Department Id

Login to WHMCS as admin, the select Support Departments from the Setup dropdown.  Hover your mouse over the edit icon and a link is displayed in the status bar, at the end of the link there is &id= the number following this is your department id.

How to add sound

Sometimes it is useful to add sound, if you don’t want it just leave the WAV fields blank.  Google for fancy wavs to produce bings or bongs or whatever takes your fancy, I prefer to hear a voice telling me whether a sales or support ticket is in and use ATT’s Text to Speech demo tool to create a WAV which can be found here:-

http://www2.research.att.com/~ttsweb/tts/demo.php

What next?

After setup you should create a couple of test tickets to make sure it all works.  If it doesn’t check the link to the PHP file is correct and try running it directly in your browser to make sure there are no errors.

What is supposed to happen?

When a new sales or support ticket arrives the icon will begin to flash.  If you have added a wav that will be played each time the application checks and finds a ticket that needs your attention.  This will probably get a little annoying after a while so you can delay the check by 5 minutes by double clicking the system tray icon (which will also stop flashing).  The idea is this application will bug you until you answer your ticket!  But you can delay that nuisance for 5 minutes as often as you like!

Support

This is a simple application and shouldn’t really need support.  You can probably think of hundreds of ways in which to improve the application and while I do not wish to steal your thunder I have probably thought of them too, this application is intended to just alert you of new tickets, stats and integrating with facetwitter are all cool ideas but they don’t really improve it.  However if you need a contractor to write additional features then please contact me at mwdev@ventrino.com or call UK +447850018933 and ask for Martyn

 Posted by Martyn at 5:56 pm  Tagged with: , ,

ASP code to block multiple ranges of IP's from your site

 CSS, Hosting, Javascript, MySQL, On The Front Line, PHP, Programming, Tech  No Responses »
Apr 242008
 

It wasn’t long before I needed to expand upon banning a few IP’s. I needed to be able to ban whole networks so here’s the Q&D solution:-

First create a text file called ipsec.txt and enter some IP’s you wish to ban, to ban a network just leave off the end of the IP class number, for example:-

10.20.30.40
10.50

In the above example the first is a specific IP that you wish to ban, the second will ban all IP addresses that start 10.50 (for example 10.50.20.1 and 10.50.100.1 will both be banned). Enter as many as you like, one per line.

Now create or edit your global.asa file and add the following code to the session_onstart sub procedure:

sub session_onstart
  remote_ip = request.servervariables("Remote_Addr")
  ipsec     = server.mappath("/ipsec.txt")
  ips       = getFileContents(ipsec)
  ips       = split(ips,vbcrlf)
 
  for each ip in ips
    if len(trim(ip)) > 0 then
      ip_parts        = split(ip,".")
      remote_ip_parts = split(remote_ip,".")
      found = true
      for x = 0 to ubound(ip_parts)
        if ip_parts(x) <> remote_ip_parts(x) then
          found = false
          exit for
        end if
      next
      If found then
        response.redirect "/redir.html?ip=" & remote_ip
      end if
    end if
  Next
end sub

Replace /redir.html with a file or location you want to redirect banned ip’s too.

As this code runs in the session_onstart section of the global.asa it will be run only once for each visitor, this means that during the session they could return and this code would be bypased. If that is a concern modify the code to be a common function and place it strategically, perhaps in a common file called throughout the website.

 Posted by Martyn at 6:51 am  Tagged with: , ,

Write ASP and need a way to block IP's from your site?

 CSS, Hosting, Javascript, MySQL, On The Front Line, PHP, Programming, Tech  No Responses »
Apr 222008
 

This demonstrates how to ban a single IP address, later, I’ll show how to ban whole networks but chances are you will be able to work that out for yourself anyway after reading this anyway.

If global.asa does not exist then create it and add the following:

sub session_onstart
 
  ip = request.servervariables("Remote_Addr")
 
  select case ip
    case "111.111.111.111", "111.111.111.112", "111.111.111.123"
      response.redirect "http://www.example.com"
  end select
 
end sub

Replace 111.111.111.11x with the IP addresses you wish to ban, note they are comma separated and the last one is not followed with a comma.

You can change the redirection to a page on your site that informs them they are no longer allowed (a bit vindictive), to a blank page is probably best or you could even forward them on to a competitor :D

 Posted by Martyn at 3:24 am  Tagged with: , ,
© 2011 Martyn Walker | Software Architect | Hiker And Hacker Suffusion theme by Sayontan Sinha