Be SMART, get advance warning that your hard drive is starting to fail

All modern hard drives support S.M.A.R.T. in which the hard drive (HD) runs self tests and reports on the status of internal metrics (seek errors, block errors, temperature, being dropped, etc). If a HD starts to do poorly in some of the internal metrics, it's one sign the drive might be starting to fail. Unfortunately predicting when the drive will actually failure is impossible right now, akin to predicting earthquakes. But it does give you some advance warning to redouble the backup efforts or to switch out that drive if it holds valuable data.

There exist many tools to access the SMART interface depending on your OS. The key thing is that the HD itself does all the work. The tools just provide access to the HD.

The beauty of SMART is that

• you can run the SMART tests on a running system on a mounted HD in use! Since the HD does the SMART scheduling internally, it can figure out when and how to continue with it's own test while getting normal OS requests.
• it's pretty easy to do a manual scan. For Linux, download the smartctl software and then start issuing commands. There's a lot of documentation on the web.
• with a bit of work you can setup regular SMART background tests and have alerts sent when a HD falls below the built in "failure" thresholds.

The main drawbacks are

• a thorough "long" test takes several hours. On a 1.5TB drive, it takes 4-8 hours.
• you need to have the HD hooked up directly via its native interface, namely PATA or SATA. If a drive is hooked up via a USB enclosure, smartctl will claim the drive does not support SMART. Ugh. (I assume eSATA will work, as this is fundamentally SATA).

I'm not going to repeat all the information out there. But here's some decent links.

• A great article on SMART geared toward Linux.
• The wikipedia article on SMART, with many more links.

System tools in Linux and Unix

These keep changing over the years, but here's a quick run down

Hardware related:

ethtool, mii-tool: view / set ethernet device settings and the MII settings

lspci: show PCI devices

lshw: show hardware

lscpu, lshal, lsusb: show CPU, HAL and USB information. Note the USB information is often the USB controllers on your M/B, not the devices actually hooked up.

Linux Kernel

lsmod , rmmod, modprobe: manage Linux kernel modules (which how most device drivers are loaded)

lsof, fuser: show processes using files/directories/file systems. Very useful if you want to unmount a disk, say a removable drive, and you are told the device is still in use. Note that if you run the samba SMB/CIFS file server, it will continue to use a filesystem if at anytime in the past a client used that file system. I restart samba in this case.

mount, umount: mount and unmount filesystems. A trick I often use is to remount a filesystem as read-only or read-write via

mount -o remount,ro /dev/... 

or 

mount -o remount,rw /dev/...

which is much faster than unmounting and remounting.

Changing the DHCP (or any other service) config file in Ubuntu 9.10

This applies to Ubuntu 9.10 Karmic Koala. I did not have this problem in 8.10 Ibex.
I keep my system config files such as for dhcpd (DHCP daemon) in a separate directory with all my system tweaks. I then set up a symbolic link from /etc/dhcpd3/dhcpd.conf --> my config.

But when I started the dhcpd server via /etc/init.d/dhcpd3-server restart
I kept getting the error message "Can't open /etc/dhcp3/dhcpd.conf: Permission denied"

Moving the config to /tmp also did not work, as tested using the -t flag, via
dhcpd3 -t -cf /tmp/dhcpd.conf

Finally, after some hunting, aka Googling, I found the problem was Apparmor, which restricts the files and directories that various services can use. So I added the following line to /etc/apparmor.d/usr.sbin.dhcpd3, where the dir should be the physical dir you get from cd -P in bash, namely remove all symlinks:

...
/dir/holding/my/dhcpd/config/** r
...

I then restarted apparmor via /etc/init.d/apparmor restart

And now I'm golden.

Essential maintainence of your Windows computer

Over the winter break, my brother brought up his wife's Win XP box which had been rendered unbootable due to a virus downloaded by their son. I pulled out the hard drive or HD and connected it as an external drive to a working XP box. I then spent many hours

1. removing the viruses. I used MS Security Essentials and then some newly downloaded AV software from malwarebytes.com
   


2. trying to get the cleaned HD to boot to no avail. Even booting via "Safe Mode with Command Prompt" which is least demanding boot process would hang in the middle.
   


3. copying the personal data in C:\Documents And Settings to a safe place. On this 2005 Acer, this consisted of copying from the main XP partition (80G) to the second "data" partition on the HD (also 80G), as Acer had conveniently split the HD into two partitions, the second being just for data. But I worried that the blind re-install might overwrite both partitions, so I also copied the personal data to another HD. And finally my brother copied select folders to another computer.


4. finally, running the Acer recovery CD 1. It turns out the recovery was a Norton Ghost image that was 2.0 GB which spanned 3 CDs. Now that the data was safe, the restore took very little time.

About 6 months ago, my own laptop got infected. I was able to remove the virus but the computer was no longer stable.

The moral: Recovering personal data and trying to remove the malware is very time consuming and stressful, as you don't want to trash anything accidently and there is sigficant uncertainty in what happened and what to do. You can easily waste 8-16 hours here. After you finally realize a clean install is necessary, reinstalling the OS is really pretty fast (less than an hour) and stress free.

Here are the essential actions you must do if you own a computer and don't want to lose your data or spend significant time/money/anguish trying to recover precious things. These are all obvious. And fortunately easier than ever before.

1. (a) Make periodic backups, say every 1-6 months. If you skip this step, eventual disaster is almost certain due to hardware failures, a virus infection, or user error. If you do this step and nothing else, the damage is contained. External USB hard drives are $100 or less for 1 TB (!) of storage at all major retailers (Target, Costco, Longs, CVS, Walmart, etc).
   (b) pick a backup program of your choice and use it. Even just dumb copying "Documents and Settings" to a new folder with the date, say "docs-2010-01-05", gets you most of the protection you need.
2. Install anti-malware (virus, rootkit, spyware) protection. Use your favorite program if you have one.
   Genuine: If you don't have a preference or don't have any protection, download the excellent free Microsoft Security Essentials which is free, lightweight and fast, comprehensive in that it protects against all sorts of malware, easy to use with a clean UI, and perpetual in that does not have a time limit. Did I mention it was free too? Reviews of it are very good with it catching most malware and it seems to be getting better. The one catch, you must be running a licensed or "genuine" copy of Windows XP, Vista or 7.
   Not necessarily genuine: If you are not running a genuine copy of Windwos, I suggest the AV and anti-spyware that comes with the free Google Pack and choose the anti-malware. (About 4 years ago a nice, super basic Norton Security Scan was included but about 2 years ago Symantec changed it to an annoying crippled Security Scan). As of today the PC Tools AV and anti-spyware package is included, which I have not tested.
3. Enable automatic updates for Windows. Microsoft releases patches regularly, some of which protect against real threats.
   
4. Use Firefox as your browser, and accept the updates. As of Jan 2010, FF is the best browser out there for security and overall usability. It keeps track of known bad web sites and will often warn you if a web site is trying to install something funny.
   What about the other browsers?
   (i) Chrome is very nice, with the ability to kill specific web pages that are causing problems, but it needs support for plugins before I can offer my highest recommendation.
   (ii) IE 8 is the best browser Microsoft has ever produced but IE 7 is only good and IE 6 is just plain scary. IE is not updated very often either, so it's simplest to stay away.

And that's it. These are the essential actions, so I've kept it short.

Assigning / mapping hard drive letters in Windows XP

Do a google search for "assign usb drive letters", but you basically get the following instructions. I used Windows XP.

Note this technique works for external hard drives that are currently mounted.

1. Right-click: My Computer -> Manage
   or
   Start Menu -> Settings -> Control Panel -> Administrative Tasks -> Computer Management
2. In the Computer Management window left pane, left click on Disk Management. (You may need to open / expand the "Storage" item in the left pane.
3. Find the disk partition in the bottom right pane that you want to assign a different drive letter. Right--click on that partition, which should have the undesired drive letter and typically has the words "NTFS... Healthy". Choose the option to "Change Drive Letter and Paths ..."
4. Fill in the desired drive letter and you are done. This mapping should persist in the future too.

Keywords: assign drive letters Windows XP, map drive letters Windows XP, drive letters for removable drives.