Researching how to do FTP from an AIR application

• FTP from Adobe AIR and Flex

From Terabyte Drives Make It Challenging To Create Regular Backups

If you didn't have so much stuff, you wouldn't need a house, you could just walk around all day....
George Carlin, Peace Be Upon Him, died two weeks ago

I keep thinking if I didn't have so much on disk drives, I wouldn't need such big drives, and so much room to create backups. But when you take into account all the time it takes to blow DVDs along with verifying them, they are almost useless as backup media. When I make a backup from my computer, instead of sitting around and waiting to feed it 4 or 5 dvds, I always set it running to make the backup to an external hard drive. Hence the files just accumulate. I got lured into making those emusic dvds the other day because I didn't have enough spare hard drive space at the time to copy the files off that way. Plus, it just seemed like a good idea to make an offline backup of my music files. Hint: Never rely on a single backup of anything important.

Even with 25gb blue ray disks, which take a really long time to write (maybe 1-2 hours), the problem is just as bad, compared with terabyte external drives, which would require 40 blue ray disks to backup. And the blue ray disks aren't cheap, that backup would probably cost $400 or more if it was done to re-writable blue ray disks. Of course rewritables are a risky proposition, but not too bad if the backup software can do a verify operation on each disk.

From Terabyte Drives Make It Challenging To Create Regular Backups

Restoring your music collection from emusic can take you a long time...

Now I get to spend the entire day downloading almost all my emusic collection from scratch. Fortunately they remember everything I previously downloaded. But they don't have a "download all" button. I have to go to each artist, then each album, then sometimes to each song individual, or to the album as a whole, and click a download link, then wait 20-30 seconds while the emusic remote software UI hangs up and becomes unresponsive while it processes the request into it's background queue. Then hit the back button, and go to the next. I've completed over 120 pages of the book "Free Lunch" by David Cay Johnston, so that gives you some idea how long this has been taking. Too bad their UI does such a poor job at coordinating the background downloads.

From Terabyte Drives Make It Challenging To Create Regular Backups

The problem of trying to get backup files off an external USB hard drive and onto a NAS: Glacial copying speeds

However another problem that I'm in this middle of is this: I have a 320GB external drive (which only shows as 279GB to file explorer), which I carry around. I keep the most recent disaster recovery backup of my laptops on it, along with various other stuff. Right now it has several checkpoint backups of my machines, eating up 90GB. Some is 4GB backup software spanned images, others are just copies of native files.

So I started a copy process from the 320GB drive to a share on a terastore NAS drive. I ran an ethernet wire direct to my laptop, I didn't want to trust sending all this stuff wirelessly. Plus one of my other XP machines with Atheros wifi (Fujitsi U810) kept periodically losing it's wireless connection to my WEP'd router as I was trying to use it to copy backups to the NAS.

The Windows vista copy dialog said it would take a 3 or 4 hours. Everytime I looked, it kept saying it needed at least another hour or two. Ok, no problem, I've got a few star wars movies to watch. (Ended up watching three complete episodes before the copy finished.) As the last movie ended, I looked at the copy box. (I was now in the process of spending a lot of time to use emusic to download all my lost music.) The dialog was then claiming it needed one or two days to finished downloading about 10,000 files, only the last 4.5 GB of the 90GB.) Huh? Why is that? I set the laptop somewhere to work all through the night, and hope that by this morning it would change its mind.

Yep, by this morning it has. At 10:20am now, it says

• Copying 61,323 items (96.2 gb)
• Time remaining: About 4 days and 10 hours
• Items remaining: 8,721 (4.43 gb)
• Speed 1.62 MB/sec

Last night the speed had been given as about 4MB per second. Thats 32 megabits per second, I guess that's pretty close to the 100mbit speed of the network adapter. (There's one 1000mbit switch, and 60 feet of ethernet cable between the laptop and the NAS.)

Now three minutes later the dialog says:

• Time Remaining: About 4 days and 8 hours
• Items remaining: 8,685 (4.43 GB)
• Speed: 1.61 MB/sec

Why is it taking 36 minutes to copy 36 small files? Is the terastore NAS just really incredible slow, or is something else going on? I don't really know.

It's now 13 minutes later (10:33), and the Windows Vista copy progress dialog says:

• Time remaining: About 3 days and 22 hours
• Items remaining: 8,482 (4.42 GB)
• Speed: 1.59MB/sec

I guess I should be happy about the process. Except, now, at 10:38, Windows is now back up to 4 days and 2 hours.

So what to do? Obviously the Buffalo Terastore (Terastation) NAS is a lot more efficient copying larger files than smaller files.

Patient: Doctor, I run out of breath waiting for small files to copy!
Doctor: Then don't copy small files!

Now at 10:40, it's up to 4 days and 7 hours. 8,385 files to go....

Would I have been better off hanging giant USB drives, or firewire drives, or e-sata drives, off of a Windows 2003 box and using shares on it for backup? Something else to spend half a day testing.....

I guess I could go through my 320GB drive, and compare it's directories to the NAS, and start erasing the files from the 320GB drive. Then when I've erased all the larger backup images, then the small files that are left, I could compress them into a single .ZIP file. Or maybe use the backup software to create a backup image of the smaller files. I probably perfer the .zip file, since that's more transportable. So after creating the zip file, I can then unzip it somewhere else to make sure that it is entirely readble.

Good thing I have a few more books to read today.....

P.S., it's now 10:50, 30 minutes later:

• Time remaining: 3 days and 11 hours
• Items remaining: 8,188 (4.42 GB)
• Speed: 1.57MB/sec

8721-8188 = 533 files copied.... Hmmm.. 8188/533 * 30 minutes = 7.7 hours to go.

That's ridiculous for backing up 4gb....

From Terabyte Drives Make It Challenging To Create Regular Backups

How to make a real backup economically? Backup devices are very expensive now compared to primary storage

I've always believed that a backup isn't a backup as long as it is on spinning magnetic media. But non-hard drive media at a reasonable price just hasn't kept up with the growth of our hard drives. The Iomega REV drives are one of the best solutions I've found, but at only 35GB capacity, they are somewhat limited when compared to a terrabyte drive, and they are also more expensive than the hard drives you'd be backing up, and they are difficult to find at any local store. I had a 30GB tape drive, but I haven't seen that drive sold in years, so I have no idea where to get more tapes. I haven't seen a tape backup solution of that size sold in a retail store for years.

It seems the only cheap backup solution is to have multiple terabyte drives, and make backup copies from a main drive to one or more backup drives. Then keep the backup drives offline until they are needed for the next backup of the main drive. One might be tempted on subsequent backups to try and do an incremental backup, but that can be difficult. One tool that might help is robocopy which is in the Windows 2003 Resource Kit. Or it might be simpler to just periodically wipe your backup drive clean, then create a complete copy of the working primary. Of course you've got some risks: What if your primary fails while your in the process of making a new copy to the backup drive you just wiped? If you keep a couple of backup drives and can alternate between them, then hopefully you're never without a complete backup of your data.

From Terabyte Drives Make It Challenging To Create Regular Backups

Taking the time to make backups? Visit my Good bible books store to buy some good books to read...

Probably the worst part about making backups is babysitting the backup process. I like making backup DVDs with native files, that way the backups can be read on any machine without the need of backup software. But it takes a lot of effort to go through hundreds of gigabytes of directories, selecting 4GB chunks of stuff to put on DVDs one at a time. Then what if you run into oddly named files that are incompatible with DVDs (for example saved web pages that russian or chinese characters in the page title?) Thats why I created .ZIP files. Problem is, then you've got a single file that a single error can make all your files unreadable....

Using backup software might make this selection process more automatic, the backup software will automatically span disks at whatever interval you select. But how would the backup software deal with a multi-dvd backup set when one of the backup disks (or maybe the final disk which often is used to contain summary information about the whole backup set) happens to fail. Would it make all the other disks useless? This is something that would need to be carefully researched and tested. Allocate at least 3 or 4 hours for playing around with this scenario to decide if it is worth the risk.

From Terabyte Drives Make It Challenging To Create Regular Backups

beware of windows vista written dvds with compression for backups....

For over a year I've been using emusic.com to download all kinds of alternative electronic and ambient music. Along with some other CDs that I've ripped to mp3, the collection is now a few gigabytes in size. I wanted to remove all the files so that some disk defragmentation could do a better job to consolidate my files. So I took all my emusic.com downloaded mp3 collection zipped it into four or so zip files, then blasted it all onto 3 dvds with windows vista dvd writing.... Now I'm trying to put the files back, and 3 out of 4 of the zip files are unreadable, either unknown compression, or windows vista itself claiming the disks are blank when I insert them. One of the zips I copied off to another drive, it is claiming unknown compression on XP. Took the file back to the Vista machine which has winzip, and winzip can't read the file. The file never gave an error while copying it off the disk on win xp.

So it seems vista's dvd writing really sucks... This HP Pavillion tx1000 tablet-pc with Windows Vista machine came with roxio, but 3/4ths of the time roxio won't start up. It gives some exception on startup then shuts itself off immediately. Sometimes I can get it to load by double clicking an iso image, but that didn't even work the other day.... I gave up and used Windows Vista disk writing. Now I'm wishing I didn't.