Open Source LessFS a Serious Competitor in the Data Deduplication Space

Data Deduplication or Data Dedupe is one of the fastest growing segments of the data backup industry. In data deduplication a disk array is used to copy the data. The data is copied once and small changes to a file are recorded with just the changes added to the array. If a file is already on the array, it does not re-record the same file. Different algorithms and compression are used, but the principles apply across vendors. A relatively small array can hold many times more data than a standard backup tape or disk based tape array. Patterns in the data are analyzed to prevent redundant data from being recorded. One of the biggest players in the industry (Data Domain) was recently acquired by a backup powerhouse, EMC. EMC can use the acquisition to combine backup software products like Networker and Retrospect with the Data Domain suite. While Data Domain might be the pre-eminent vendor in the industry there is open source competition that is developing. In reality, a standard Data Domain box is simply a rebranded Supermicro storage chassis with a standard server motherboard, a 512 MB NVRAM card and a hardened 2GB Industrial ATA drive for storage of OS Data. The hardware can be had for not very much money. The price you pay is for the software. I believe Open Source will represent a major competitor to Data Domain in the near future. An open source product called LessFS, which boasts some very quick speeds and is currently at revision .0.6.1. According to the Project homepage it is currently capable of speeds up to 350 MB/sec. This is very impressive! I suggest giving LessFS a serious look especially as it reaches its first production release. The project is available at: http://www.lessfs.com/wordpress/