IMG TITLE

Interface Overview

img

Bit Hoarder Software-Defined-Storage (SDS) Network-Attached-Storage (NAS) appliances are designed to connect your users in new and unique ways and to provide highly reliable services for your data. With data demands in such high need and an ever increasing need for throughput, our Professional and Enterprise class of appliances deliver blazing fast storage suitable for any application.

With access to a state of the art file system, you will gain flexibility, file integrity and performance. With our software you will gain intuitive access to snapshots, deduplication, user management, monitoring and trending and storage management. Please select a link on the left to learn more.

The ZFS Filesystem

img

What is ZFS?

ZFS stands for Zettabyte File System. ZFS is a highly robust, scalable and state of the art file system that fundamentally changes the way file systems are created and managed. ZFS provides features and benefits not found in any other file system available today.

Primary Benefits

  • Endless scalability: With a 128-bit filesystem, there are no tangible limits to the size of your storage pools. The file system allows for up to one BILLION Terabytes of data! To put that in perspective, that's equal to approximately 36 million years of high definition video. No matter how much storage you require, ZFS will be suitable for managing it.
  • Ease of Scaling: ZFS virtualizes your hard drives as virtual devices. There are numerous types of virtual devices such as RAID 1 or RAID 5. Virtual devices are then grouped into a pool and served up as one or more shares as seen in the diagram above. When you require more storage, you simply add another virtual device.
  • Exceptional File Integrity: Every pixel of video is susceptible to corruption in a traditional filesystem. With every file saved to a Bit Hoarder appliance, a checksum and hash is generated. This allows for fast detection of read/write errors should they occur. With our broad range of possible RAID configurations, additional layers of redundancy or error detection and correction can be easily achieved. With the right configuration, no file will ever be corrupted that can't be repaired.
  • Maximum Performance: ZFS can be a very fast file system, primarily due to self-tuning and the inherent nature of the algorithms at play. With ZFS, you can easily exceed the performance of most hardware based RAID controllers.
  • Transactional Data Processing: With traditional file systems, corruption of the entire filesystem can occur due to power loss during complex write operations. ZFS is a transactional file system which uses a copy on write philosophy. With a transactional file system, data is never overwritten until all space on the pool is consumed. Any write sequences are either committed or are ignored entirely. As a result, although you may lose the most recently written data, the file system can never become corrupted due to power failure

Software Defined Storage

img

What is Software Defined Storage?

Software Defined Storage (SDS) is a revolutionary new technique for managing digital data using policy-based provisioning defined by software. Traditional storage systems use firmware (e.g. BIOS) or hardware (e.g. a RAID controller) to manage storage policies.

Why should I use SDS?

  • Economics: SDS can reduce capital expenditures by eliminating the need for additional expensive hardware usually required by traditional storage methods. SDS can also reduce operating costs by centralizing management of your storage.
  • Central and Flexible Management: SDS allows for central control of distributed storage and does not require lengthy initiation processes or complicated meta-data controllers.
  • Ease of Management: SDS can be expanded with ease and is inherently able to maintain system symmetry when needed.
  • Maximum Performance: ZFS can be a very fast file system, primarily due to self-tuning and the inherent nature of the algorithms at play. With ZFS, you can easily exceed the performance of most hardware based RAID controllers.
  • Heterogeneity: Though it is not recommended for media production use, SDS can eliminate the need for storage symmetry. Traditionally constraints such as media type, speed and location can limit the overall size and performance of storage systems, but not with SDS.
  • Hardware Abstraction: By creating a layer of abstraction between storage hardware and data, using software to virtualize the hardware, SDS creates unlimited possibilities for how hard drives can be grouped, arranged and utilized.
  • Share Abstraction: By creating a layer of abstraction between the virtualized hardware and the data, SDS provides for unlimited volumes or shares. Traditional file systems require a partition, or block of data, to be set aside for a volume or share, but not with SDS.

Network Attached Storage

img

What is Network Attached Storage?

Network Attached Storage (NAS) is a data storage device that provides file-based shared storage through a local area network (LAN).

Heterogeneity

Multiple users on completely different platforms are able to access and edit the same files using low latency protocols such as SMB and NFS.

Privacy and Control

Owning a NAS is like owning your own private cloud except that it's faster, less expensive and gives you complete control over the balance between storage capacity, speed and redundancy.

NAS vs SAN

  • NAS appliances can leverage a company's existing IP infrastructure which can greatly reduce the initial costs associated with shared storage, SAN appliances cannot.
  • NAS appliances define and control the entire filesystem while SAN appliances provide block level access to storage. Thus, a NAS provides better integration for resource management and offers increased flexibility and advanced functionality such as snapshots and deduplication.
  • With one or more optional 10 Gbps Ethernet cards, NAS devices can offer on par performance as compared to a SAN.
  • NAS appliances offer greater availability and redundancy thanks to RAID technology. SANs require block by block copying operations, even if blocks are empty, to achieve any form of redundancy or backup.

Snapshots

img

What is a Snapshot?

A snapshot is a virtual copy of an entire volume for a single moment in time. A snapshot is not a backup.

Snapshots vs. Backups

  • Snapshots are created instantaneously while backups can take many hours.
  • Snapshots initially consume no additional disk space. Space is only consumed by the snapshot as data within the pool changes. A backup on the other hand can require an enormous amount of disk space, slightly less than the original disk space if compression is used.
  • The theoretical limit on the number of snapshots is 264 (18 million trillion), regardless of the size of the storage pool. The actual limit on the number of backups is entirely dependent on how much space you're willing to consume, and how many hours you have to do it.

Do I still need to take backups if I own a Bit Hoarder?

Backups are always recommended, but maintaining multiple versions of backups is not as critical as it used to be. Now, all changes are tracked with the concept of the snapshot and one off site backup is usually sufficient to achieve peace of mind.

Deduplication

img

What is deduplication?

Deduplication is the process of eliminating identical copies of data.

But wait, isn't redundancy a good thing?

Intentional redundancy is a good thing, unintentional redundancy only wastes disk space. Deduplication only involves removing unnecessary copies of data. It is block level deduplication; to the user, duplication is still apparent but to the hard drive it's the same data on disk.

For example, say a coworker downloads a project folder and works on the project locally while on an airplane. When he or she gets back to the office, they load the revised project back onto the server as a new revision, without deleting the old folder just in case. In a traditional filesystem, all common data in these two folders would be replicated on disk as a side effect. However, SDS deduplication allows for automatic block level linking to common data. With SDS deduplication, only the data that was added or changed would consume additional disk space. We also have a file level deduplication option for scheduled deduplication tasks.

Are there any costs associated with deduplication?

Due to the automatic linking discussed above, block level deduplication requires stronger checksum algorithms, which could have a small performance impact because they take up more RAM. Performance is not generally impacted unless all RAM is utilized. Adding additional RAM or a cache device, or switching to file level scheduled deduplication tasks, will provide the solution if that occurs.

Generally speaking, the more space deduplication saves, the more the benefits will outweigh the cost.

An advantage of ZFS deduplication is that it doesn't require nightly tasks to run, data is de-duplicated before it is even written to disk.

How much space can I really save with deduplication?

It all depends on the type of data you are dealing with. If there is a high potential for duplicated data, than the savings will be significant. For example, hosting hundreds of nearly identical virtual machines, or where multiple versions of the same product are saved for historical purposes would both provide significant savings. However, storing any number of completely unique photographs will not provide the benefit, albeit with a slight additional overhead.

User Managment

img

User Management

Multiple users can be created with a few easy mouse clicks. Each user account is assigned a profile which includes enough information to identify and contact the associated user, which eliminates the possibility of orphaned accounts.

With Bit Hoarder's unique web based interface, users can be assigned to groups, and groups can then be assigned to volumes. This restricts access when needed. When access restrictions are not required, Bit Hoarder provides a guest mode for all volumes allowing anyone to access the volume.

Group Management

Bit Hoarder also provides an interface for managing users relative to the group. This additional interfaces allows faster management of users under certain scenarios, such as when a large number of users need to be removed from a single group.

Interface Access

Access to Bit Hoarder's interface is accomplished using the same accounts that are used for volume access. This eliminates the need to have two interfaces for managing what a user can do, as is typical with other systems.

RAID Storage Options

img

Hybrid (Nested) RAID

Bit Hoarder works off the fundamental concept of virtualizing your hardware. When you understand that, everything else makes sense. "Virtual devices" (vdevs) are defined by the user and then combined into a pool to create what is known as hybrid or nested RAID (i.e. RAID 0/10/50/60/70).

Data written to a pool is then stripped across all virtual devices within that pool to achieve exceptional performance, so virtual devices need to be resilient.

Virtual devices come in the form of RAID 0, 1, 5, 6 or 7 and special virtual devices used to bolster performance. The RAID levels, or virtual device types, are discussed below.

Virtual Device Types

img

RAIDz 0 (striping): All data is striped across every disk. This configuration is not recommended due to the potential catastrophic loss of data that you would experience if you lost even a single drive from a striped array. The advantage is maximum space savings since this is a non-redundant configuration.

img

RAIDz 1 (mirror): A mirror of two or more physical devices. Data is replicated in an identical fashion across all components of a mirror.

img

RAIDz 5 (single parity): A variation on RAID-5 (single parity) that allows for better distribution of parity and eliminates the 'RAID-5 write hole' (in which data and parity become inconsistent after a power loss). Data and parity is striped across all disks.

img

RAIDz 6 (double parity): A variation on RAIDz-5, but with two parity bits instead of one.

img

RAIDz 7 (triple parity): A variation on RAIDz-5, but with three parity bits instead of one.

Virtual Device Comparison

Assuming N is the number of disks and all disks are the same, of size X, we can estimate some important properties about each virtual device type.

RAIDz 0 RAIDz 1 RAIDz 5 RAIDz 6 RAIDz 7
Minimum number of disks 1 2 3 4 5
Recommended Number of disks1 Any 2-6 3-9 4-12 5-15
Approximate bytes available N × X X (N-1) × X (N-2) × X (N-3) × X
Maximum disk failures without potential data lose 0 N-1 1 2 3

1Actual recommended number of disks depends on the anticipated size of the entire pool, but should be in multiples of the minimum number of disks.