Wiki code for ZFS FAQ

Hide Line numbers
1: == ZFS Frequently Asked Questions (FAQ) ==
2: 
3: {{toc start="3" depth="3" numbered="true" scope="local"/}}
4: 
5: === ZFS Product Release Questions ===
6: 
7: {{toc start="4" depth="4" numbered="true" scope="local"/}}
8: 
9: ==== **How can I get ZFS?** ====
10: 
11: ZFS is available in the following releases:
12: 
13: * [[Solaris Nevada release, build 27a and later>>Main.downloads]]
14: * [[Solaris Express releases>>http://www.sun.com/software/solaris/solaris-express/get.jsp]]
15: * [[Starting in the Solaris 10 6/06 release>>http://www.sun.com/solaris/get.jsp/]]
16: 
17: ==== **When will ZFS be available for <insert OS here>** ====
18: 
19: Projects are under way to port ZFS to [[FreeBSD>>http://lists.freebsd.org/pipermail/freebsd-current/2006-August/065306.html]] and to [[Linux>>http://zfs-on-fuse.blogspot.com/]] (using FUSE). For more information on CDDL, see the [[licensing FAQ>>Main.licensing_faq]].
20: 
21: ==== **What does ZFS stand for?** ====
22: 
23: Originally, ZFS was an acronym for "Zettabyte File System." The largest [[SI prefix>>http://en.wikipedia.org/wiki/SI_prefix]] we liked was ’zetta’ (’yotta’ was out of the question). Since ZFS is a 128-bit file system, the name was a reference to the fact that ZFS can store 256 quadrillion zettabytes (where each ZB is 270 bytes). Over time, ZFS gained a lot more features besides 128-bit capacity, such as rock-solid data integrity, easy administration, and a simplified model for managing your data.
24: 
25: === ZFS Technical Questions ===
26: 
27: {{toc start="4" depth="4" numbered="true" scope="local"/}}
28: 
29: ==== **Why does ZFS have 128-bit capacity?** ====
30: 
31: File systems have proven to have a much longer lifetime than most traditional pieces of software, due in part to the fact that the on-disk format is extremely difficult to change. Given the fact that UFS has lasted in its current form (mostly) for nearly 20 years, it’s not unreasonable to expect ZFS to last at least 30 years into the future. At this point, Moore’s law starts to kick in for storage, and we start to predict that we’ll be storing more than 64 bits of data in a single filesystem. For a more thorough description of this topic, and why 128 bits is enough, see [[Jeff’s blog entry.>>http://blogs.sun.com/roller/page/bonwick?entry=128_bit_storage_are_you]]
32: 
33: ==== **What limits does ZFS have?** ====
34: 
35: The limitations of ZFS are designed to be so large that they will never be encountered in any practical operation. ZFS can store 16 Exabytes in each storage pool, file system, file, or file attribute. ZFS can store billions of names: files or directories in a directory, file systems in a file system, or snapshots of a file system. ZFS can store trillions of items: files in a file system, file systems, volumes, or snapshots in a pool.
36: 
37: === ZFS/UFS Comparison Questions ===
38: 
39: {{toc start="4" depth="4" numbered="true" scope="local"/}}
40: 
41: ==== **Why doesn’t ZFS have an fsck-like utility?** ====
42: 
43: There are two basic reasons to have an fsck-like utility:
44: 
45: * **Verify file system integrity** - Many times, administrators simply want to make sure that there is no on-disk corruption within their file systems. With most file systems, this involves running fsck while the file system is offline. This can be time consuming and expensive. Instead, ZFS provides the ability to ’scrub’ all data within a pool while the system is live, finding and repairing any bad data in the process. There are future plans to enhance this to enable background scrubbing.
46: * **Repair on-disk state** - If a machine crashes, the on-disk state of some file systems will be inconsistent. The addition of journalling has solved some of these problems, but failure to roll the log may still result in a file system that needs to be repaired. In this case, there are well known pathologies of errors, such as creating a directory entry before updating the parent link, which can be reliably repaired. ZFS does not suffer from this problem because data is always consistent on disk.
47: A more insidious problem occurs with faulty hardware or software. Even file systems or volume managers that have per-block checksums are vulnerable to a variety of other pathologies that result in valid but corrupt data. In this case, the failure mode is essentially random, and most file systems will panic (if it was metadata) or silently return bad data to the application. In either case, an fsck utility will be of little benefit. Since the corruption matches no known pathology, it will be likely be unrepairable. With ZFS, these errors will be (statistically) nonexistent in a redundant configuration. In an non-redundant config, these errors are correctly detected, but will result in an I/O error when trying to read the block. It is theoretically possible to write a tool to repair such corruption, though any such attempt would likely be a one-off special tool. Of course, ZFS is equally vulnerable to software bugs, but the bugs would have to result in a consistent pattern of corruption to be repaired by a generic tool. During the 5 years of ZFS development, no such pattern has been seen.
48: 
49: ==== **Why does du(1) report different file sizes for ZFS and UFS? Why doesn’t the space consumption that is reported by the df command and the zfs list command match?** ====
50: 
51: On UFS, the du command reports the size of the data blocks within the file. On ZFS, du(1) reports the actual size of the file as stored on disk. This size includes metadata as well as compression. This reporting really helps answer the question of "how much more space will I get if I remove this file?" So, even when compression is off, you will still see different results between ZFS and UFS.
52: When you compare the space consumption that is reported by the df command with the zfs list command, consider that df is reporting the pool size and not just file system sizes. In addition, df doesn’t understand descendent datasets or whether snapshots exist. If any ZFS properties, such as compression and quotas, are set on file systems, reconciling the space consumption that is reported by df might be difficult.
53: Consider the following scenarios that might also impact reported space consumption:
54: 
55: * For files that are larger than recordsize, the last block of the file is generally about 1/2 full. With the default recordsize set to 128 KB, approximately 64 KB is wasted per file, which might be a large impact. The integration of RFE 6812608 would resolve this scenario. You can work around this by enabling compression. Even if your data is already compressed, the unused portion of the last block will be zero-filled, and compresses very well.
56: * On a RAIDZ-2 pool, every block consumes at least 2 sectors (512-byte chunks) of parity information. The space consumed by the parity information is not reported, but because it can vary, and be a much larger percentage for small blocks, an impact to space reporting might be seen. The impact is more extreme for a recordsize set to 512 bytes, where each 512-byte logical block consumes 1.5 KB (3 times the space).
57: Regardless of the data being stored, if space efficiency is your primary concern, you should leave the recordsize at the default (128 KB), and enable compression (to the default of lzjb).
58: 
59: ==== **Can I set quotas on ZFS file systems? ** ====
60: 
61: Yes, ZFS provides several different quota features:
62: 
63: * File system quotas (quota property) - ZFS file systems can be used as logical administrative control points, which allow you to view usage, manage properties, perform backups, take snapshots, and so on. For home directory servers, the ZFS model enables you to easily set up one file system per user. ZFS quotas are intentionally not associated with a particular user because file systems are points of administrative control. ZFS quotas can be set on file systems that could represent users, projects, groups, and so on, as well as on entire portions of a file system hierarchy. This allows quotas to be combined in ways that traditional per-user quotas cannot. Per-user quotas were introduced because multiple users had to share the same file system. ZFS file system quotas are flexible and easy to set up. A quota can be applied when the file system is created. For example:
64: 
65: {{{# zfs create -o quota=20g tank/home/users}}}
66: 
67: User file systems created in this file system automatically inherit the 20-Gbyte quota set on the parent file system. For example: 
68: 
69: {{{# zfs create tank/home/users/user1
70: # zfs create tank/home/users/user2
71: # zfs list -r tank/home/users
72: NAME                   USED  AVAIL  REFER  MOUNTPOINT
73: tank/home/users        76.5K  20.0G  27.5K  /tank/home/users
74: tank/home/users/user1  24.5K  20.0G  24.5K  /tank/home/users/user1
75: tank/home/users/user2  24.5K  20.0G  24.5K  /tank/home/users/user2}}}
76: 
77: ZFS quotas can be increased when the disk space in the ZFS storage pools is increased while the file systems are active, without having any down time.
78: 
79: * Reference file system quotas (refquota property) - File system quota that does not limit space used by descendents, including file systems and snapshots
80: * User and group quotas (userquota and groupquota properties) - Limits the amount of space that is consumed by the specified user or group. The userquota or groupquota space calculation does not include space that is used by descendent datasets, such as snapshots and clones, similar to the refquota property.
81: 
82: In general, file system quotas are appropriate for most environments, but user/group quotas are needed in some environments, such as universities that must manage many student user accounts. RFE [[6501037>>http://bugs.opensolaris.org/view_bug.do?bug_id=6501037]] has integrated into Nevada build 114 and the Solaris 10 10/09 release.
83: 
84: An alternative to user-based quotas for containing disk space used for mail, is using mail server software that includes a quota feature, such as the Sun Java System Messaging Server. This software provides user mail quotas, quota warning messages, and expiration and purge features.
85: 
86: === ZFS Administration Questions ===
87: 
88: {{toc start="4" depth="4" numbered="true" scope="local"/}}
89: 
90: ==== **Why doesn’t the space that is reported by the zpool list command and the zfs list command match?** ====
91: 
92: The available space that is reported by the zpool list command is the amount of physical disk space. The zfs list command lists the usable space that is available to file systems, which is disk space minus ZFS redundancy metadata overhead, if any.
93: 
94: ===== **What can I do if ZFS file system panics on every boot?** =====
95: 
96: ZFS is designed to survive arbitrary hardware failures through the use of redundancy (mirroring or RAID-Z). Unfortunately, certain failures in non-replicated configurations can cause ZFS to panic when trying to load the pool. This is a bug, and will be fixed in the near future (along with several other nifty features, such as background scrubbing). In the meantime, if you find yourself in the situation where you cannot boot due to a corrupt pool, do the following:
97: 
98: {{{<boot using ’-m milestone=none’>
99: # mount -o remount /
100: # rm /etc/zfs/zpool.cache
101: # reboot}}}
102: 
103: This will remove all knowledge of pools from your system. You will have to re-create your pool and restore from backup.
104: If a ZFS root file system panics, then you must boot from alternate media, import the root pool, resolve the issue that is causing the failure, export the root pool, and reboot the system. For more information, see the [[ZFS Troubleshooting Guide>>http://http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide]].
105: 
106: ==== **Does ZFS support hot spares?** ====
107: 
108: Yes, the ZFS hot spares feature is available in the Solaris Express Community Release, build 42, the Solaris Express July 2006 release, and the Solaris 10 11/06 release. For more information about hot spares, see the ZFS Administration Guide.
109: 
110: ==== **Can devices be removed from a ZFS pool?** ====
111: 
112: Removal of a top-level vdev, such as an entire RAID-Z group or a disk in an unmirrored configuration, is not currently supported. This feature is planned for a future release and can be tracked with CR [[4852783>>http://bugs.opensolaris.org/view_bug.do?bug_id=4852783]].
113: You can remove a device from a mirrored ZFS configuration by using the zpool detach command.
114: You can replace a device with a device of equivalent size in both a mirrored or RAID-Z configuration by using the zpool replace command.
115: 
116: ==== **Can I use ZFS as my root file system? What about for zones?** ====
117: 
118: You can install and boot a ZFS root file system starting in the SXCE build 90 release and starting in the Solaris 10 10/08 release. For more information, see [[ZFS Boot>>Community Group zfs.boot]].
119: ZFS can be used as a zone root path in the Solaris 10 10/08 release, but configurations that can be patched and upgraded are limited. Additional ZFS zone root configurations that can be patched and upgraded are supported starting in the Solaris 10 5/09 release.
120: For more information, see the [[ZFS Admin Guide>>http://docs.sun.com/app/docs/doc/819-5461]].
121: In addition, you cannot create a cachefs cache on a ZFS file system.
122: 
123: ==== **Can I split a mirrored ZFS configuration?** ====
124: 
125: Currently, ZFS does not support the ability to split a mirrored configuration for cloning or backup purposes. The best method for cloning and backups is to use ZFS clone and snapshot features. For information about using ZFS clone and snapshot features, see the [[ZFS Admin Guide>>Community Group zfs.docs]]. See RFE [[6421958>>http://bugs.opensolaris.org/view_bug.do?bug_id=6421958]] to recursively send snapshots that will improve the replication process across systems.
126: In addition to ZFS clone and snapshot features, remote replication of ZFS file systems is provided by the Sun StorageTek Availability Suite product. AVS/ZFS demonstrations are available [[here>>Project avs.WebHome]].
127: Keep the following cautions in mind if you attempt to split a mirrored ZFS configuration for cloning or backup purposes:
128: 
129: * Splitting a mirrored ZFS configuration is not supported by ZFS. RFE [[6421958>>http://bugs.opensolaris.org/view_bug.do?bug_id=6421958]] is filed to provide this feature.
130: * You cannot remove a disk from a mirrored ZFS configuration, back up the data on the disk, and then use this data to create a cloned pool.
131: * If you want to use a hardware-level backup or snapshot feature instead of the ZFS snapshot feature, then you will need to do the following steps:
132: ** zpool export pool-name
133: ** Hardware-level snapshot steps
134: ** zpool import pool-name
135: * Any attempt to split a mirrored ZFS storage pool by removing disks or changing the hardware that is part of a live pool could cause data corruption.
136: 
137: === ZFS and Other Product Interaction Questions ===
138: 
139: {{toc start="4" depth="4" numbered="true" scope="local"/}}
140: 
141: ==== **Is ZFS supported in a clustered environment?** ====
142: 
143: Solaris Cluster 3.2 supports a local ZFS file system as highly available (HA) in the Solaris 10 11/06 release. This support allows for live failover between systems, with automatic import of pools between systems.
144: If you use Solaris Cluster 3.2 to configure a local ZFS file system as highly available, review the following caution:
145: Do not add a configured quorum device to a ZFS storage pool. When a configured quorum device is added to a storage pool, the disk is relabeled and the quorum configuration information is lost. This means the disk no longer provides a quorum vote to the cluster. After a disk is added to a storage pool, you can configure that disk as a quorum device. Or, you can unconfigure the disk, add it to the storage pool, then reconfigure the disk as a quorum device.
146: Solaris Cluster 3.2 is not supported on the OpenSolaris or Nevada releases. For information about using the open-source Solaris Cluster version, go to the [[Open High-Availability Cluster community page>>Community Group ha-clusters.WebHome]].
147: ZFS is not a native cluster, distributed, or parallel file system and cannot provide concurrent access from multiple, different hosts.ZFS works great when shared in a distributed NFS environment.
148: In the long term, we plan on investigating ZFS as a native cluster file system to allow concurrent access. This work has not yet been scoped.
149: 
150: ==== **Which third party backup products support ZFS?** ====
151: 
152: * EMC Networker 7.3.2. backs up and restores ZFS file systems, including ZFS ACLs.
153: * Veritas Netbackup 6.5 backs up and restores ZFS file systems, including ZFS ACLs.
154: * IBM Tivoli Storage Manager client software (5.4.1.2) backs up and restores ZFS file systems with both the CLI and the GUI. ZFS ACLs are also preserved.
155: * Computer Associates’ BrightStor ARCserve product backs up and restores ZFS file systems, but ZFS ACLs are not preserved.
156: 
157: ==== **Does ZFS work with SAN-attached devices?** ====
158: 
159: Yes, ZFS works with either direct-attached devices or SAN-attached devices. However, if your storage pool contains no mirror or RAID-Z top-level devices, ZFS can only report checksum errors but cannot correct them. If your storage pool consists of mirror or RAID-Z devices built using storage from SAN-attached devices, ZFS can report and correct checksum errors.
160: For example, consider a SAN-attached hardware-RAID array, set up to present LUNs to the SAN fabric that are based on its internally mirrored disks. If you use a single LUN from this array to build a single-disk pool, the pool contains no duplicate data that ZFS needs to correct detected errors. In this case, ZFS could not correct an error introduced by the array.
161: If you use two LUNs from this array to construct a mirrored storage pool, or three LUNs to create a RAID-Z storage pool, ZFS then would have duplicate data available to correct detected errors. In this case, ZFS could typically correct errors introduced by the array.
162: In all cases where ZFS storage pools lack mirror or RAID-Z top-level virtual devices, pool viability depends entirely on the reliability of the underlying storage devices.
163: If your ZFS storage pool only contains a single device, whether from SAN-attached or direct-attached storage, you cannot take advantage of features such as RAID-Z, dynamic striping, I/O load balancing, and so on.
164: ZFS always detects silent data corruption. Some storage arrays can detect checksum errors, but might not be able to detect the following class of errors:
165: 
166: * Accidental overwrites or phantom writes
167: * Mis-directed reads and writes
168: * Data path errors
169: 
170: Keep the following points in mind when using ZFS with SAN devices:
171: 
172: * Overall, ZFS functions as designed with SAN-attached devices, as long as all the drives are only accessed from a single host at any given time. You cannot share SAN disks between pools on the same system or different systems. This limitation includes sharing SAN disks as shared hot spares between pools on different systems.
173: * If you expose simpler devices to ZFS, you can better leverage all available features.
174: In summary, if you use ZFS with SAN-attached devices, you can take advantage of the self-healing features of ZFS by configuring redundancy in your ZFS storage pools even though redundancy is available at a lower hardware level.
OpenSolaris

Top Menu

Wiki code for ZFS FAQ

Search

Collectives

Community Group

Project

User Group

Subsites

Community Group zfs Pages
