points by cryptonector 4 years ago

I'm quite aware of the BP rewrite problem.

I should explain that what I had in mind is that nearly worn out blocks should be marked no-longer-to-be-written-to. I left that part out due to writing too quickly. The point is there's no need to move blocks, just stop writing to worn-out blocks, and then the CoW nature of the filesystem will take care of the rest. Eventually you'll run out of non-worn-out blocks and the filesystem will become effectively read-only.

(Naturally one would want to keep a count of the number of worn-out blocks in the volume, and one would need an internal file in which to keep track of worn-out blocks, probably.)

As for BP rewrite, the fundamental design fault in ZFS that makes BP rewrite hard is that block pointers include locations in them and so the checksums of blocks that have block pointers must change when any of those pointer-to blocks are relocated. What should have happened instead is that physical locations should have been separated so as to avoid block checksums binding physical block addresses: a) block pointers should have had no physical location in them, b) every block that contain block pointers should have been followed immediately by a "cache" array of physical pointers corresponding to the logical block pointers in that block, c) the "cache" of physical block addresses would then be easily excluded from block checksum computations, d) block caches would have been easy to overwrite.

Once block checksums in block pointers do not bind physical addresses, you can then traverse the tree and relocate blocks by copying them to new locations then re-writing the physical block addresses in the caches (that are now not bound into those checksums). This is a very simple process. You do need to avoid creating multiple copies of blocks that are reachable via multiple paths (because of hardlinks).

This proposal tends to obtain replies to the effect of that design increasing the risk of checksum collisions. That's fallacious for cryptographic checksums.

I continue to be shocked that the BP rewrite problem isn't properly understood.

However, it is probably too late to fix ZFS to make BP rewrite easy. The reason is two-fold: 1) the lack of BP rewrite has been worked around for things like vdev evacuation, 2) moving physical locations out of the block pointer would essentially yield a completely different on-disk layout for znodes and indirect nodes and so on, and moving to the new on-disk format would result in special-casing a lot of code, and would be quite a process. It would almost be easier to start over.