FileSystem
Latest
Samsung creates F2FS file system for NAND flash storage, submits it to the Linux kernel
Has the lack of NAND flash storage-optimized file systems been bugging you? Then you've got something in common with Samsung, which has developed F2FS (or "Flash-Friendly File-System") for the memory of choice for mobile devices and its specific "internal geometry." It's based on a log-structured method, but tackles problems associated with older file systems intended mainly for retro, spinning-disk storage. The company isn't keeping its hard work behind lock-and-patent either -- it's gone open-source and submitted the file system to the Linux kernel, meaning you could see it implemented in Android hardware of the future. It's nice to see Sammy contributing code for the greater good, and if you've got the skills to understand it, a low-down of F2FS is available at the source below.
Linux 3.6 kernel released with 'hybrid sleep' capability, Google's TCP Fast Open extension
The last couple of Linux kernel updates have been a bit of a circus, with Android merging in the 3.3 ring and graphics integration on a (noisy) 3.4 center stage. Version 3.6 of the open source kernel has less flashy, more serene features this time, with hybrid standby, a revised file system and other esoteric networking and architectural features befalling the penguin. That sleep mode, which caches the RAM contents to a disk to prevent data loss, has been available for a while now on Windows and Mac machines and will be especially handy on Linux laptops. The so-called Btrfs file system has also been updated (though is still in a developmental mode) and Google's experimental TCP Fast Open extension has been added, along with additional network drivers, more virtualisation options and additional processor support. Sleepwalk to the source for the complete changelog.
RIM licenses Microsoft's exFAT file system to stuff extra-large files into BlackBerry devices
RIM has big ambitions for media on BlackBerry hardware -- really big. Accordingly, it just reached a patent licensing deal with Microsoft that lets it use the exFAT file system on "certain BlackBerry devices." We have a hunch that's a reference to future BlackBerry 10 phones and tablets rather than retrofits of existing (and likely incapable) mobile gear. The move will let any devices that do recognize exFAT handle much larger files, such as long videos, in addition to streamlining transfers between computers and other gadgets. The conditions of the deal haven't been given out, but we suspect RIM's negotiations with Microsoft were on more voluntary terms than those faced by Android supporters.
ZEVO ZFS on OS X project now part of GreenBytes
Update: GreenBytes has announced that starting in September, the ZEVO Community Edition product will be offered as a freely downloadable binary. As they get closer to the launch date, more info will be shared -- including a rundown on new, improved features from the previous commercial version. --- The ZEVO ZFS technology for Mac, developed by former Apple engineer Don Brady at his company Ten's Complement, has been acquired by enterprise storage vendor GreenBytes. Brady announced the transition on Twitter and on the Ten's Complement site; he is joining the GreenBytes development team. ZFS is a file system invented in 2004 at Sun (now part of Oracle) which offers next-generation support for huge volumes, fault tolerance and other essential Big Storage features. The history of ZFS on OS X is somewhat convoluted. In the run-up to OS X Leopard 10.5, it seemed likely that ZFS would be used as an alternative or replacement for HFS+ as the Mac's primary file system format. That never materialized, and the announced/anticipated support for ZFS in Snow Leopard never shipped either. Apple's open source ZFS project was closed down in 2009. Ten's Complement originally announced the ZEVO product in March of 2011. As to the future of the project, GreenBytes' spokesman Michael Robinson responded to TUAW's inquiry: "ZEVO's ZFS on OS X is safe. Ten's Complement has joined GreenBytes to continue their work and now they have more development support." Robinson says that GreenBytes will have more to share regarding ZEVO's future "down the line." Thanks Bill.
CHKDSK is changing how it works (step 1 of 1) 56 percent completed...
PC users of a certain age will be all too familiar with defragging and disk checking, normally as a last-ditch attempt to reinvigorate a flagging or faulty system. Fast-forward to 2012, and Microsoft is reassessing the role of the whole NTFS health model for the modern world (well, Windows 8 at least). It turns out that these days actual corruptions are rare, but people still like to run chkdsk just in case -- or out of habit. In the old approach, health check was either happy or unhappy, and the machine was taken offline for as long as was needed to fix. Even with optimization and improvements in later versions, the galloping sizes of hard drives has swallowed up much of the benefit. In the redesigned model there are four states: healthy, spot verification needed, scan needed and spot fix needed. In any of these states, the system remains online, with the user deciding when to restart if a fix is needed. The reboot process should also be much quicker, with the spot fix already targeted. Advanced users can go a stage further and invoke the spot fix while still online for sections of the disk not in use. The proof, of course, is in the pudding, but anything that involves less death-staring at a disk check is a good thing in our book. Hit the source for a blow-by-blow breakdown.
Microsoft introducing ReFS file system with Windows Server 8
Hungry for a shiny new file system? Windows 8's got your back, or at least, Windows Server 8 will. In his latest Building Windows 8 post, Steven Sinofsky introduces the Resilient File System, or ReFS, as a "next generation file system" built on the foundations of the NTFS. By reusing NTFS' API / semantics engine, ReFS hopes to retain a high level of compatibility with NTFS features. Underneath the existing semantics engine, the new file system introduces a new storage engine that hopes to protect against latent disk errors, resist data corruption, uphold metadata integrity, grant large volume, file and directory size -- and well, just build a better storage system in general. It's all quite complicated, but if you feel up to the technical snuff, click through the source link below.
Mac 101: Format choices for USB flash drives
The overwhelming majority of USB flash drives you buy are going to come in one of two formats: FAT32 or NTFS. The first format, FAT32, is fully compatible with Mac OS X, though with some drawbacks that we'll discuss later. If the drive comes formatted in NTFS, which is the default file system for Windows, you're going to want to re-format the drive because Mac OS X can't write files to NTFS-formatted volumes (at least not without a bunch of extra work that's beyond the scope of this article). How do you tell which format your brand-new USB drive has? Hook it up to your Mac and launch the Disk Utility app, located in your Utilities folder (which is in Applications). Your new drive should appear in the left-hand column, and clicking the "Partition" tab will bring up info on the drive which includes its current format. If the drive's format comes up as MS-DOS (FAT) or, less likely, ExFAT, you may be able to simply leave the drive as-is and not bother reformatting it. If the drive is listed as NTFS-formatted, you're going to need to format it to something else if you want full compatibility with Mac OS X. There are several possible file system formats you can use for a USB flash drive, and changing them in Disk Utility is as easy as selecting the number of partitions you want on the drive (usually just one), picking the format you want for the drive, and clicking "Apply." Note that this will erase all information on the drive, so make absolutely sure you've got copies of everything before moving forward. Disk Utility gives you five different choices for drive formats in OS X Lion. Unless you have extraordinary needs, you can safely ignore two of them: Mac OS Extended (Case-sensitive, Journaled) and Free Space. I'll discuss the advantages and disadvantages of the other three formats below. Mac OS Extended (Journaled) - This is the default file system format for Mac OS X drives. Advantages: Formatting your USB flash drive this way will give you full interoperability with Macs. It also includes support for features from OS X Lion, such as Versions. You can even set up an OS X startup drive if you have the right files, the know-how, and a big enough flash drive, which will allow you to boot your Mac off an external disk if something goes wrong with your built-in drive. The "Mac OS Extended (Journaled)" option will have the highest degree of support for Mac OS X features, and there's no limit to the size of files you can put on the drive. Disadvantages: Windows-running PCs can read files from drives formatted this way, but they can't write to them (at least not without the same amount of work it takes to get OS X to write to NTFS-formatted drives). If you're transferring files from Macs to PCs, this won't be an issue; however, transferring files from PCs to Macs won't be possible if your drive is formatted in "Mac OS Extended (Journaled)." If you will only be working with Macs and not PCs, this may not be an issue. Otherwise, you may need to consider one of the file formats discussed below. MS-DOS (FAT) - This is Disk Utility's name for the FAT32 filesystem. Advantages: FAT32 offers near-universal interoperability with virtually every computing system on the planet. A drive formatted this way can easily transfer files between Macs and PCs. You can also move files to video game systems like the PlayStation 3, Xbox 360, and Wii. Virtually all cameras and videocameras support FAT32, too. It's the closest thing we have to a universal file system format, which is why most flash drives are formatted this way right out of the box. Disadvantages: FAT32 doesn't support files larger than 4 GB, and that's its greatest drawback. You also can't create a startup drive for your Mac using this format. Furthermore, FAT32 doesn't support OS X Lion's Versions feature -- something users have discovered the hard way when working directly off of files stored on a USB flash drive (something we recommend against doing). However, those downsides may be more than outweighed by FAT32's near-universal support, and if you don't think you're going to be dealing with files bigger than 4 GB, this may be the optimal choice. ExFAT - A newer file format, supported in Mac OS X 10.6.5 or later. Advantages: exFAT has many of the same advantages as FAT32 in that it's a disk format that offers interoperability between Macs and PCs. It has one big advantage over FAT32: exFAT supports file sizes larger than 4 GB, so if you have a need to move huge files between Macs and PCs, this is likely the format you'll want for your flash drive. exFAT is supported by the following operating systems: Mac OS X Snow Leopard (10.6.5 or greater) OS X Lion Windows XP SP2 or later (with an additional update for exFAT support) Windows Vista SP1 or later Windows 7 Disadvantages: As a relatively newer file system format, exFAT isn't supported in older versions of Mac OS X (anything prior to 10.6.5) or anything older than Windows XP SP2. If you won't be dealing with older Macs or PCs, this may not be a problem. Of greater issue is that most consumer electronics (cameras, camcorders, video game systems) don't support exFAT, either. If you need to transfer files between your Mac and one of these non-PC devices, you're almost certainly going to have to format your flash drive in FAT32 instead. The Bottom Line Here's a basic rundown of which format we recommend for your USB flash drive, broken down by use case. If you absolutely, positively will only be working with Macs and no other system, ever: Use Mac OS Extended (Journaled). If you need to transfer files larger than 4 GB between Macs and PCs: Use exFAT. In all other cases: Use MS-DOS (FAT), aka FAT32.
IBM rig doesn't look like much, scans 10 billion files in 43 minutes
Someone ought to gift these IBM researchers a better camera, because their latest General Parallel File System is a back-slapping 37 times faster than their last effort back in 2007. The rig combines ten IBM System xSeries servers with Violin Memory SSDs that hold 6.5 terabytes of metadata relating to 10 billion separate files. Every single one of those files can be analyzed and managed using policy-guided rules in under three quarters of an hour. That kind of performance might seem like overkill, but it's only just barely in step with what IBM's Doug Balog describes as a "rapidly growing, multi-zettabyte world." No prizes for guessing who their top customer is likely to be. Full details in the PR after the break.
iPad OS digging reveals hints of future iPhone, iPod touch, iPad and Nessie
Well, well -- what have we here? A little digging into the mainframe that lies beneath Apple's freshly launched iPad has uncovered a few telltale signs of next-generation hardware, and while this is obviously in no way confirmation that new wares are on the way from Cupertino, this kind of stuff has led to similar reveals in the past. If we're seeing this right, we're looking at not one, but two new mentions of future iPhones (iPhone 3,2 and iPhone 3,3), alongside a new iPod touch (iPod 4,1) and... a future iPad (iProd 2,1). 'Course, none of this gives us any indication of the all-important whos or whens, but at least this confirms that mythical creatures still exist in the hearts of those who believe.
Apple post server-related updates for Xsan and Mail Services
Apple yesterday posted a few new server-related updates for Xsan and Mac OS X Server. The first, Xsan 2.2.1, improves file system reliability and cvfsck repair utility. This update is available for Mac OS X 10.6 Snow Leopard and Mac OS X 10.5 Leopard. There's also an updated version of the Xsan uninstaller. Next, Mail Services Update 1.0 addresses issues affecting overall mail service reliability and performance. This update is available and recommend for all users on Mac OS X 10.6 Snow Leopard. These updates are available in Software Update, or from Apple's Support Downloads page.
ZFS open source project abruptly shuts down, Snow Leopard weeps icy tears
The on-again / off-again love affair between Apple and ZFS seems to be all but over, with a brief but potent message on the Mac OS Forge project site stating the following: "The ZFS project has been discontinued. The mailing list and repository will also be removed shortly." If you'll recall, the implementation of the ZFS file system within Snow Leopard server was so close to happening that Apple actually published it as a feature of the forthcoming OS back in June of 2008. Now, however, all hope has presumably been lost. We'd bother explaining the rumors behind why all of this has suddenly crumbled, but honestly, will knowing the reasons really help the pain? No, no it won't. [Via TUAW]
ZFS project for Mac OS X discontinued
Is it a forgotten project? A fit of open-source pique? Or is it just that everyone got really busy on other stuff? Looks like Apple has just shut down the ZFS project for Mac OS X. The site for the project on Mac OS Forge reads: ZFS Project Shutdown 2009-10-23 The ZFS project has been discontinued. The mailing list and repository will also be removed shortly. It's hinted that this was caused by legal issues surrounding the merger of Sun Microsystems and Oracle, which caused Apple to back away from the next-generation file system technology. After many rumors, including ZFS support that was going to be added in Snow Leopard Server, but that ended up being removed in the final version, it now seems like the saga of ZFS on Mac OS X is ending with a whimper. [via Daring Fireball]
Friday Favorite: R-Name
R-Name is a simple, single-purpose application that takes the names of files and folders on your Mac and renames them one at a time or in batches. I use it almost every day. Yes, there are a dozen ways to batch rename files. I like R-Name for a few reasons: It's fast, both in loading and renaming Easy to rename just files, folders or recurse folder Hard to mess things up because you have to preview the new names before committing Drag and drop (Apple forgets how important this feature is all too frequently) Here are the functions you can perform with R-Name: find and replace, number sequentially, add characters (beginning, before extension, at end), remove characters with the same options, change case, add/replace/remove extensions. There are some smart prefs too, but that's icing on the cake. Oh, and did I mention: it's donationware! The author asks you to make a donation to UNICEF. Unfortunately, R-Name appears to be abandonware -- the author's site is currently down (thus the source code is unavailable) and best I can tell the app is only supported to 10.3. But it works for me. What renaming tool or workflow do you use?
SanDisk announces ExtremeFFS for "100 times faster" SSD write speeds
SSDs will apparently get a serious (extreme, if you will) upgrade in the coming year, with SanDisk's announcement of a new technology dubbed ExtremeFFS (Flash File System). It is, as you can possibly guess, a new file system, but it'll use page-based methods so that the data's location on the drive won't be tied to its physical space, as it is now. SanDisk says this means that random write speeds are going to be 100 times faster than they are on drives using current technology. The company hopes that this advance, coupled with other upcoming standards, will lead to widespread adoption of SSDs in PCs (and much, much more money in their pockets). That should fit in nicely with news we just heard that Microsoft's Windows 7, unlike Vista, will include optimization for use with the drives. While actual drives that carry ExtremeFFS have yet to be announced, SanDisk expects them to ship in 2009. Fun times.[Via Electronista, CNET]
Sun says Apple is switching to ZFS in Leopard
You dealt with it when Apple switched you from MFS to HFS, and again you were switched from HFS to HFS+ (and even journaled HFS+), and you'll deal with it again: according to Sun CEO Jonathan Schwartz (whom you may know for his totally righteous pony tail), Apple is going to use Sun's crazy advanced ZFS filesystem when they move users over to Leopard. The material advantages may not be immediately apparent to the average user (when was the last time you whipped up a multi-exabyte file?), but it will do some excellentay things like storage pooling (aka virtual storage), block-journaling, and plenty of other nerdy things about which you can read up on elsewhere. Expect to hear more about this one next Monday at WWDC.Watch - Schwartz talks it up [Via MacRumors]Read - More about why this is a Good Thing (TM)
TUAW Tip: Check the list of files that get installed on your Mac
Most Mac OS X users appreciate the simplicity of Mac application installation; it usually involves nothing more than a drag and drop out of a disk image, or simply unzipping a download. Still, some apps, plug-ins and other digital goodies require an installer, which means you're entering your password and granting permission for a program to place files anywhere on your machine. If you've ever been curious (or concerned) about this operation, Apple's default Installer app (found in /Applications/Utilities/ and used by most independent software vendors - but not all - use to distribute their products) can display a log of every file a piece of software installed on your machine, accessible under File > Show Files after the Installer finishes its work (but before you quit/close it, of course). As you can see, the log is even searchable, making it easy to sift through those big install operations.If you want to grab a copy of this log you can of course select all the text and copy/paste to your own file, but most installers (including those not from Apple) should place a copy of the install log in ~/Library/Logs, in which case you can use the Console app (also in the Utilities folder) to browse through those logs and easily mail a copy (with help from this tip) to a friend or developer. Some installers, however, place the log directly on your Desktop or on the root of your hard drive, and in my experience, most of those don't put a copy in the default logs location at all. Tsk, Tsk, guys.These tools should make a handy addition to your tool belt if you're beginning to keep a more watchful eye on what gets put where by these installers, or of course if you're simply interested in learning more about how Mac OS X is built and where files go to get the job done.
ZFS in Leopard?
The French site Mac4Ever appears to have found evidence (in the newest developer release) that Leopard will include the ZFS file system to supplement HFS+. What is ZFS you ask, and why should you care? Well ZFS is a pretty darn cool next generation file system created by Sun that includes a variety of cool new features for protecting your data (if any file system features can properly be called "cool"). John Siracusa over at arstechnica has written about ZFS several times, rather excitedly. I'll let him explain what the excitement is about: "ZFS does away with the old restrictions on volume size and scope, while also addressing data integrity and performance issues, all from a purely software perspective. (Like one slide says, "ZFS loves cheap disks!")The end game is a world where storage-even personal storage-actually behaves like the magically intelligent, infinitely expandable cloud that we'd all like to think it is, and less like those temperamental little cylinders (to use some diagram-speak, if I may). It's daring, free-thinking stuff."If this is true and ZFS is in fact shipping with Leopard, it may go a long way towards explaining how Time Machine will work in the final release, despite the fact that in the original developer release Time Machine did not use ZFS. It would be yet another example of Apple being out on the edge and leading the mainstream PC industry (linux hackers don't count) forward.[Via Digg through OSNews]
TUAW Tip: Mac OS X can verify itself now
Disk Utility is a great maintenance and troubleshooting tool for those times when your Mac starts acting up. Repairing permissions can sometimes be handy (even though its true validity is in question as of late), but using the Verify and Repair Disk operations is often a more useful tactic. However, until Mac OS X 10.4.3, you had use Disk Utility to verify or repair your Mac's drive from some sort of an external source. You either had to plug into another Mac in Target Disk mode, or reboot and use the install disks that came with your Mac - not the most friendly process to your workflow. But now, With 10.4.3 and later, you can start up Disk Utility right from within Mac OS X and chose the verify option to make sure system level elements like your file catalog are all in order. One warning though: it is completely normal for your Mac to temporarily freeze or 'seize up' at the beginning of running this operation (you should get the infamous beachball), as this Apple support document explains. After a little while, depending on your Mac's speed and all that jazz, it will come out of its daze and you can continue working/chatting/surfing while it finishes the disk verification process.This is a really handy tool, perhaps even more valuable than checking permissions, and should be a welcome addition to any Mac maintenance arsenal.
