Federico Biancuzzi interviews Solar Designer, creator of the popular John the Ripper password cracker. Solar Designer discusses what's new in version 1.7, the advantages of popular cryptographic hashes, the relative speed at which many passwords can now be cracked, and how one can choose strong passphrases (forget passwords) that are harder to break.
Could you introduce yourself?
Solar Designer: For the past 9 years I've been spending much of my time on computer and network security. In particular, I've been developing free Unix security tools and other (non-security) software designed to be safe to use, as well as making existing software and technologies safer to use (discovering, dealing with, and sometimes publicizing vulnerabilities whenever that seemed appropriate). This is what the Openwall Project is about.
Although I am the author of most of the individual pieces of software released under Openwall, our biggest development project, Openwall GNU/*/Linux (or Owl for short), is a team effort which I am leading. Owl is a security-enhanced OS intended primarily for Internet servers.
What new features does the latest version 1.7 of John the Ripper include?
Solar Designer: The new "features" this time are primarily performance improvements possible due to the use of better algorithms (bringing more inherent parallelism of trying multiple candidate passwords down to processor instruction level), better optimized code, and new hardware capabilities (such as AltiVec available on PowerPC G4 and G5 processors).
In particular, John the Ripper 1.7 is a lot faster at Windows LM hashes than version 1.6 used to be. (Since JtR is primarily a Unix password cracker, optimizing the Windows LM hash support was not a priority and hence it was not done in time for the 1.6 release.) John's "raw" performance at LM hashes is now similar to or slightly better than that of commercial Windows password crackers such as LC5 - and that's despite John trying candidate passwords in a more sophisticated order based on statistical information (resulting in typical passwords getting cracked earlier).
John the Ripper 1.7 also improves on the use of MMX on x86 and starts to use AltiVec on PowerPC processors when cracking DES-based hashes (that is, both Unix crypt(3) and Windows LM hashes). To my knowledge, John 1.7 (or rather, one of the development snapshots leading to this release) is the first program to cross the 1 million Unix crypts per second (c/s) boundary on a general-purpose CPU. Currently, John 1.7 achieves up to 1.6M c/s raw performance (that is, with no matching salts) on a PowerPC G5 at 2.7 GHz (or 1.1M c/s on a 1.8 GHz) and touches 1M c/s on the fastest AMD CPUs currently available. Intel P4s reach up to 800k c/s. (A non-public development version making use of SSE also reaches 1M c/s on an Intel P4 at 3.4 and 3.6 GHz. I intend to include that code into a post-1.7 version.)
Additionally, John 1.7 makes an attempt at generic vectorization support for bitslice DES (would anyone try to set DES_BS_VECTOR high and compile this on a real vector computer, with compiler vectorizations enabled?), will do two MD5 hashes at a time on RISC architectures (with mixed instructions, allowing more instructions to be issued each cycle), and includes some Blowfish x86 assembly code optimizations for older x86 processors (the Pentium Pro family, up to and including Pentium 3) with no impact on newer ones due to runtime CPU type detection.
Speaking of the actual features, John 1.7 adds an event logging framework (John will now log how it proceeds through stages of each of its cracking modes - word mangling rules being tried, etc.), better idle priority emulation with POSIX scheduling calls (once enabled, this almost eliminates any impact John has on performance of other applications on the system), system-wide installation support for use by *BSD ports and Linux distributions, and support for AIX, DU/Tru64 C2, and HP-UX tcb files in the "unshadow" utility.
Finally, there are plenty of added pre-configured make targets with optimal settings, including ones for popular platforms such as Linux/x86-64, Linux/PowerPC (including ppc64 and AltiVec), Mac OS X (PowerPC and x86), Solaris/sparc64, OpenBSD on almost anything 32-bit and 64-bit, and more.
Of course, all platforms supported by John 1.6 (including plain x86 running most Unix-like systems, Win32, or DOS) are still supported. Similarly, pre-compiled binary distributions of John 1.7 for Win32 and DOS are made available.
Did you do any tests with projects such as BrookGPU and a modern GPU? Do you plan to try general-purpose computation using graphics hardware?
Solar Designer: No. I haven't seriously looked into this, but I don't expect GPUs to be very efficient at bitwise operations - which is what we need for John. If John would ever officially support algorithms involving floating-point operations, things could be different.
FPGAs are far more promising for this application (in fact, they're a guaranteed success).
Buying a 64bit cpu is now easy and cheap. How much does the availability of x86-64 CPUs affect the time needed to crack passwords?
Solar Designer: I am sorry to disappoint you, but this is not a major change for John.
As far as John is concerned, Intel's MMX was already 64-bit. It was not a true 64-bit architecture (most importantly, there was no 64-bit addressing), but in order to implement bitslice DES, only bitwise operations were needed - and those were available. The only things which change for John the Ripper on x86 with the availability of x86-64 CPUs are the ability to generate 64-bit code from C source (MMX code had to be hand-written or generated with self-made scripts), the availability of more 64-bit registers (15 usable instead of just 8), and the lack of an AND-NOT operation with x86-64. While the extra registers provide some speedup for bitslice DES, the existing MMX code is hand-optimized whereas the x86-64 code is currently being generated by C compilers. Overall, x86/MMX and x86-64 builds of John deliver similar performance, with differences for most hash types being within 20% (on the same CPU). As of this writing, the c/s rate for traditional crypt(3) is between 600k and 1 million for all modern x86 and x86-64 CPUs.