Jan
28
2013
28
Ten Years of TetrUSS on Mac OS X
With the ten year anniversary of Mac OS X in 2011, the recent ten year anniversary of Safari, and the recent 30 year anniversary of the Mac, another anniversary has been on my mind -- ten years of NASA Langley's TetrUSS CFD package on Mac OS X. In reality, it's been a bit longer than ten years since I started the unofficial Mac port of TetrUSS all the way back in 1997 (with the first "Rhapsody" release of Apple's then-new operating system). But the real work, stuff that directly relates to what we have today with Cocoa and other technologies in the modern Mac OS X, began with the public beta in 2000 and the official 10.0 "Cheetah" release in 2001, all leading to our first first public release of Mac TetrUSS in 2003. That's where the clock starts.
First some background material -- NASA's TetrUSS (which is short for "Tetrahedral Unstructured Software System") is a computational fluid dynamics (CFD) simulation package, mainly used for aerodynamic analysis. Over the years, TetrUSS has been used to analyze the aerodynamic performance of all types of vehicles, including aircraft, spacecraft, rockets, and cars, across the speed range from subsonic through hypersonic, and in atmospheres on Earth and other planets. I'll show some of these fascinating applications a bit later on.
TetrUSS originated with Neal Frink's Ph.D. dissertation back the late 80s, which focused on the development of what became the USM3D flow solver. From the beginning, TetrUSS was a team effort. While Neal was developing USM3D, Paresh Parikh and Shahyar Pirzadeh were developing enabling technologies for grid generation which would later become "VGRID". VGRID was a breakthrough in the sense that it allowed us to make quality, high fidelity grids on complex configurations in a short amount of time. One of their early achievements was a grid and aerodynamic simulation for the Boeing 747, which was used for a giant wall mural in the How Things Fly gallery at the National Air & Space Museum:
This gives a good idea of the complexity TetrUSS could model with ease, as far back as the early 90s. Back in that timeframe, the most complex grids were around 500,000 cells in size, and we ran simulations on a Cray supercomputer (typically for about a week per simulation). Grid generation, preprocessing, and postprocessing were done on SGI workstations. Though some CFD'ers worked with a single SGI on their desk, many of us had two computers -- an SGI for CFD work, and a Mac or PC for day to day stuff like making presentations, writing papers, plotting data, doing e-mail, and so on.
As we moved into the mid to late 90s, this dual computer approach was becoming expensive and tedious, and I started looking for other options. At the same time, PC class graphics cards were beginning to rival the capabilities in SGIs, so the SGI no longer held a unique niche. If our research group could move towards a single desktop workstation that served both engineering and office needs, we would save a lot of money and I believed productivity would improve.
There were a few options. Windows was strong on the office side and had great support for basic engineering software, but totally lacked capabilities for high-end research like CFD where a UNIX based OS was a must. Mac OS was also strong for office use, especially publishing, but was lacking in both basic and high-end engineering support. Linux was a strong option for engineering, but it suffered badly on the office side. We looked at options for putting UNIX and Linux on Macs, we looked at dual-boot Windows/Linux systems, and we looked at running emulators under Linux. None of these solutions was really ideal.
Then, in the midst of Apple's meltdown and rebirth in the late 90s, the path to Mac OS X was born. And it looked good, bringing the promise of native office productivity and publishing software to a UNIX based OS. So I started a bit of a skunkworks project to port and test our CFD tools on Mac OS X. It was tough in the early days, and FORTRAN compiler support was starting from scratch on this new OS. But by the time Cheetah shipped in 2001, we had some decent options. I was able to port USM3D over to OS X and run test cases with decent performance. With some tuning to take advantage of AltiVec on the G4 processor, performance not only exceeded our older SGIs, but started rivaling our Cray supercomputers (albeit limited to single precision when using AltiVec). Things looked pretty good for the flow solver.
On the grid generation side, we had a tough time porting software since early versions of Mac OS X had very primitive OpenGL and X11 support. It wasn't until Mac OS X 10.2 arrived in 2002 that things really started cooking, with good implementations of OpenGL, GLUT, and X11, all with proper hardware acceleration. In fact, 2002 was a watershed year for Mac TetrUSS for many reasons. First, we started seeing commercial CFD tools show up for Mac OS X, like Tecplot. Second, unbeknownst to me, an engineer at the Air Force Research laboratory by the name of Matt Grismer was working on his own skunkworks project to port some of the TetrUSS software. Matt had used a NeXT workstation in grad school, and was excited enough by Mac OS X to work on ports of both VGRID and GridTool, a preprocessing app originally developed by Jamshid Samareh at NASA Langley.
GridTool is about the most complicated piece of software I have ever worked with, evolving out of the complex needs of grid generation in the early to mid 90s. The interface has dozens of panels and hundreds of buttons (and believe it or not, we tend to use most of them on a routine basis):
Click for larger image
Jamshid put a monumental effort into developing the original GridTool software for SGI, and Matt devoted an equally monumental effort porting that software over to Cocoa. To this day, it is the single most complex one-man Cocoa port I am aware of, and I consider that port to be pivotal in the eventual success of Mac TetrUSS. Over the years, we have added to Matt's original port, tweaking the interface and putting in many Cocoa-specific features and capabilities that weren't even possible back on SGI.
So with all the pieces in place, 2003 was our first full public release of TetrUSS for Mac OS X, and response was amazing. We had hundreds of requests for the software in the first week, enough that it swamped NASA's software release queue for months. Since then, Mac TetrUSS has been used by thousands of CFD'ers in government, industry, military, and academia across the USA.
Besides being a big year for us, 2003 was a big year for Mac OS X in scientific computing. FORTRAN compiler support was maturing nicely on Mac OS X, with good options from Absoft and NAG. Mid-year, the excellent XL compilers from IBM joined the party when the Power Mac G5 was announced. Thanks to our dedicated Apple rep Russ Pond, we were able to secretly get a pre-production G5 system for benchmarking and testing the IBM XL compilers. I could write a lot about that experience alone, but let's just say it was a blast -- as any secret project embedded in a skunkworks project would be. The G5 was the first PowerPC system in many years that could blow away x86 hardware in terms of scalar floating point calculations in both single and double precision, and this was especially important for CFD where double precision is required for many full scale flight problems. Combined with the XL Fortran compiler and all of the capabilities in OS X 10.3, the G5 was the ultimate desktop CFD workstation I envisioned back in the late 90s. Things were finally in full swing on Mac OS X.
Our hard work was recognized in 2004, with both an Apple Design Award for "Best Scientific Computing Solution on Mac OS X" and NASA's Software of the Year Award (the second for TetrUSS, which first won in 1996). As fate would have it, the Apple Design Award ceremony was happening at WWDC in California the night before Neal Frink and I were making a presentation to compete for the NASA Software of the Year Award in West Virginia. We sent Matt Grismer out to WWDC with instructions to call us as soon as results were announced, so that we could mention it the next day (I had special charts pre-printed just in case we won).
Neal and I rehearsed our NASA presentation in cool evening air of West Virginia, said good night, and headed off to our hotel rooms to get some sleep. Upon checking my voicemail, I heard an excited Matt telling me we won the Apple Design Award, complete with rock music blaring in the background (just like you'd expect from a developer's conference in California). The contrast between West Virginia and California couldn't have been more stark or more amusing. To this day, that is one of the most memorable voicemails I have gotten.
I'll go into some applications of TetrUSS shortly, but this is a good time to point out that no part of the Mac TetrUSS project at NASA was ever planned, funded, or officially staffed, nor was it architected like a typical software project. It was all done by motivated people on their own time in a variety of skunkworks projects. I take that as a valuable lesson when I see the massive amount of planning, manpower, management, and coordination that goes into most software projects (some of which will fail, head off into vaporware, or lead nowhere). Planning and management definitely has its place, but real innovation and progress often happens off the books and without management oversight, driven by curiosity, enthusiasm, and determination.
Over the years, we got some great support for the Mac TetrUSS project from Apple. Russ Pond and Mike Mills always gave great sales and engineering support, and we also got a lot of help from various folks on Apple's SciTech marketing team. Richard Crandall, head of Apple's Advanced Computation Group, and Ian Ollmann of Apple's Vector and Numerics group, were huge helps with vectorization and numerical schemes. Sadly, Richard passed away in late 2012. Back in the thick of early Mac TetrUSS development, Richard and I had some monumental phone calls and e-mail exchanges, at all hours of the day and night, from the office and from home, covering all manner of topics on mathematics, physics, and coding. I enjoyed every minute of it -- Richard was a one of a kind, and I always felt grateful to work with him.
OK, on to some applications of Mac TetrUSS. I won't go into too much engineering detail, or even include plot legends -- I want the images to speak for themselves in terms of the beauty of aerodynamics and fluid dynamics simulations.
This image shows Mach Number contours around a rocket-powered robotic airplane meant to scout Mars. Aircraft flying in the atmospheres of other planets will become key exploration tools in the future, looking at things like atmospheric chemistry, geology, magnetic fields, and mapping.
Surface pressure contours and stream-trace ribbons on a generic advanced fighter concept that has become the unofficial test case and mascot for Mac TetrUSS. We've run this case on everything from iBooks and Mac Minis up to Mac Pros and racks of Xserves.
Surface pressure contours and surface oil-flow lines on the vertical tail and rudder of American Airlines Flight 587, which crashed on November 12, 2001 in Belle Harbor NY, shortly after takeoff from JFK. NASA TetrUSS team members worked with the NTSB to compute and analyze aerodynamic loads, responding to a critical need on a tight schedule.
Surface pressure contours and computational mesh on an unmanned aerial vehicle concept.
Mach Number around the Ares 1-X experimental launch system during stage separation. Ares 1-X successfully launched from Cape Canaveral on October 28, 2009.
VGRID computational mesh of a C-130 and cargo release parachute.
Created by a student at the US Air Force Academy.
Concept, test, flight sequence of an advanced quiet nozzle jointly developed by NASA and Boeing. From left to right, TetrUSS-computed pressure loads on the nacelle and nozzles, testing in Boeing's Wind Tunnel, flight test on a Boeing 777.
Surface pressure (directly related to road drag) on a semi truck.
VGRID computational mesh of an F-18 aircraft.
Surface pressure on a green "hybrid wing body" passenger aircraft concept. This type of aircraft could someday replace traditional "tube and wing" transport aircraft with a configuration that is quieter, more fuel efficient, and more environmentally friendly.
Simulation of flow around the Orion crew module and launch abort system (LAS). Orion is a new exploration vehicle that will carry NASA astronauts beyond low earth orbit, into deep space. The module and LAS provide emergency abort capability, sustain the crew during space travel, and provide safe re-entry from deep space return velocities. Look for the first test flight of Orion in 2014.
First some background material -- NASA's TetrUSS (which is short for "Tetrahedral Unstructured Software System") is a computational fluid dynamics (CFD) simulation package, mainly used for aerodynamic analysis. Over the years, TetrUSS has been used to analyze the aerodynamic performance of all types of vehicles, including aircraft, spacecraft, rockets, and cars, across the speed range from subsonic through hypersonic, and in atmospheres on Earth and other planets. I'll show some of these fascinating applications a bit later on.
TetrUSS originated with Neal Frink's Ph.D. dissertation back the late 80s, which focused on the development of what became the USM3D flow solver. From the beginning, TetrUSS was a team effort. While Neal was developing USM3D, Paresh Parikh and Shahyar Pirzadeh were developing enabling technologies for grid generation which would later become "VGRID". VGRID was a breakthrough in the sense that it allowed us to make quality, high fidelity grids on complex configurations in a short amount of time. One of their early achievements was a grid and aerodynamic simulation for the Boeing 747, which was used for a giant wall mural in the How Things Fly gallery at the National Air & Space Museum:
This gives a good idea of the complexity TetrUSS could model with ease, as far back as the early 90s. Back in that timeframe, the most complex grids were around 500,000 cells in size, and we ran simulations on a Cray supercomputer (typically for about a week per simulation). Grid generation, preprocessing, and postprocessing were done on SGI workstations. Though some CFD'ers worked with a single SGI on their desk, many of us had two computers -- an SGI for CFD work, and a Mac or PC for day to day stuff like making presentations, writing papers, plotting data, doing e-mail, and so on.
As we moved into the mid to late 90s, this dual computer approach was becoming expensive and tedious, and I started looking for other options. At the same time, PC class graphics cards were beginning to rival the capabilities in SGIs, so the SGI no longer held a unique niche. If our research group could move towards a single desktop workstation that served both engineering and office needs, we would save a lot of money and I believed productivity would improve.
There were a few options. Windows was strong on the office side and had great support for basic engineering software, but totally lacked capabilities for high-end research like CFD where a UNIX based OS was a must. Mac OS was also strong for office use, especially publishing, but was lacking in both basic and high-end engineering support. Linux was a strong option for engineering, but it suffered badly on the office side. We looked at options for putting UNIX and Linux on Macs, we looked at dual-boot Windows/Linux systems, and we looked at running emulators under Linux. None of these solutions was really ideal.
Then, in the midst of Apple's meltdown and rebirth in the late 90s, the path to Mac OS X was born. And it looked good, bringing the promise of native office productivity and publishing software to a UNIX based OS. So I started a bit of a skunkworks project to port and test our CFD tools on Mac OS X. It was tough in the early days, and FORTRAN compiler support was starting from scratch on this new OS. But by the time Cheetah shipped in 2001, we had some decent options. I was able to port USM3D over to OS X and run test cases with decent performance. With some tuning to take advantage of AltiVec on the G4 processor, performance not only exceeded our older SGIs, but started rivaling our Cray supercomputers (albeit limited to single precision when using AltiVec). Things looked pretty good for the flow solver.
On the grid generation side, we had a tough time porting software since early versions of Mac OS X had very primitive OpenGL and X11 support. It wasn't until Mac OS X 10.2 arrived in 2002 that things really started cooking, with good implementations of OpenGL, GLUT, and X11, all with proper hardware acceleration. In fact, 2002 was a watershed year for Mac TetrUSS for many reasons. First, we started seeing commercial CFD tools show up for Mac OS X, like Tecplot. Second, unbeknownst to me, an engineer at the Air Force Research laboratory by the name of Matt Grismer was working on his own skunkworks project to port some of the TetrUSS software. Matt had used a NeXT workstation in grad school, and was excited enough by Mac OS X to work on ports of both VGRID and GridTool, a preprocessing app originally developed by Jamshid Samareh at NASA Langley.
GridTool is about the most complicated piece of software I have ever worked with, evolving out of the complex needs of grid generation in the early to mid 90s. The interface has dozens of panels and hundreds of buttons (and believe it or not, we tend to use most of them on a routine basis):
Jamshid put a monumental effort into developing the original GridTool software for SGI, and Matt devoted an equally monumental effort porting that software over to Cocoa. To this day, it is the single most complex one-man Cocoa port I am aware of, and I consider that port to be pivotal in the eventual success of Mac TetrUSS. Over the years, we have added to Matt's original port, tweaking the interface and putting in many Cocoa-specific features and capabilities that weren't even possible back on SGI.
So with all the pieces in place, 2003 was our first full public release of TetrUSS for Mac OS X, and response was amazing. We had hundreds of requests for the software in the first week, enough that it swamped NASA's software release queue for months. Since then, Mac TetrUSS has been used by thousands of CFD'ers in government, industry, military, and academia across the USA.
Besides being a big year for us, 2003 was a big year for Mac OS X in scientific computing. FORTRAN compiler support was maturing nicely on Mac OS X, with good options from Absoft and NAG. Mid-year, the excellent XL compilers from IBM joined the party when the Power Mac G5 was announced. Thanks to our dedicated Apple rep Russ Pond, we were able to secretly get a pre-production G5 system for benchmarking and testing the IBM XL compilers. I could write a lot about that experience alone, but let's just say it was a blast -- as any secret project embedded in a skunkworks project would be. The G5 was the first PowerPC system in many years that could blow away x86 hardware in terms of scalar floating point calculations in both single and double precision, and this was especially important for CFD where double precision is required for many full scale flight problems. Combined with the XL Fortran compiler and all of the capabilities in OS X 10.3, the G5 was the ultimate desktop CFD workstation I envisioned back in the late 90s. Things were finally in full swing on Mac OS X.
Our hard work was recognized in 2004, with both an Apple Design Award for "Best Scientific Computing Solution on Mac OS X" and NASA's Software of the Year Award (the second for TetrUSS, which first won in 1996). As fate would have it, the Apple Design Award ceremony was happening at WWDC in California the night before Neal Frink and I were making a presentation to compete for the NASA Software of the Year Award in West Virginia. We sent Matt Grismer out to WWDC with instructions to call us as soon as results were announced, so that we could mention it the next day (I had special charts pre-printed just in case we won).
Neal and I rehearsed our NASA presentation in cool evening air of West Virginia, said good night, and headed off to our hotel rooms to get some sleep. Upon checking my voicemail, I heard an excited Matt telling me we won the Apple Design Award, complete with rock music blaring in the background (just like you'd expect from a developer's conference in California). The contrast between West Virginia and California couldn't have been more stark or more amusing. To this day, that is one of the most memorable voicemails I have gotten.
I'll go into some applications of TetrUSS shortly, but this is a good time to point out that no part of the Mac TetrUSS project at NASA was ever planned, funded, or officially staffed, nor was it architected like a typical software project. It was all done by motivated people on their own time in a variety of skunkworks projects. I take that as a valuable lesson when I see the massive amount of planning, manpower, management, and coordination that goes into most software projects (some of which will fail, head off into vaporware, or lead nowhere). Planning and management definitely has its place, but real innovation and progress often happens off the books and without management oversight, driven by curiosity, enthusiasm, and determination.
Over the years, we got some great support for the Mac TetrUSS project from Apple. Russ Pond and Mike Mills always gave great sales and engineering support, and we also got a lot of help from various folks on Apple's SciTech marketing team. Richard Crandall, head of Apple's Advanced Computation Group, and Ian Ollmann of Apple's Vector and Numerics group, were huge helps with vectorization and numerical schemes. Sadly, Richard passed away in late 2012. Back in the thick of early Mac TetrUSS development, Richard and I had some monumental phone calls and e-mail exchanges, at all hours of the day and night, from the office and from home, covering all manner of topics on mathematics, physics, and coding. I enjoyed every minute of it -- Richard was a one of a kind, and I always felt grateful to work with him.
OK, on to some applications of Mac TetrUSS. I won't go into too much engineering detail, or even include plot legends -- I want the images to speak for themselves in terms of the beauty of aerodynamics and fluid dynamics simulations.
Created by a student at the US Air Force Academy.