Reconnecting with the Past, Training the Future

May 18, 2012 Leave a comment

Dr. Anirban Jana

Throughout our lives, people come and go.  Sometimes we stay in touch; more often, especially if both parties are not avid social media users, we lose track of all but our closest friends.  A chance meeting in a new city with someone from an earlier stage of life can renew a friendship, but it’s meant more than that for Anirban Jana and Mark Kimber.  Reconnecting in Pittsburgh has led them to a professional collaboration and a grant of nearly $900,000 from the DOE’s Nuclear Energy University Programs (NEUP).

Jana, a Sr. Scientific Specialist at the PSC, came to Pittsburgh in the spring of 2008.  Some months later, he was browsing through the University of Pittsburgh mechanical engineering department web site when he noticed a Mark Kimber, an assistant professor in the Department of Mechanical Engineering and Materials Science, on the list.  He wondered if it could be the same Mark Kimber he shared an office with at Purdue when he was a graduate student in mechanical engineering.   “It’s not like that is a very common name,” says Jana.  He sent Kimber an email; it was his old friend. The two began planning projects that they could work on together.

Dr. Mark Kimber

Photo courtesy of the University of Pittsburgh

“He is an experimentalist, and I am a computational person, so that was a good fit,” says Jana.  Both skill sets will be called on in the work funded by the NEUP.  Along with John Brigham, Assistant Professor in the Department of Civil and Environmental Engineering at Pitt and Milorad Dzodzo, PhD, of Westinghouse Electric Company, they will computationally model the turbulent mixing in the lower plenum of Very High Temperature Reactors (VHTR) and validate their results experimentally.

The VHTR is one of the Generation IV nuclear energy systems being studied by the Generation IV International Forum(GIF), a collaboration of the world’s leading nuclear technology nations. GIF has defined eight technology goals for generation IV systems in the areas of sustainability, economics, safety and reliability, and “proliferation resistance” – which means that the fuel used should be undesirable for nuclear weapons. Eight of the GIF signatories have agreed to participate in the development of one or more of the generation IV systems; the United States will study the VHTR.

One benefit of the VHTR design is its ability to supply “process heat” to other industries.  Other industries — the chemical industry in general, for example, says Jana — require heat for their processes to proceed.   Previous reactor designs operated at lower temperatures, and were unable to provide sufficient heat for these purposes.

In the VHTR, helium absorbs heat from the nuclear reaction, is expelled into part of the reactor vessel called the lower plenum, and then passes into a heat exchanger where it gives up that heat to be used as process heat.   Helium spews into the lower plenum at speeds of up to 100 meters per second, and at temperatures approaching 1000° C.   The temperature can vary by as much as 100°C among individual helium jets.  The speed of the gas and the temperature differences cause the turbulent mixing that Jana, Kimber and their co-PIs will study.

VHTR model. The lower plenum is the yellow area to the left of the human figure. Helium enters the lower plenum at high speeds and temperatures, then passes into the heat exchanger. It cools and then re-enters the reactor vessel.
Courtesy of Idaho National Laboratory

The project will also train future nuclear engineering experts.  Students involved in the research will have the opportunity to intern at Westinghouse under Dzodzo.   There is an age gap in nuclear expertise in the U.S., says Jana, and one of the goals of NEUP is to train the next generation of leaders in the U.S. nuclear industry.  This training extends to the project team, according to Jana. “Three of us, me and Mark and John from Pitt, we are just starting out.   But Dr. Dzodzo, from Westinghouse, he is an established expert, so we have both new and old blood in the team.”

OpenFOAM, an open source Computational Fluid Dynamics (CFD) code, will be used for the initial simulations.   The team has also proposed to use FLUENT and STAR-CCM, commercial CFD packages.  “We may need to verify something that we did on OpenFOAM:  is it right or not?” says Jana, and one way to verify computational results is to run the simulation with  different software and see if the results match.  Because Fluent and STAR-CCM are available at Westinghouse,  “That is another point of collaboration between Westinghouse and us,” says Jana. “Students can go to Westinghouse in the summer and run those commercial codes on their machines.” Once a computational model is built, it will be experimentally verified.

The thermal mixing simulation will be restricted to just a portion of the lower plenum, including some of its support structure.   Simulating mixing in the entire lower plenum is too complicated for this initial study, says Jana. For now, they will also ignore stresses on the plenum structure caused by the temperature differences.

“If you heat up one side of a rod, and the other side is cold, it will bend.  That means there are stresses being generated,” Jana says.  So similarly, if you have temperature fluctuations in a structure, that means you have fluctuating stresses.”

Knowing how the lower plenum is being stressed can help to predict its useful lifetime, but those calculations are beyond the scope of this project.

Jana says,“Later on, we’ll feed [the results of the thermal mixing study] into a stress computation model.  We haven’t proposed to simulate the whole lower plenum right now, because that’s too big.  We have no idea, actually, how much computational power would be required to simulate the whole lower plenum in all its glorious detail.  Full lower plenum simulations as well as stresses, those are ideas for the future.”

Big Props to Our Systems Support Team!

May 4, 2012 Leave a comment

Recently our Systems Support Team was honored with the Mellon of College of Science’s Outstanding Achievement award!  The award was presented at last week’s award ceremony held in the historic Mellon Institute here on CMU’s campus.  Here’s the team consisting of Joe Lappa, Steve Cunningham, Shane Filus, Ken Goodwin, Brian Johanson, Adam Fest, Ray Nardozzi, Bryan Webb and Ed Hanna (not in photo e Susan Litzinger, Jason Sommerfield, and Clint Perrone)  along with the other staff award winners:

2012 Mellon College of Science award winners.

You may be wondering what exactly these peeps did to deserve an award?  The simple answer is their jobs.  The more complicated answer:

The members of the systems and operations group worked with networking and business departments to facilitate a massive reorganization of our machine room.   We’ve been around for 25+ years and every once in a while we’ve got to reassess and adjust things to account for our ever changing needs.  The team updated our networking configuration which includes over a mile of new Ethernet cable to create over 400 new connections in our machine room.  They also managed power distribution and air handling improvements to better handle future infrastructure growth.   All of these configuration changes meant that the team also had to tweak the floor plan as well. This required that some of our machines get new homes, in different areas of the machine room. Speaking of new, the team also installed 5 peta-bytes of new storage equipment!

I don’t know about you but my eyes are starting to glaze over just at the logistical challenges involved in such improvements!  So while the earlier simpler answer was they were just doing their jobs, this group of PSC staffers are Supermen/Superwoman in our book.  They ensured that all production systems remained up and functional during this complicated reorganization, all while the rest of us kept doing what we do, uninterrupted.

So thanks for being so awesome at your jobs Joe, Steve, Shane, Ken, Brian, Adam, Ray, Bryan, Ed, Susan, Jason and Clint!

Categories: Uncategorized

Workshop Recap

April 24, 2012 Leave a comment

Last week we were happy to work with NVIDIA and PGI this to produce the OpenACC workshop:  Accelerating Applications with OpenACC. Participants learned how to use OpenACC to simplify their GPGPU code.  Using OpenACC directives in standard C and Fortran, programmers insert compiler hints into their code to automatically execute compute-intensive regions of code on accelerators, simplifying code development and improving performance portability.  After the workshop one participant reported his results in an email:

After two days of hacking on a kernel “toy” prototype of the gradient routine from our Discontinuous Galerkin shallow water model, I measured a 10x speedup (30 Gflops vs 3 Gflops) … using OpenACC.

10x speed up… not bad for a two day workshop! Helping to enable these performance improvements were NVIDIA staffers Carl Ponder and Sarah Tariq and PSCer John Urbanic who taught the workshop sessions.  PSC staffers Marcela Madrid,  Roberto Gomez, Anirban Jana and Tom Maiden were also there to assist.

The workshop took place in the David Deerfield Training Center in our office space in Oakland.  We had participants from over 15 different institutions and even a few staff members sat in on the fun.

If you’d like more information about OpenACC or would like to talk to one of our staff members, email us at and we’d love to talk to you about it!

Congratulations to Yang Wang!

April 20, 2012 1 comment

One of our very own, Yang Wang, was recently honored with membership in the National Physics Honor Society Sigma Pi Sigma!!  This membership was in recognition of Yang’s mentorship of Duquesne University student Alex Arrico during his internships at PSC the past two summers.  Also, we’d like to give a special shout out to Alex and congratulate him as well. He was accepted as a graduate student in the Materials Science Department at the University of Tennessee.  Congratulations to both Yang and Alex!

Categories: People Tags: ,

Unleashing the Power of Data at PSC

April 13, 2012 Leave a comment

On a beautiful April day, we opened our doors and let the spring air in.  Streaming in along with the cool breeze came people from many walks of life to learn about what resources and services PSC can offer them.  We met faculty and researchers from both Carnegie Mellon University and the University of Pittsburgh;  students from both CMU and Pitt and a large contingent from nearby Slippery Rock University; local and some not-so-local K-12 educators;  and even some students newly admitted to Carnegie Mellon in town for a campus visit.

They came to learn about the resources and services that PSC offers. Some tucked away knowledge about our new Data Supercell, our high capacity, high reliability, low latency “near-line” data management system.  Others processed information about the Digital Manufacturing initiative, offering modeling and simulation tools for product design, analysis, prototyping and testing  to small and mid-size Pennsylvania businesses.   The K-12 crowd were educated about the resources and training we provide to STEM (Science, Technology, Engineering and Math) teachers.

Those interested in biomedical matters connected with PSC scientists involved in mapping brain pathways, and saw how Anton, our innovate computing system for biomedical scientists,  enables us to see molecular dynamics in action.    Our close friends and collaborators from the MIDAS Center of Excellence at the University of Pittsburgh spread information about their study of infectious disease dynamics and prevention.  Scientists and science aficionados were wowed over research using Blacklight ; some got sucked in by MassiveBlack, a simulation of the formation of supermassive black holes in the early days of the universe, while others came to understand how Blacklight helped researchers test better ways to translate languages.

Others  connected with our networking group over 3ROX, our GigaPoP, or hastened to talk with the Web10G crew about improving network speeds.

And everyone enjoyed the refreshments!

We hope you joined us… if you didn’t, here are some pictures of what you missed.  Look for an announcement next spring about our next open house.  We really hope to see you there!

Greg Hood explains how supercomputing is used in biomedical research.

Katie Phillips discusses the research done at the MIDAS Center of Excellence with Discover12 attendees.

Steve Cunningham and Bryan Webb talk to an attendee about 3ROX, which
provides high speed commodity and research network access.

Students discuss high-performance computing over snacks.

Categories: General, Outreach, People

Discover12: Who’s Ready to Unleash the Power of Data?

April 6, 2012 Leave a comment

We certainly are!  Discover12 is just a week away and we’ve been very busy planning exciting demonstrations and science features with a special emphasis on data to share with you next week when you visit us in Oakland.

We will have something for just about everyone. K-12 teachers and administrators will find projects that can enhance their curricula with PSC STEM education programs; scientific and university researchers will be able to talk to PSC staff about HPC resources and their current research projects; corporations will find out how HPC can help give them the competitive edge they’ve been looking for.

To participate in our open house, all you need to do is let us know you’re coming by completing our RSVP form, and then just show up next Friday, April 13th.

We’re looking forward to seeing everyone next week!

XSEDE Outreach in the Burgh

March 30, 2012 Leave a comment

It seemed like everyone has been taking some sort of spring break so we treated ourselves with a little blog spring break!  We’re back this week to tell you a little bit about the National Science Foundation’s XSEDE (pronounced like exceed)  project.  You can find the official overview of what XSEDE is here.  The Cliff Notes version of the overview is:  In 2011 NSF gave about 16 institutions $121 million dollars over 5 years to provide high performance computing resources and services to scientists and researchers.

One of PSC’s roles in the XSEDE project is to coordinate the Outreach efforts.  Specifically one of the projects is the Speakers Bureau which focuses on coordinating XSEDE presence at various conferences and speaking events with the ultimate goal of recruiting new communities of users.

As part of this initiative XSEDE has participated in the Grace Hopper Celebration of Women in Computing, Emerging Researchers Conference, and American Physical Society’s March Meeting just to name a few.  This week we were excited to participate in the National Society for Black Engineer’s Conference right in here Pittsburgh.

We were really excited about participating in this conference for two reasons:

1) We had to travel a whopping 20 minutes from our offices to get to the convention.

2) The conference had a lot of great talks and activities scheduled and would be a good opportunity for us to meet some potential new users.

XSEDE booth at the NSBE conference in Pittsburgh.

If you’re anything like me, you’re probably thinking, oh geez they’re probably those annoying people at conferences that stand there and use gimmicky things to lure you in and then peg you in a corner and talk to you for 20 minutes about something you lost interest in after 2 minutes.

 Don’t worry, that’s not us!  We’re pretty laid back, and we’re there to tell you all about our XSEDE programs that may be of interest to you.  We even have some great visualizations that we bring with us (they’re not in the picture since we don’t leave our electronic equipment in the booth unattended) to show potential new users some examples of what kind of awesome results some of our users are getting.  In addition to the cool science we like to show off, sometimes we even have candy  XSEDE notepads or collapsible water bottles for people that stop by and talk to us.

So next time you’re at a conference check to see if XSEDE is there and we’ll be happy to talk to you about our resources and services, and if you’re lucky we might even have a fun giveaway or some candy to send you home with!

Categories: XSEDE

Get every new post delivered to your Inbox.