Low bandwidth supercomputing
When television first broadcast in color back in 1954, it was fitting that the Tournament of Roses parade was the chosen feature presentation. Now according to my mother, it was met with cartwheel-spinning glee in her childhood home. It wasn’t just moving with the times, it was seeing presentations in a completely different way, although she’ll admit that her favorite shows were still in black & white.
Just as television evolved into a better presentation platform, so too does the World Wide Web. Let’s face it. In the early days, most who had access to a computer and a dial-up Internet connection could get all kinds of information on multiple things from around the world. News, weather, sports and health resources available on the web grew to become commonplace and now are most valued during times of crises.
One of those valued resources is the Supercourse, an expansive repository of lectures on global health and prevention designed to improve the teaching of prevention. Connecting a network of over 56,000 scientists in 174 countries, the core developers of Supercourse, Ronald LaPorte, Ph.D., Faina Linkov, Ph.D., and Eugene Shubnikov M.D., at the WHO Collaborating Center University of Pittsburgh — produced a free library of 5,100 lectures in 31 languages.
It was arguably the best kept secret for world health resources on the internet and its website had been broadcasting in black & white since 1986. Drastically falling behind the times, this website was no Rose Parade.
“In today’s age, having a professional-looking website is a key to attracting web traffic, as well as funding sources,” says Faina Linkov, “ While the old site was serving its mission of dissemination of materials, our big concern was that we can turn people away by having an outdated site.”
Struggling for years with funding and becoming buried in the ever-growing resources it was providing, the website was always put on the back burner, but it desperately needed an update. Turning to Pittsburgh Supercomputing Center (PSC) was a way to not only manage the future growth of the resource, but to put Supercourse in full technicolor almost overnight. Well, not literally overnight because it brought with it some interesting baggage.
Five PSC staff members, myself included, looked at the problem in stages. We determined this was a worthy project that needed a simple face lift and a database for holding all that information.
The challenge presented to us by Ron and his colleagues, however, was not trivial. We had to make the site attractive and easy to navigate. They couldn’t use modern programming code such as php or host their own database in their existing web space, we couldn’t change any of the existing URLs, to avoid breaking bookmarks, and I’ll never forget “it must be low bandwidth.” PSC’s team almost collectively fainted. We’re a SUPERcomputing center, we simply don’t know “slow”.
Now my mother would say, “Shandra, just make it pretty, you’re good at that.” That was the easy part of this project — to design a template and navigation that displays the hoard of resources consistently, loads quickly and can be seen from a third-world country on Netscape 3 using dial-up.
Did you hear what I said? … low bandwidth!
The bigger task, managing all that data, was handled by Rob L. and Deb N. at PSC. Because there was no good single data source for the Supercourse data, a database had to be created using data from multiple excel spreadsheets and from recursive passes through the html of the old version of the Supercourse site. The entire collection of 5100 lectures was updated manually with assistance from the team at Supercourse. Not being able to use scripts to serve the content meant that we had to create static pages for each of the thousands of lectures currently in Supercourse.
As the project went past its original three-month schedule and loomed into what we later realized was a year of data cleaning, processing, merging, and finding, just to get it into the database, Rob then had to create a method for the Supercourse team to update the website content in the future without going back to updating and adding lectures by hand in deprecated html from 1992.
Deb N. recalled the daunting process saying “Part of my job was to identify functions that were needed in the back-end, bother Rob to implement them, and then test them. I survived it very well. You may want to ask Rob how he felt about this part of it though. Sometimes I think he hid when he heard me coming down the hall.”
As it happens, Rob wasn’t hiding at all. He was finishing up the final pieces to the new launch by creating an administrative interface and a two-step update process for Pitt to use to update the site for years to come. PSC’s investment of time and effort into a modern database schema provided the glue to organize this resource.
“There is much that could be done,” says Nathan S., PSC team’s silent leader, “and at its core, Supercourse capitalizes on the notion of capturing lectures experts are already creating and may be willing to share with a bit of coaxing. People will come back to the site not because of how it is organized, per se, but because of the content of the lectures they find there.”
We’re all very enthusiastic about the completion of the “slick” update to the website but none more than our collaborators at the Supercourse. Ron LaPorte put it this way, “Our global health project is one of the tops in the world, now we are very proud to also have a world class web site.” And my mother would completely agree!
To learn more about the Supercourse, visit them on the “slick” web at www.pitt.edu/~super1.Note: The Supercourse collaboration with PSC is an ongoing project as we continue to improve the way researchers share and access global health prevention resources.