XSEDE12

TUTORIAL: Introduction to BigJob - A SAGA-Based Interoperable, Extensible and Scalable Pilot-Job for XSEDE

ABSTRACT: The SAGA-based Pilot-Job, known as BigJob, provides the unique capability to use Pilot-Jobs on the highest-performing machines as well as collectively on distributed cyberinfrastructure. It supports very large-scale parallel jobs, as well as high throughput of many smaller jobs. In addition to the number and range of job sizes that it supports, what makes BigJob unique among all Pilot-Jobs is its ability to be programmatically extended to support a range of "simple workflows," provide application-level control of both the Pilots and the tasks assigned to the Pilot-Job, and its interoperability over all XSEDE and OSG platforms. This half-day tutorial will bring together SAGA team members, XSEDE staff, and XSEDE end users (scientists using BigJob) to deliver: The basic concepts behind Pilot-Jobs, several science exemplars that routinely use BigJob on XSEDE for extreme-scale science, introduction on how to use BigJob, how to use BigJob on XSEDE and OSG, how to program and customize BigJob for your needs, building frameworks using BigJob, and advanced concepts and application-level scheduling using BigJob.

REQUIRES: Laptop

ABSTRACT: The Appro Gordon Compute Cluster was put into production at the San Diego Supercomputer Center (SDSC) in early 2012. In addition to providing academic users with their earliest access to the Intel EM64T Xeon E5 (Sandy Bridge) processor, it contains a number of unique features that make it ideal for data-intensive applications. All current and potential Gordon users are invited to attend, but we especially encourage practitioners from domains that have not traditionally been major users of NSF compute resources (e.g., political science, linguistics, economics, finance, data analytics, and sociology) to participate. The tutorial covers the Gordon architecture, the types of applications that are ideally suited for the system, and how to run jobs and get the best performance on the system.

PREREQUISITES: Linux experience

BOF: "How Did We Get Here?" Gordon Design and Planning

Appro will hold a drawing for a $200 amazon.com gift card at this BOF. Details available at the event.

Steve Lyness and Shawn Strande

Abstract -- Learn how Appro, Intel and San Diego Supercomputing Center (SDSC) at the University of California collaborated for a major design win with a skip-generation architecture called “Gordon” Supercomputer three years in advance of the system being deployed. Learn how this early preparation resulted in a grant from the National Science Foundation (NSF) to allow this system to be built and be available to offer a powerful supercomputer resource dedicated to solving critical science and societal problems using advanced HPC technology. Discover what is behind the Gordon design innovations, the processors, flash memory, interconnect network and the entire system configuration. Explore the ideas and planning of how industry trends, partnerships, early access to future technology roadmaps and system configuration adjustments were used to extrapolate to the time the system would actually be built. Learn how reliability, availability, manageability and system configuration compatibility were essential for this successful data intensive supercomputer be able to deliver over 200 TFlops of peak performance and up to 35M IOPS from 300TB of Solid State Storage. Also, learn how Gordon’s scientific applications benefit from fast interaction and manipulation of large volumes of structured data and how Gordon is helping the HPC research community by being available through an open-access national grid.

BOF by Gold Sponsor - Penguin Computing. Discussion lead: Matt Jacobs