|
|
|
What is High Performance Storage System? HPSS is software that manages
petabytes of data on disk and robotic tape libraries.
HPSS provides highly flexible and scalable hierarchical storage management
that keeps recently used data on disk and less recently used data on tape.
HPSS uses cluster, LAN and/or SAN technology to aggregate the capacity and performance
of many computers, disks, and tape drives into a single virtual file system
of exceptional size and versatility. This approach enables HPSS to easily meet
otherwise unachievable demands of total storage capacity, file sizes, data rates,
and number of objects stored. HPSS provides a variety of user and filesystem interfaces
ranging from the ubiquitous vfs, ftp, samba and nfs to higher performance pftp,
client API, local file mover and third party SAN (SAN3P). HPSS also provides hierarchical storage management (HSM) services for IBM General Parallel File System (GPFS).
Who developed HPSS? HPSS is the result of over a decade
of collaboration among five Department of Energy laboratories and IBM,
with significant contributions by universities and other laboratories worldwide.
Jointly developing open software with end users that have the most demanding
requirements and decades of high-end computing and storage system experience
has proven the recipe for continued innovation, industry leadership and leading
edge technology since 1992. The founding collaboration partners are
- IBM Global Services
- Lawrence Livermore National Laboratory
- Los Alamos National Laboratory
- National Energy Research Supercomputer Center (NERSC) at Lawrence Berkeley National Laboratory
- Oak Ridge National Laboratory
- Sandia National Laboratories
Why HPSS?
- Scalable Capacity - As architects continue to exploit hierarchical storage
systems to scale critical data stores beyond a petabyte (1024 terabytes)
towards an exabyte (1024 petabytes), there is a equally critical need to
deploy a high performance, reliable and scalable HSM.
- Scalable I/O Performance - As processing capacity grows from trillions
of operations per second towards a quadrillion operations per second and
data ingest rates grows from 10s of terabytes per day to 100s of terabytes per day,
HPSS provides a scalable data store with reliability and performance to sustain
24x7 operations in demanding high availability environments.
- Incremental Growth - As data stores with 100s of terabytes and petabytes
of data become increasing common place, HPSS provides a reliable and flexible solution
that scales seamlessly using heterogeneous storage devices, robotic tape libraries,
and processors connected via LAN, WAN and SAN.
- Reliability - As data stores increase in from 10s of million of files to 100s
of millions of files and collective storage I/O rates grow to 100s of terabyte per day,
HPSS provides a scalable metadata engine based on IBM's DB2 to ensures highly reliable
and recoverable transactions down to the I/O block level.
Who has a petabyte or more of data? These HPSS Collaboration Members' sites have
accumulated a petabyte or more of data, in a single HPSS file system.
Some have passed three petabytes, heading for four. One site has reached 6 petabytes.
- Brookhaven National Laboratory (BNL)
- Commissariat à l'Energie Atomique/Direction des Applications Militaires (CEA/DAM) Compute Center in France
- European Centre for Medium-Range Weather Forecasts (ECMWF) in the United Kingdom
- Institut National de Physique Nucléaire et de Physique des Particules (IN2P3) in France
- Lawrence Livermore National Laboratory (LLNL)
- Los Alamos National Laboratory (LANL)
- National Centers for Environmental Prediction (NCEP)
- National Energy Research Scientific Computing Center (NERSC)
- Oak Ridge National Laboratory (ORNL)
- Institute of Physical and Chemical Research (RIKEN) in Japan
- San Diego Supercomputer Center (SDSC)
- Stanford Linear Accelerator Center (SLAC)
What do HPSS users store? HPSS provides storage management
for a diverse set of digital library, science, engineering and defense applications
and safeguards a range of data including nanotechnology, genomics, chemistry,
biochemistry, radiology, functional magnetic resonance imaging, fusion energy,
energy efficiency, astrophysics, nuclear physics, accelerator physics, geospatial data,
digital audio, digital video, weather, climate, and computational fluid dynamics.
Contact Information
Information about the HPSS Collaboration
Information about acquiring HPSS
|
|
|
|
|
|
What's New With HPSS
|
 |
2008 HPSS Users Forum
After receiving feedback from the HPSS user community, Oak Ridge National Lab(ORNL) has been selected to host the HPSS Users Forum(HUF). The conference will be held at ORNL March 11-13, 2009. More information regarding registration and logistics will be forthcoming in the next couple of weeks.
|
|
|
|
HPSS Integration with SRB
General Atomics Nirvana Division now supports HPSS with their commercial SRB products that is based on Data Grid technology from SDSC. Click here to read.
|
|
|
|
HPSS Petabyte Club
Do you know who is part of the HPSS Petabyte Club? Click here to find out.
|
|
|
|
HPSS World Map
Come view HPSS users around the world.
|
|
|
|
Storage Networking in European Grid Projects
Ex@Grid is a proposal for a high-performance computational and storage grid of the German Gauss Alliance, www.gcfs.eu, serving industry and research institutions. The projected mass storage system at HLRS is baselined to use GPFS and HPSS, including the new GPFS-HPSS Interface.
Click here to read the presentation.
|
|
|
|
HPSS GridFTP Documentation
NERSC has prepared documentation on how to set up gridFTP servers with HPSS 6.2. View the information here.
|
|
|
|
Massive Data Storage Service at Indiana University
IU upgrades their HPSS installation and makes expanded storage services available to TeraGrid. Spanning two campuses, IU is positioned with 1.8 PB of disk and tape capacity and 2GB/sec network data rate. For more details, click here to see the presentation.
|
|
|
|
NERSC Develops Archiving Strategies for Genome Data.
Click here to read the article. |
|
|
|
|
|
