Indiana University recently upgraded its research data archival
storage system. The total storage capacity now exceeds 1.8PB, and IU
has demonstrated an aggregate read and write rate of almost 2GB/s.
IU is making this system available to TeraGrid users.
The largest allocation of TeraGrid storage space ever was recently
made to the Linked Environments for Atmospheric Discovery (LEAD),
to use IU's archival storage system. LEAD makes meteorological data,
forecast models, and analysis and visualization tools available to
anyone who wants to interactively explore the weather as it evolves.
LEAD will also take advantage of IU's Big Red supercomputer in order
to analyze that data. More information about Big Red is available at
http://kb.iu.edu/data/aueo.html. More information on LEAD is
available at http://lead.ou.edu
IU uses the High Performance Storage System (HPSS) software to
manage archival storage. IU's HPSS system is geographically
distributed between Indianapolis and Bloomington. By default, IU
will create two copies of any data, one copy at the Bloomington
campus and one copy at the Indianapolis campus, ensuring data
integrity and disaster recovery capability.
TeraGrid users can access this service using standard GridFTP tools,
either globus-url-copy or uberftp (the tgcp tool does not currently
support Indiana UniversityÕs HPSS installation). More information is
available at http://storage.iu.edu/. Researchers interested in using
this service are encouraged to write to researchcomputing@iu.edu.
Indiana University Cyberinfrastructure News
-
-
TeraGrid users can access Indiana University's High Performance
Storage System via GridFTP clients like uberftp and globus-url-copy.
Other methods of access require a normal IU Network ID.
Class of service (COS) is a mechanism used by HPSS to determine how
user data will be cached and stored, based on file size, required
performance, storage location, and other considerations. COS may be
regarded as an analog, for data storage, to the various queues of a
computational resource. IU's HPSS currently supports classes 1, 2, 3
for replicated small (10GB), medium (40GB), and large (10TB) files.
The class of service can be specified when creating a file in HPSS
by appending two commas followed by the numeric class of service to
the filename. For example, if "datafile" is a file in the current
working directory, and you are using uberftp, then for COS=3 use
put datafile datafile,,3
More information can be found at http://kb.iu.edu/data/auli.html.
For data transfers in the course of job scripts, globus-url-copy
is generally preferred. To upload a file in the current working
directory to HPSS at IU, with COS=2, use
globus-url-copy -rp file://`pwd`/datafile \
gsiftp://gridftp.archive.iu.teragrid.org/datafile,,2/
[The "\" is a line continuation, which you may not need].
To download that file to the current working directory on Big Red,
globus-url-copy -rp \
gsiftp://gridftp.archive.iu.teragrid.org/datafile \
file://`pwd`/datafile
More information on GridFTP clients is available from
http://kb.iu.edu/data/auxj.html and
http://www.teragrid.org/userinfo/data/gridftp.php -
In the 21st century, progress requires collaborations across
disciplines and the utilization of advanced cyberinfrastructures.
In scientific fields, increasing interdisciplinary research demands
novel means for sharing data, services, computer resources, and
expertise.
The Cyberinfrastructure for Network Science Center at Indiana
University is developing user-friendly tools that address this need.
Weixia (Bonnie) Huang and Bruce Herr recently presented the CIShell
and Network Workbench (NWB) in the OSGi track at EclipseCon 2007.
The OSGi Alliance was initiated by technology companies including
IBM, Intel, Nokia, Motorola, NEC, and Siemens, to assure the
interoperability of applications and services based on its service-
oriented, component-based Java environment.
Huang remarked, "By demonstrating the CIShell and NWB Tool, both
of which use OSGi techniques to implement their plug-and-play
architecture, we want to deliver a message that OSGi can be extended
to (and is useful in) academic research as well."
Currently, the development team is interested in integrating not
only diverse algorithms and datasets into the NWB tool but also
computational resources. "We use HPSS as a backup solution for our
database and file systems attached to our departmental servers,"
Huang reports. "We plan to develop plug-in modules in CIShell to
connect to supercomputers, like Big Red, and leverage the Network
Workbench so that researchers can easily run complex analyses with
large datasets."
More information is available at http://cishell.org and
http://www.osgi.org -
Stephen Simms, IUÕs Data Capacitor Project Manager, introduced the
535 TeraByte filesystem on March 28, in the Research Computing
Brownbag Series. Designed to store intense data bursts coming from
modern scientific instruments for short periods, the Data Capacitor
complements the home directories and working disk space attached to
computational resources, and archival High Performance Storage
System, and is attached to both.
The Data Capacitor goes to hearty user status on April 1. Early
users will include the Indiana University Molecular Structure
Center and the WIYN Observatory. For more information, see
http://datacapacitor.iu.edu/. -
Digital Library Program Brown Bag Talk
All programs will be held in the Wells Library in Room E174, the
Media Showing Room, from 12:00-1:15PM unless otherwise noted.
Wednesday, April 4, 2007
Indiana Magazine of History: Encoding Challenges
Michelle Dalmau and Melanie Schlosser
Wednesday, April 18, 2007
JiTT DL and Cognitive Work Analysis
Elin Jacob and Nicholas George
--------
Wednesday, April 11 at the ICTC, 9:00AM-4:00PM
LinuxFest 2007 -- Linux for the People!
Don Vosburg of Novell will give the keynote speech.
The Unix Systems Support Group is looking for input about the Unix
Users' Group. Please send suggestions about what you would like to
see and do with the Unix Users' Group to researchcomputing@iu.edu.
--------
Wednesday, April 25, 12:30-1:30 -- ICTC Room 497 & IMU Walnut Room
Research Computing Brown Bag Talk
Michel Tavares, Scientific Data Services
To stimulate interest and dialogue between Indiana University's
research computing community and UITS's Research Computing division,
monthly brownbag lunches will be held the last Wednesday of each
month from 12:30-1:30.
Fetal Alcohol Spectrum Disorder (FASD) is a term used to describe
the range of disabilities caused by prenatal exposure to alcohol.
The mission of the Collaborative Initiative on FASDs (CIFASD) is to
inform and develop effective interventions and treatment approaches
for FASD, through multidisciplinary research involving basic,
behavioral and clinical investigators and projects.
The Scientific Data Services group contributes to this initiative
by providing custom software, support, as well as a central
repository for research data. Michel Tavares, FASD Technical
Architect, will introduce the FASD project, describe the different
technologies that support it,and discuss how similar projects could
benefit from this approach.
--------
On Tuesday, May 1, UITS will retire the AVIDD clusters. If you use
AVIDD, you should migrate applications to Big Red, Libra, or the
TeraGrid. To get help with porting codes or applying for a TeraGrid
allocation, send email to researchcomputing@iu.edu. -
Outages
-------
Name Date [mmddyy] Time Failure
N5500 02/24/07 23:35-01:18+4d NAS failure
Libra 02/28/07 LoadLeveler restarted
EMC 03/02/07 23:51-20:30+2d NAS failure
Big Red 03/07/07 13:13-15:52 GPFS unavailable
Big Red 03/07/07 21:30-01:00+1d GPFS unavailable
Big Red 03/10/07 06:00-20:45 GPFS unavailable
Libra 03/16/07 17:15-18:00 Libra00 restarted
Planned Maintenance
-------------------
System Date Time Action
Steel 04/01/07 06:00-10:00 OS Patches
AVIDD 04/03/07 08:00-17:00 OS Patches
RDC 04/03/07 08:00-17:00 OS Patches
Libra 04/03/07 08:00-17:00 OS Patches
Big Red 04/03/07 20:00-24:00 OS Patches -
If you have questions pertaining to IU's cyberinfrastructure, or you
are encountering some difficulty, there are several ways to obtain
help.
An introduction and overview titled "Indiana University's
CyberInfrastructure: The least you need to know" has been updated
and is available at http://rc.uits.iu.edu/education_and_training/ .
The IU Knowledge Base (http://kb.iu.edu) is an excellent source of
help on how to do things. If you have problems which the KB does not
enable you to solve, questions about system outages, or if you just
have a problem and you don't know who to contact, send email to
researchcomputing@iu.edu.
