IBM Storage Scale

For parallel I/O ENEAGRID provides PFS based on IBM Spectrum Scale (GPFS) over a fast interconnection InfiniBand (IB) or Omnipath (OPA) according to the cluster.

PFS at the moment is available on cluster CRESCO6, CRESCO4C and CRESCO4F. On each cluster two data spaces are available to users:

• SCRATCH space, with high performances, large quota and a delete policy for older files and no backup. The file systems are local to ENEAGRID sites and they are not accessible directly over wide area. The environment variable name is $PFS_SCRATCH0 in all clusters and the link in the HOME is ~/PFS/tmp. The file systems characteristics are:

CLUSTER	QUOTA	DELETE POLICY
CRESCO6	10 TB quota	30 days since last data access
CRESCO4F	0.5 TB quota	30 days since last data access
CRESCO4C	0.5 TB quota	30 days since last data access

• STORE0 space, lower performances, reduced quota and daily backup. The file systems characteristics are:i

CLUSTER	QUOTA	VARIABLE	HOME LINK
CRESCO6	100 GB quota	$PFS_POR_STORE0	~/PFS/por
CRESCO4F	100 GB quota	$PFS_FRA_STORE0	~/PFS/fra
CRESCO4C	50 GB quota	$PFS_CAS_STORE0	~/PFS/cas

Note

There is backup for data on STORE0 but inactive files, that is files deleted on the file system, will be kept in the backup for one year.

In order to show the disk quota run the gpfsquota command.

The SHELL and the navigation through symbolic links

The access from user HOME to the PFS is done via symbolic links and the navigation through symbolic links depends somewhat from the current shell and its settings.

For bash and ksh the downward and upword track is the same. It a directory is accessed through the link the command cd .. and pwd follow the link notation. The option -P in both commands permits to access the through file system names.

For csh and tcsh the default behaviour in navigating symbolic links is the opposite one, that is the file system names are always used, even if a directory is reached using a symbolic link. The environment variable symlinks must be set to the value “ignore” in order to follow the symbolic link in the upward track. pwd command returns alwayst the file system names.

To set the symlinks variable the user .login or .tcshrc must contain the statement:

set symlinks=ignore

How to use shared directories in GPFS using UNIX groups

UNIX groups permit to manage the access to shared directories. This applies mainly to project area in the GPFS file systems.

To take advantage of this feature the users must provide to ENEAGRID administrators the definition of the group and of the area that should be shared by sending a ticket providing:

• a group name

• the list of users that should belong to the group

• the path of the directory that must be shared

The user must be aware in accessing the shared area that the new files will belong to the user and to the UNIX group that has been provided in the previous step. The user must specify which kind of access mode will have the other members of the group by setting the proper umask, as described in the following:

• login in an ENEAGRID node with access to the shared area

• Choose the access mode for the directory in the current session; this mode will affect only the files created in the current session. To grant to all the users in the group a full control of all the new files enter the command:

  umask 0007
  

  to allow all the users in the group only to read and execute the new files.

  umask 0027
  

• Enter into the shared directory e.g.

  cd /gpor_proj/PROG_ROOT
  

  and start working.