IBM Storage Scale ============================== For parallel I/O ENEAGRID provides PFS based on `IBM Spectrum Scale (GPFS) `_ over a fast interconnection InfiniBand (IB) or Omnipath (OPA) according to the cluster. PFS at the moment is available on cluster **CRESCO6**, **CRESCO4C** and **CRESCO4F**. On each cluster two data spaces are available to users: - **SCRATCH space**, with high performances, large quota and a delete policy for older files and no backup. The file systems are local to ENEAGRID sites and they are not accessible directly over wide area. The environment variable name is ``$PFS_SCRATCH0`` in all clusters and the link in the HOME is ``~/PFS/tmp``. The file systems characteristics are: .. list-table:: :header-rows: 1 :widths: 30 25 25 * - CLUSTER - QUOTA - DELETE POLICY * - CRESCO6 - 10 TB quota - 30 days since last data access * - CRESCO4F - 0.5 TB quota - 30 days since last data access * - CRESCO4C - 0.5 TB quota - 30 days since last data access - **STORE0 space**, lower performances, reduced quota and daily backup. The file systems characteristics are:i .. list-table:: :header-rows: 1 :widths: 30 25 25 25 * - CLUSTER - QUOTA - VARIABLE - HOME LINK * - CRESCO6 - 100 GB quota - $PFS_POR_STORE0 - ~/PFS/por * - CRESCO4F - 100 GB quota - $PFS_FRA_STORE0 - ~/PFS/fra * - CRESCO4C - 50 GB quota - $PFS_CAS_STORE0 - ~/PFS/cas .. note:: There is backup for data on STORE0 but inactive files, that is files deleted on the file system, will be kept in the backup for one year. In order to show the disk quota run the ``gpfsquota`` command. The SHELL and the navigation through symbolic links --------------------------------------------------- The access from user HOME to the PFS is done via symbolic links and the navigation through symbolic links depends somewhat from the current shell and its settings. For bash and ksh the downward and upword track is the same. It a directory is accessed through the link the command ``cd ..`` and ``pwd`` follow the link notation. The option ``-P`` in both commands permits to access the through file system names. For csh and tcsh the default behaviour in navigating symbolic links is the opposite one, that is the file system names are always used, even if a directory is reached using a symbolic link. The environment variable symlinks must be set to the value "ignore" in order to follow the symbolic link in the upward track. pwd command returns alwayst the file system names. To set the symlinks variable the user ``.login`` or ``.tcshrc`` must contain the statement: .. code-block:: bash set symlinks=ignore How to use shared directories in GPFS using UNIX groups ------------------------------------------------------- UNIX groups permit to manage the access to shared directories. This applies mainly to project area in the GPFS file systems. To take advantage of this feature the users must provide to ENEAGRID administrators the definition of the group and of the area that should be shared by sending a `ticket `_ providing: - a group name - the list of users that should belong to the group - the path of the directory that must be shared The user must be aware in accessing the shared area that the new files will belong to the user and to the UNIX group that has been provided in the previous step. The user must specify which kind of access mode will have the other members of the group by setting the proper umask, as described in the following: - login in an ENEAGRID node with access to the shared area - Choose the access mode for the directory in the current session; this mode will affect only the files created in the current session. To grant to all the users in the group a full control of all the new files enter the command: .. code-block:: bash umask 0007 to allow all the users in the group only to read and execute the new files. .. code-block:: bash umask 0027 - Enter into the shared directory e.g. .. code-block:: bash cd /gpor_proj/PROG_ROOT and start working.