Snapshots
The data in your home directories is protected by nightly file system snapshots, which are kept for at most 7 days. In order to access theses snapshots, look into the directory /dss/dsshome1/.snapshots/. In this directory you'll find the individual snapshots as subdirectories, which have the date and time at which the snapshot was taken encoded as YYYY-MM-DD_HHMM in their directory name. In order to restore files you can simply copy them back to your HOME directory. A little tool may help you to access the snapshots:
module load lrztools
SN {file| deleted file| directory|deleted dierectory]
Tape Backup
Additionally the data in your home directories is also protected by nightly backups to tape. However, due to security reasons these backups are not directly accessible by the users. If you really need to have data in your home directory restored that cannot be found in the snapshots anymore, please submit a ticket to the Servicedesk for SuperMUC-NG. But bear in mind that this is a man power consuming activity and may take some time to complete.
In the future we also plan to replicate data on HOME asynchronously to a secondary system and keep more than 7 snapshots. However, this may still take some months to implement.
Please note that, as on SuperMUC, no automated backups in any way will be taken for the WORK or SCRATCH file systems.
Status of the new Archive System for SuperMUC-NG
Currently we are in the process of designing and procuring a successor system of the SuperMUC Archive, which will be called Data Science Archive (DSA). This system will be able to cope with the increased data volumes and rates that are anticipated to be created by SuperMUC-NG and also provide much more convenience in managing and sharing this data. The general availability of this system is currently planned for Q4/2020.
Status and future of the Archive System/Data of SuperMUC
Access to the archive system and data of SuperMUC is currently possible via the special login node: skx-arch.supermuc.lrz.de
Change of access policy to the archive data
The access policy to the tape archive on SuperMUC-NG differs from the policy on SuperMUC! Access rights on SuperMUC were granted to individuell users (only the users who wrote the data could access it), whereas on SuperMUC-NG the rights are granted to the particular project.
- Permissions to access the data will now be granted to all users of a project i.e., all users in a group can retrieve data from other users of the project group.
- The previous policy was that only the users who wrote the data could access it.
- If you project is for example ‘pr12ab’, you can see the members of this group by
getent group pr12ab-d - This change also affects data stored before the transition of the archive to SuperMUC-NG
If the new policy is not appropriate for you, the master user/project manager must put a request to the Servicedesk for SuperMUC-NG. We will then restrict the access only to the master user/project manager of the project. However, all other user in the project group will have no access.
Explicitly specifying the target project
Please note that because of the new, user centric usage model of SuperMUC-NG, you have to explicitly specify the project name on the dsmc command line, whose archive data you want to access. This is done by using the -se option. E.g. dsmc archive -se=prXXXX myFile
Write access to the SuperMUC Archive
Until the new Data Science Archive is available, you are still allowed to write data into the old SuperMUC archive. However, please bear in mind that the SuperMUC Archive was designed for a total capacity of approximately 20PB, which is only 30% of the total capacity of SuperMUC-NGs file systems and the 20PBs are already consumed by data from SuperMUC. We've now added some more capacity in the range of 2PB to the archive and kindly ask you to use this resource very whisely.
Details on the usage can be found here.
Status and future of the Archive Data of HLRB2 and HLRB1
We still retain the archive data of HLRB2 and HLRB1. However, due to technical and security reasons users don't have access to the data via self service. If you need data from these archives, please submit a ticket to the Servicedesk for SuperMUC-NG.