105 - Wofür ist der Institutional Cloud Storage (Projektspeicher) gut und weniger gut geeignet?

While institutional cloud storage is quite excellent for document storage and workgroup use, it is not a good solution for every deployment. This article lists the most important criteria for useful, not recommended and wrong uses:

Useful applications of cloud storage

Replacement of local file servers for documents

Small workgroups with 10-100 employees often have their own file servers with a few terabytes of storage space. These can easily be migrated to the institutional cloud storage (project storage). The local file servers can then be dissolved.

Workgroup-, department-, university- wide projects

If employees from different departments of a university are to work together in a project, one of the participating institutes can release part of its own storage area for the project and grant access rights to all project employees.

MWN-wide file access

You have access to your data within the entire MWN. The LRZ ensures that local firewalls within the Munich Scientific Network also allow traffic to the file services to pass through to the LRZ Cloud Storage. This is not easy to implement with own file servers, because the CIFS typical ports are often blocked. If local firewalls at the chair (or/and the own computer) block access, the responsible administrators must be asked to enable access for the CIFS ports and IP addresses.

Remote Access

Access to the file services is possible worldwide via the LRZ VPN. Since the file services support SMB 2.x and 3.x, the performance over WAN lines is quite good. If no VPN is available, the data can be accessed with a web browser via the web disk (https://webdisk.ads.mwn.de).

Storage for CIP pools

Zentrale Rechnerräume können für die Dateidienste konfiguriert werden. Damit können die Benutzer sehr einfach von jedem Rechner im CIP-Pool auf ihre Dateien zugreifen.

Disrecommended usage of cloud storage

Server operation on the file services

If applications with intensive I/O with individual files (measurement data analysis, translating programs, etc.) are run on the LRZ Cloud Storage and the processing computer is located locally, performance losses are to be expected. Test whether the performance is acceptable for you. If the load on the storage systems increases too much, the LRZ reserves the right to throttle the corresponding storage area in terms of IOPs and/or data throughput. Alternatively, you should consider either keeping the server and data local or moving both to an LRZ hosted environment.

Storage for large media and sensor data

File services are less well suited for large media and sensor data as well as for storing data from GPU/GPGPU-systems (e.g. AI-applications. etc. ) because the storage systems were not designed for this purpose. If you decide to keep or purchase a local file server, we recommend that the file server be integrated into the central Active Directory. Backup should be performed using the LRZ offerings.

When it comes to archiving data for a longer period of time (good scientific practice), we recommend using the archive and backup service of the LRZ or the ISAR Cloud Storage.

Image-based encryption

It is not recommended to use image-based encryption on the file services on a larger scale (e.g. Truecrypt with very large directories). Due to the way these encryption methods work, only one large file is created from the entire directory tree, which can only be handled as a whole by the file server. Restoring individual files from the original directory using snapshots is therefore only possible in a roundabout way. In addition, several people can no longer access the encrypted area at the same time.

If you consider encryption necessary, we recommend the use of file-based encryption methods (e.g. PGP). With any encryption, attention must be paid to the long-term management of the keys. A key loss means the irretrievable loss of the data.

Incorrect usage of cloud storage

There are some use cases where we explicitly advise against using the central storage.

Database server on project storage

It is not recommended to operate multi-user databases or other applications with parallel access by multiple users/instances on the LRZ Cloud Storage. The technical configuration (activated oplocks) is optimized for normal file operation and can lead to problems up to and including data loss in the event of connection interruptions.

However, there are no objections to storing e.g. simple Access databases that are operated in single-user mode.

Backup

The project storage is not to be used for storing backups. For this purpose, the LRZ offers specific solutions that not only provide a longer retention period (at least 6 months instead of 4 weeks), but are also optimized for this purpose.

Home directories for Linux/Unix

Native home directories for Linux/Unix servers are currently not possible, since no NFS is offered and the CIFS clients do not support some Unix specialties (soft links). However, it is easily possible to provide project storage or personal storage as a subdirectory in the home directory. The CIFS semantics then also cause no problems when sharing e.g. office documents.

Offline Folder

Offline folders repeatedly cause problems in daily operation, since synchronization with the storage system is sometimes interrupted or no longer functions properly. This can lead to files not being saved or only older versions of a file being saved on the storage system. For this reason, the use of offline folders is NOT recommended!