Exam AWS Certified Solutions Architect - Professional SAP-C02 topic 1 question 23 discussion

A: Lazy loading is cost-effective because only a subset of data is used at every job B: There are hundreds of EC2 instances using the volume which is not possible (one EBS volume is limited to 16 nitro instances attached) C: Batching would load too much data D: storage gateway is used for on premises data access, I don't know is you can install a gateway in AWS, but Amazon would never advise this

Answer: D I think the main point here is to understand what they mean by "The file system must provide high performance access to the needed data" while "provide the LARGEST overall cost reduction"? For answer A, we have to remember that lazy load is SLOW for the first time you try to access the file (as it is being fetched from S3), BUT, as we are talking about hundreds of instances, then it might be OK. S3 Intelligent-Tiering, although doesn't seem to fit much, the part that says "The job runs once monthly, reads a subset of the files from the shared file system", indicates that at least part of the 200TB of data won't be accessed, which helps not going for answer C, for example. My only issue with answer D is that Storage Gateway can be slower than FSx for Lustre, HOWEVER, what is the cost X performance ratio they are seeking here? We can guess that costs trumps maximum performance here: "Which solution will provide the LARGEST overall cost reduction" and, as Storage Gateway is way cheaper than FSx for Lustre per TB, it's safe to say that D is the most correct answer.

With our S3 objects imported into our Lustre file system, we can now lazy-load the files we need by simply reading the particular files. After a file is lazy-loaded, its contents are fully copied from S3 onto the Amazon FSx for Lustre file system, where it can be accessed with extremely low latency. I will go for A. https://aws.amazon.com/blogs/storage/new-enhancements-for-moving-data-between-amazon-fsx-for-lustre-and-amazon-s3/

I lean more towards D but I am not sure whether the Gateway is only intended for on-premise as few are mentioning here. If that is not the case then the right option is D.

FSx for Lustre, is only for Linux where in the question is Linux noted. It's states only EC2 instance not which OS is on it!

'The job runs once monthly', 'cost reduction' = S3 Intelligent-Tiering storage class, lazy loading. 'Scalable file system', 'shared file system', 'data-intensive ' = Amazon FSx for Lustre

By choosing Option A, the company can leverage the cost-effectiveness of Amazon S3 Intelligent-Tiering for storage and the high performance of Amazon FSx for Lustre for temporary file access, while minimizing the overall cost by creating and deleting the file system only when needed. Option B (using Amazon EBS Multi-Attach) is not ideal because EBS volumes are designed for persistent storage, and attaching and detaching a large volume to multiple instances can be time-consuming and potentially disruptive. Option C (using Amazon FSx for Lustre with batch loading) is less cost-effective than Option A because batch loading requires loading the entire 200 TB of data into the file system, which can be expensive and time-consuming. Option D (using AWS Storage Gateway File Gateway) is not the most cost-effective solution because the File Gateway is designed for on-premises file storage integration and may not provide the same level of performance as FSx for Lustre for this data-intensive workload.

A. Migrate the data from the existing shared file system to an Amazon S3 bucket that uses the S3 Intelligent-Tiering storage class. Before the job runs each month, use Amazon FSx for Lustre to create a new file system with the data from Amazon S3 by using lazy loading. Use the new file system as the shared storage for the duration of the job. Delete the file system when the job is complete.

A or D : confusion. I wish they can provide explanation about their answers when it is not the most voted one

I vote D instead A because the requirement in the question is "modifies the data on the shared file system" Fsx imported data from s3 and lost the relationship to s3 after import is done Without explicitly copy back to s3, the change stays on shared file system only. Answer A solution doesn't provide a step to copy the modification back to s3. Storage gateway presents s3 storage to the OS as shared file system. Any modification on the shared file system will be automatically saved on s3.

A: Lazy loading is cost-effective because only a subset of data is used at every job

Problem with D is that, AWS Storage GW and File GW are solutions for integrating on-premise storage with AWS storage solutions, particularly (but not limited to) S3. https://aws.amazon.com/storagegateway/ https://aws.amazon.com/storagegateway/file Compute resources are residing in AWS, so having Storage GW and File GW won't solve a thing. As far as option B is concerned, it comes down to the limitations of EBS (such as the max block size, and max number of instance that can be attached etc). Also, attaching and detaching of the EBS volumes seems a bit complicated too. On top of that, EBS does not offer the cost optimizations offered by S3 Intelligent Tiering. The question clearly mentions that only a subset of the data will be used. Intelligent tiering ensures a substantial cost optimization over time. Hence, the answer should be A.

Option D, migrating the data to an Amazon S3 bucket and using AWS Storage Gateway, seems to provide the largest overall cost reduction while meeting the requirements of high-performance access during the job run and minimizing costs when the storage is not actively being used. Therefore, Option D is the most suitable choice.

https://aws.amazon.com/blogs/storage/new-enhancements-for-moving-data-between-amazon-fsx-for-lustre-and-amazon-s3/

Option A - This option might work. However, AWS FSx for Lustre does not have a feature called "lazy loading" - its default behavior is to load a file from S3 when it is first accessed (restore). It can provide high-performance as needed though nothing is said in the question about whether a slow initial load time due to restore operations could be an issue. S3 Intelligent-Tiering minimizes storage costs. Option B - This option will provide a high-performance storage option. However, storage in EBS is expensive compared to other AWS storage services Option C - This option might work. However, AWS FSx for Luster does not have a feature called "batch loading". Files can be pre-loaded issuing a hsm-restore command. S3 Standard is a cheap storage option yet not the cheapest option in S3 Option D - This option does not work as described in the option

Not B because using EBS still involves EC2 instances that are expensive (the instances that host the shared file system run continuously). Also, multi-attach is supported only for io1/oi2 EBS disk types that are expensive; Not C as batch loading does not exists in the doc/console, I think they might refer to the option to pre-populate the data using lfs hsm_restore command as mentioned here https://docs.aws.amazon.com/fsx/latest/LustreGuide/preload-file-contents-hsm-dra.html. This would be a more expensive option Not D as Storage Gateway provide less performance than FSx for Lustre and it requires at least an EC2 instance and this will introduce additional cost AA is a viable option as S3 is cheaper storage, FSx for Lustre provides performance. Lazy loading allows to actually move in the filesystem data that is actually used and intelligent tiering make sure those files that are not used are moved to less expensive S3 storage tiers.

Intelligent tiering is not required, because the job would be running for every month, so there is no purpose for intelligent tiering, The question is having cost impact also one of the option. So go with option D.