Exam AWS Certified Data Analytics - Specialty topic 1 question 6 discussion

Answer should be A according to the link provided. Thoughts?

B is wrong. We don't need a staging DB here which is costly and moreover MySQL is not the right choice. C. dropDuplicates() is used to remove duplicate records in the Spark not destination DB D. ResolveChoice is to cast data with unidentified data type to a specified data type and also work on Spark not destination DB. A is the answer

It's n A

To me A looks correct answer, check this link https://stackoverflow.com/questions/52397646/aws-glue-to-redshift-duplicate-data

A: I passed the test

A. Modify the AWS Glue job to copy the rows into a staging table. Add SQL commands to replace the existing rows in the main table as postactions in the DynamicFrameWriter class. To update the Redshift table without duplicates when AWS Glue jobs are rerun, the company should modify the AWS Glue job to copy the rows into a staging table. The job should then add SQL commands to replace the existing rows in the main table as postactions in the DynamicFrameWriter class. This approach ensures that the data written to the Redshift table does not contain any duplicates, and the table only contains the latest data. Loading the previously inserted data into a MySQL database and performing an upsert operation may be a feasible approach but adds complexity to the architecture. Using Spark's dropDuplicates() API to eliminate duplicates may not always work correctly when dealing with large datasets. Using the ResolveChoice built-in transform is used for handling schema changes in a column, not for removing duplicates.

With option B, it is to copy the RedShift data into SQL and back to RedShift. Option A is simpler

A, for sure

Correct Answer - A B is is incorrect because you can't use the COPY command to copy data directly from a MySQL database into Amazon Redshift. A workaround for this is to move the MySQL data into Amazon S3 and use AWS Glue as a staging table to perform the upsert operation. Since this method requires more effort, it is not the best approach to solve the problem.

Correct answer is A as Redshift does not support merge or upsert on the single table. However, a staging table can be created and data merged with the main table. Option B is wrong as a staging DB as MySQL is not required. Option C is wrong as dropDuplicates() is used to remove duplicate records in the Spark and not destination DB. Option D is wrong as ResolveChoice is to cast data with an unidentified data type to a specified data type. It does not handle duplicates.

Answer is A

Answer is A

Answer is A

Answer: A. Modify the AWS Glue job to copy the rows into a staging table. Add SQL commands to replace the existing rows in the main table as postactions in the DynamicFrameWriter class.

The answer is A.

A should be the right answer. I found a link that helps to explain why. https://aws.amazon.com/pt/premiumsupport/knowledge-center/sql-commands-redshift-glue-job/

A is the right answer