Exam DP-201 topic 25 question 1 discussion

How come batch processing using Azure Stream analytics why not Azure data bricks? seems like wrong answer this should be D.

Technology choices for batch processing are 1. Azure Synapse Analytics 2. Azure HDInsight 3. Azure Data Lake Analytics 4. Azure Databricks https://docs.microsoft.com/en-us/azure/architecture/data-guide/technology-choices/batch-processing

for batch processing is databricks

Why is it that no body is choosing Azure stream analytics as the input of the processing solution is messages generated by the website.

Correct Answer: B Explanation/Reference: Explanation: Scenario: ADatum identifies the following requirements for the Health Interface application: Support a more scalable batch processing solution in Azure. Reduce the amount of time it takes to add data from new hospitals to Health Interface. Data Factory integrates with the Azure Cosmos DB bulk executor library to provide the best performance when you write to Azure Cosmos DB. Reference: https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-cosmos-db

I would choose ADF. https://devblogs.microsoft.com/cosmosdb/migrating-relational-data-into-cosmos-db-using-azure-data-factory-and-azure-databricks/

D. Azure Databricks

Not sure if databricks can access on prem data source. If yes, then no question D. If not, then you have to use ADF copy data activity to opy from on prem to staging. But as different hospitals have different data formats then you have to transform it to common format. ADF can use mappng data flow or call databricks notebook to do that (but only from staged data already in Azure). dataflow unfortunately is not auto scalable, you have to redefine how many cores you want to use, so I would call databricks notebook from ADF after copy data in ADF. Cosest anwer seems C - ADF.

https://docs.microsoft.com/en-us/azure/architecture/data-guide/technology-choices/batch-processing it seems Databricks

Don't go by word "batch". read this: Health Interface - ADatum has a critical application named Health Interface that receives hospital messages related to patient care and status updates. So stream analytics seems to be correct.

The more reactions I read, the more confused I get. My 2 cents: in this case, the hospitals send the data in batch. This means not message-by-message, but a file containing several messages or records. Most of the discussion here looks at "batch processing", which is another story to do with analysing big data stored in files. To me, batch processing is not the correct context of this case. What we need is to ingest files coming from the hospital from time to time. Azure Data Factory seems right to me. The answer's comment also seems to point to this solution, so the answer itself might be a typo.

Can I use ADF only for solution of both Health Insights and Health Interface?

Which product would provide the best performance?

It has B showing as the answer, but then the description underneath implies C where it talks about data Factory and Cosmos DB. Data Factory is scalable.

"Minimize the number of services required to perform data processing, development, scheduling, monitoring, and the operationalizing of pipelines." I would pick Data Factory as the answer

The answer should be D: Databricks. Purely because of Scalability factor. ADF can be used but Databricks is better when it comes to scaling.

They mentioned, health interface application received data in batches (group of messages as batch from existing c# application). If ADF is answer how solution is expecting to receive data (http source / json files on blob store?) with varying schema and perform bulk insert into cosmodb? It has to be ADB receiving messages / batches on stream and ingesting them into cosmodb.