Exam DP-203 topic 2 question 13 discussion

My answer will be B Stream Analytics supports "extending SQL language with JavaScript and C# user-defined functions (UDFs)". There is no mention of Python support; hence Stream Analytics is not correct. https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-introduction Azure Databricks supports near real-time data from Azure Event Hubs. And includes support for R, SQL, Python, Scala, and Java. So I will go for option B.

I'm sure it's Stream Analytics cause Event Hubs already supports Python (https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-python-get-started-send). We don't need the other service to support it. We just need to lower costs. Hence ASA is the correct solution

Should be B. See the link below. Under the heading "When to use other technologies", it mentions "Azure Stream Analytics supports user-defined functions (UDF) or user-defined aggregates (UDA) in JavaScript for cloud jobs and C# for IoT Edge jobs. C# user-defined deserializers are also supported. If you want to implement a deserializer, a UDF, or a UDA in other languages, such as Java or Python, you can use Spark Structured Streaming. You can also run the Event Hubs EventProcessorHost on your own virtual machines to do arbitrary streaming processing." As the question mentions user-defined-function (UDF) in Python, ASA seems not support UDF in Python. Should use Spark Structured Streaming, which in this case here is Azure Databricks. https://learn.microsoft.com/en-us/azure/stream-analytics/streaming-technologies

C is wrong, stream analytics is a SQL based analytics

ChatGPT 4o Azure Databricks is well-suited for real-time data processing and analytics. It provides a collaborative environment for working with Apache Spark, which is ideal for performing complex statistical analyses and machine learning tasks in real-time.

It says we are using python after it is sent to event hubs: "custom proprietary Python functions on near real-time data FROM Azure Event Hubs". Yes, we can send events to Event Hub with python but it says that we are running statistical analysis AFTER we send it to Event Hub. Therefore, my answer is Databricks

Azure Stream Analytics: Azure Stream Analytics is designed for real-time data processing and can directly ingest data from Azure Event Hubs. However, it has limited support for custom Python functions. It is more suitable for simple real-time analytics and transformations rather than complex statistical analysis with custom code. correct answer: Azure Databricks, we have custom python function

B is correct

Azure Databricks provides a fast and scalable Apache Spark-based analytics platform that supports Python, among other programming languages. It allows you to perform near real-time data processing and analysis efficiently, making it ideal for scenarios where low latency is a priority. Additionally, it offers seamless integration with Azure Event Hubs, enabling you to ingest data in real-time and apply custom Python functions for statistical analysis.

B. Azure Databricks Azure Databricks provides a fully managed Apache Spark-based analytics platform that is well-suited for processing and analyzing real-time streaming data. It offers native integration with Azure Event Hubs, allowing you to ingest data in real-time and apply custom Python functions for statistical analysis with minimal latency. Additionally, Databricks provides scalable compute resources, optimized processing capabilities, and support for various programming languages, making it an ideal choice for near real-time data analysis scenarios.

C is correct. At near realtime ,the window functions in azure stream analytics can be employed in compute some statical values (e.g count,maximum,min,avg. etc) of the data streaming from the even hub.

Corrected!!! FROM Azure Event Hubs, not ON Azure Databricks

FROM Azure Event Hubs, not ON

chatGPT explains - Azure Stream Analytics is designed for real-time data stream processing and analytics. It can ingest data from various sources, including Azure Event Hubs, and allows you to run near real-time analytics using a SQL-like language. With Stream Analytics, you can easily apply custom Python functions using user-defined functions (UDFs) and achieve low-latency processing. Azure Synapse Analytics and Azure Databricks are powerful analytics services, but they are more suitable for complex analytics and big data processing rather than near real-time, low-latency scenarios. Azure SQL Database is a relational database service and is not specifically designed for real-time stream processing. Therefore, in this case, Azure Stream Analytics is the recommended choice for minimizing latency in statistical analysis on near real-time data from Azure Event Hubs.

Simply, they always want the Stream Analytics answer. It's the most straightforward.

Azure Databricks

From ChatGPT, To minimize latency for statistical analysis on near real-time data from Azure Event Hubs, I recommend using Azure Stream Analytics (Option C). Azure Stream Analytics is designed for real-time data processing and can ingest and analyze data from Event Hubs with low latency, making it a suitable choice for this scenario.