Exam AI-102 topic 1 question 58 discussion

Answer is correct: Azure OpenAI allows you to manage how frequently your application can make inferencing requests. Your rate limits are based on Tokens-per-Minute (TPM). For example, if you have a capacity of 1, this equals 1,000 TPM, and the rate limit of requests you can make per minute (RPM) is calculated using a ratio. For every 1,000 TPM, you can make 6 RPM. So, if you need to process 600 requests every minute, you'll require a TPM that supports that many RPM. Using the ratio, for 600 RPM, you need 100,000 TPM (because 600 divided by 6 equals 100, and 100 multiplied by 1,000 equals 100,000). In this scenario, you would set the capacity to 100, since each capacity unit equals 1,000 TPM.

Answer is correct From here https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/quota?tabs=rest When a deployment is created, the assigned TPM will directly map to the tokens-per-minute rate limit enforced on its inferencing requests. A Requests-Per-Minute (RPM) rate limit will also be enforced whose value is set proportionally to the TPM assignment using the following ratio: 6 RPM per 1000 TPM. From here: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/quota?tabs=rest capacity integer This represents the amount of quota you are assigning to this deployment. A value of 1 equals 1,000 Tokens per Minute (TPM). A value of 10 equals 10k Tokens per Minute (TPM). So the math is 600 RPM needs 100000 TPM which translates to capacity 100

"capacity" is CORRECT because it specifies the scale unit of the deployment, which determines the number of requests the resource can handle per minute. Setting the appropriate capacity value ensures that the resource can respond to 600 requests per minute. 100 is CORRECT because a throughput of 600 requests per minute requires the ability to process a substantial number of tokens per minute. Each capacity unit supports 1,000 tokens per minute (TPM). For a typical request, assuming it processes up to 100 tokens per request (input + output), the capacity required is 600 requests * 100 tokens = 60,000 TPM. Setting the capacity to 100 ensures the resource can handle up to 100,000 TPM, meeting the 600 requests per minute requirement with sufficient buffer.

This was in the exam on 15/08/2024

1. capacity 2. 100

Parameter Type Description sku Sku The resource model definition representing SKU. capacity integer This represents the amount of quota you are assigning to this deployment. A value of 1 equals 1,000 Tokens per Minute (TPM). A value of 10 equals 10k Tokens per Minute (TPM).

capacity and 100.

capacity and 100. From Takedajuku perspective, if you study for 4 days and spend 2 days reviewing, you will have a better chance of passing the exam.

on exam today 24/05/2024, I selected capacity and 600

The accurate capacity value to meet the requirement of processing 600 requests per minute is indeed 100. I apologize for any confusion caused earlier

In the provided options, the correct choice is: "capacity" : 600 The capacity parameter in the SKU configuration of an Azure OpenAI resource specifies the maximum number of requests per minute (RPM) that the resource can handle. By setting "capacity": 600, you are configuring the resource to handle up to 600 requests per minute.