Practice Exams:

Microsoft AI-300 Dumps

EXAM: Operationalizing Machine Learning and Generative AI Solutions
Verified By Experts
Premium Bundle
$49.99

AI-300 Premium ETE File


  • Premium File 213 Questions & Answers
Whats Included:
  • Latest Questions
  • 100% Accurate Answers
  • Fast Exam Updates

Microsoft AI-300 Practice Test 47

Question 460

An educational customer support bot must reject harmful requests without losing its ability to answer ordinary campus policy questions. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. Verify: Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets.
  2. Implement: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Verify: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.
  3. Implement: Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. Verify: Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets.
  4. Implement: Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. Verify: Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.

Answer & Explanations

Correct answer: A

A.

The question is set in this situation: An educational customer support bot must reject harmful requests without losing its ability to answer ordinary campus policy questions. Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. Confirm it through this test: Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets. Task-quality metrics cannot substitute for content safety checks; a response can be relevant and grounded while still revealing confidential data or producing prohibited instructions.

B.

This action/test pair addresses “Automate generative AI quality regression gates within a reviewed release pipeline”. Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed. The case instead calls for: Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant.

C.

The test would be relevant: Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets. But the offered implementation—Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness.—addresses “Measure response relevance and coherence against the original user intent”. The action needed here is: Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant.

D.

The action is applicable: Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. But its proposed test—Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.—addresses “Measure response relevance and coherence against the original user intent”. This requirement is confirmed instead by: Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets.

Question 461

A financial FAQ assistant is released weekly and needs an automated benchmark that stops a new prompt when citation accuracy drops on a fixed policy set. Which Azure AI operations action best addresses this requirement?

  1. Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.
  2. Instrument prompt and completion token usage, model deployment pricing or quota, retries and per-request attribution so GenAIOps can set budgets and improve inefficient prompts.
  3. Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness.
  4. Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant.

Answer & Explanations

Correct answer: A

A.

Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements. Verify the result with this test: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

B.

Instrument prompt and completion token usage, model deployment pricing or quota, retries and per-request attribution so GenAIOps can set budgets and improve inefficient prompts. This addresses “Track token consumption and model inference cost by feature and deployment”, where the governing principle is: Token-based charges depend on input and output usage and repeated requests; a low count of web page visits can still generate large costs from oversized context and retries. The stated case instead calls for: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

C.

Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. This addresses “Measure response relevance and coherence against the original user intent”, where the governing principle is: Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision. The stated case instead calls for: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

D.

Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. This addresses “Run safety and harmful-content evaluations on generative applications and agents”, where the governing principle is: Task-quality metrics cannot substitute for content safety checks; a response can be relevant and grounded while still revealing confidential data or producing prohibited instructions. The stated case instead calls for: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

Question 462

A financial FAQ assistant is released weekly and needs an automated benchmark that stops a new prompt when citation accuracy drops on a fixed policy set. The engineer proposes the following change: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Which mechanism explains why it works?

  1. Token-based charges depend on input and output usage and repeated requests; a low count of web page visits can still generate large costs from oversized context and retries.
  2. Task-quality metrics cannot substitute for content safety checks; a response can be relevant and grounded while still revealing confidential data or producing prohibited instructions.
  3. Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision.
  4. Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

Answer & Explanations

Correct answer: D

A.

Token-based charges depend on input and output usage and repeated requests; a low count of web page visits can still generate large costs from oversized context and retries. That explains “Track token consumption and model inference cost by feature and deployment”, not the mechanism behind the described change. The relevant principle is: Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

B.

Task-quality metrics cannot substitute for content safety checks; a response can be relevant and grounded while still revealing confidential data or producing prohibited instructions. That explains “Run safety and harmful-content evaluations on generative applications and agents”, not the mechanism behind the described change. The relevant principle is: Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

C.

Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision. That explains “Measure response relevance and coherence against the original user intent”, not the mechanism behind the described change. The relevant principle is: Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

D.

Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements. The corresponding implementation is: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

Question 463

A financial FAQ assistant is released weekly and needs an automated benchmark that stops a new prompt when citation accuracy drops on a fixed policy set. After implementing this change—Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.—which test most directly validates its intended outcome?

  1. Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets.
  2. Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.
  3. Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.
  4. Run a fixed sample of short and long requests, record measured tokens and unit cost by deployment and verify alerts respond to an approved budget threshold.

Answer & Explanations

Correct answer: B

A.

Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets. That would help test “Run safety and harmful-content evaluations on generative applications and agents”, not the outcome required in this scenario. The relevant acceptance check is: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

B.

Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed. This check is decisive because continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

C.

Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them. That would help test “Measure response relevance and coherence against the original user intent”, not the outcome required in this scenario. The relevant acceptance check is: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

D.

Run a fixed sample of short and long requests, record measured tokens and unit cost by deployment and verify alerts respond to an approved budget threshold. That would help test “Track token consumption and model inference cost by feature and deployment”, not the outcome required in this scenario. The relevant acceptance check is: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

Question 464

A financial FAQ assistant is released weekly and needs an automated benchmark that stops a new prompt when citation accuracy drops on a fixed policy set. Which proposed shortcut most directly risks customer-facing hallucination regressions after untested prompt or model promotions and should therefore be rejected?

  1. Estimate generative AI cost using only the number of employees with access while ignoring model tokens, repeated retrieval context and retry behavior.
  2. Rate a support assistant as fully successful whenever it quotes authentic documentation even when its response never addresses the customer's actual question.
  3. Use GitHub Actions to deploy every prompt commit immediately and assume passing unit tests for repository syntax guarantees acceptable generative answer quality.
  4. Approve an agent solely because its average groundedness score is high and omit all harmful-content and confidential-data leakage evaluation cases.

Answer & Explanations

Correct answer: C

A.

Estimate generative AI cost using only the number of employees with access while ignoring model tokens, repeated retrieval context and retry behavior. That shortcut risks unexpected AI service bills after ignoring prompt length and token-volume drivers. Here the specific shortcut is: Use GitHub Actions to deploy every prompt commit immediately and assume passing unit tests for repository syntax guarantees acceptable generative answer quality.

B.

Rate a support assistant as fully successful whenever it quotes authentic documentation even when its response never addresses the customer's actual question. That shortcut risks unhelpful and off-target assistant output despite accurate retrieved reference material. Here the specific shortcut is: Use GitHub Actions to deploy every prompt commit immediately and assume passing unit tests for repository syntax guarantees acceptable generative answer quality.

C.

Use GitHub Actions to deploy every prompt commit immediately and assume passing unit tests for repository syntax guarantees acceptable generative answer quality. The foreseeable failure is customer-facing hallucination regressions after untested prompt or model promotions. The safer response is: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

D.

Approve an agent solely because its average groundedness score is high and omit all harmful-content and confidential-data leakage evaluation cases. That shortcut risks unsafe or privacy-violating assistant behavior passing an incomplete quality gate. Here the specific shortcut is: Use GitHub Actions to deploy every prompt commit immediately and assume passing unit tests for repository syntax guarantees acceptable generative answer quality.

Question 465

A financial FAQ assistant is released weekly and needs an automated benchmark that stops a new prompt when citation accuracy drops on a fixed policy set. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Verify: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.
  2. Implement: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Verify: Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.
  3. Implement: Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. Verify: Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets.
  4. Implement: Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. Verify: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

Answer & Explanations

Correct answer: A

A.

Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Confirm it through this test: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed. Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

B.

The action is applicable: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. But its proposed test—Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.—addresses “Measure response relevance and coherence against the original user intent”. This requirement is confirmed instead by: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

C.

This action/test pair addresses “Run safety and harmful-content evaluations on generative applications and agents”. Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets. The case instead calls for: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

D.

The test would be relevant: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed. But the offered implementation—Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness.—addresses “Measure response relevance and coherence against the original user intent”. The action needed here is: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

Question 466

A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Which Azure AI operations action best addresses this requirement?

  1. Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.
  2. Configure structured application logs and trace identifiers with appropriate redaction, sampling and access controls so engineering teams can debug model or retrieval failures without publishing confidential user inputs.
  3. Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness.
  4. Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

Answer & Explanations

Correct answer: D

A.

Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. This addresses “Evaluate groundedness independently from fluent natural-language style”, where the governing principle is: A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources. The stated case instead calls for: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

B.

Configure structured application logs and trace identifiers with appropriate redaction, sampling and access controls so engineering teams can debug model or retrieval failures without publishing confidential user inputs. This addresses “Log generative AI traces without leaking private prompts or credentials”, where the governing principle is: Request tracing can include sensitive prompts, retrieved records and tool arguments, so detailed observability needs privacy boundaries rather than unconditional logging of every raw conversation. The stated case instead calls for: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

C.

The question is set in this situation: A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. This addresses “Measure response relevance and coherence against the original user intent”, where the governing principle is: Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision. The stated case instead calls for: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

D.

The question is set in this situation: A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements. Verify the result with this test: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

Question 467

A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. The engineer proposes the following change: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Which mechanism explains why it works?

  1. A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.
  2. Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.
  3. Request tracing can include sensitive prompts, retrieved records and tool arguments, so detailed observability needs privacy boundaries rather than unconditional logging of every raw conversation.
  4. Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision.

Answer & Explanations

Correct answer: B

A.

A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources. That explains “Evaluate groundedness independently from fluent natural-language style”, not the mechanism behind the described change. The relevant principle is: Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

B.

The question is set in this situation: A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements. The corresponding implementation is: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

C.

Request tracing can include sensitive prompts, retrieved records and tool arguments, so detailed observability needs privacy boundaries rather than unconditional logging of every raw conversation. That explains “Log generative AI traces without leaking private prompts or credentials”, not the mechanism behind the described change. The relevant principle is: Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

D.

The question is set in this situation: A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision. That explains “Measure response relevance and coherence against the original user intent”, not the mechanism behind the described change. The relevant principle is: Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

Question 468

A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. After implementing this change—Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.—which test most directly validates its intended outcome?

  1. Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.
  2. Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.
  3. Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.
  4. Use a synthetic confidential fixture in staging, generate a failed agent call and verify traces retain failure codes and correlation IDs while suppressing the protected fields.

Answer & Explanations

Correct answer: C

A.

Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them. That would help test “Evaluate groundedness independently from fluent natural-language style”, not the outcome required in this scenario. The relevant acceptance check is: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

B.

The question is set in this situation: A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them. That would help test “Measure response relevance and coherence against the original user intent”, not the outcome required in this scenario. The relevant acceptance check is: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

C.

The question is set in this situation: A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed. This check is decisive because continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

D.

Use a synthetic confidential fixture in staging, generate a failed agent call and verify traces retain failure codes and correlation IDs while suppressing the protected fields. That would help test “Log generative AI traces without leaking private prompts or credentials”, not the outcome required in this scenario. The relevant acceptance check is: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

Question 469

A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Which proposed shortcut most directly risks customer-facing hallucination regressions after untested prompt or model promotions and should therefore be rejected?

  1. Rate a support assistant as fully successful whenever it quotes authentic documentation even when its response never addresses the customer's actual question.
  2. Send complete patient prompts, raw model API keys and every tool response to a publicly accessible diagnostic workspace for convenience.
  3. Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate.
  4. Use GitHub Actions to deploy every prompt commit immediately and assume passing unit tests for repository syntax guarantees acceptable generative answer quality.

Answer & Explanations

Correct answer: D

A.

The question is set in this situation: A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Rate a support assistant as fully successful whenever it quotes authentic documentation even when its response never addresses the customer's actual question. That shortcut risks unhelpful and off-target assistant output despite accurate retrieved reference material. Here the specific shortcut is: Use GitHub Actions to deploy every prompt commit immediately and assume passing unit tests for repository syntax guarantees acceptable generative answer quality.

B.

Send complete patient prompts, raw model API keys and every tool response to a publicly accessible diagnostic workspace for convenience. That shortcut risks privacy breach caused by uncontrolled generative AI diagnostic data collection. Here the specific shortcut is: Use GitHub Actions to deploy every prompt commit immediately and assume passing unit tests for repository syntax guarantees acceptable generative answer quality.

C.

Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate. That shortcut risks convincing hallucinations passing a language-quality check without evidence verification. Here the specific shortcut is: Use GitHub Actions to deploy every prompt commit immediately and assume passing unit tests for repository syntax guarantees acceptable generative answer quality.

D.

The question is set in this situation: A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Use GitHub Actions to deploy every prompt commit immediately and assume passing unit tests for repository syntax guarantees acceptable generative answer quality. The foreseeable failure is customer-facing hallucination regressions after untested prompt or model promotions. The safer response is: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

Question 470

A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Verify: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.
  2. Implement: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Verify: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.
  3. Implement: Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. Verify: Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.
  4. Implement: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. Verify: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

Answer & Explanations

Correct answer: B

A.

The action is applicable: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. But its proposed test—Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.—addresses “Evaluate groundedness independently from fluent natural-language style”. This requirement is confirmed instead by: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed.

B.

The question is set in this situation: A university chatbot team tests new retrieval models and wants a repeatable deployment gate for student safety and answer relevance rather than manual spot checks. Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress. Confirm it through this test: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed. Continuous evaluation links a proposed release to measured outputs; a successful infrastructure deployment does not prove a prompt or model change preserves previously reviewed behavioral requirements.

C.

This action/test pair addresses “Measure response relevance and coherence against the original user intent”. Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them. The case instead calls for: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

D.

The test would be relevant: Submit one deliberately degraded prompt to a CI evaluation job, confirm the regression gate fails and verify the stable production model and prompt remain deployed. But the offered implementation—Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.—addresses “Evaluate groundedness independently from fluent natural-language style”. The action needed here is: Run built-in or custom Foundry evaluations for an immutable model-prompt-retriever candidate, capture metrics by dataset version and block promotion when key quality or safety thresholds regress.

Operationalizing Machine Learning and Generative AI Solutions (AI-300) Practice Tests