Practice Exams:

Microsoft AI-300 Dumps

EXAM: Operationalizing Machine Learning and Generative AI Solutions
Verified By Experts
Premium Bundle
$49.99

AI-300 Premium ETE File


  • Premium File 213 Questions & Answers
Whats Included:
  • Latest Questions
  • 100% Accurate Answers
  • Fast Exam Updates

Microsoft AI-300 Practice Test 44

Question 430

A university research chatbot is being evaluated on a repository of course policies and must prove each benchmark answer uses the intended evidence record. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. Verify: Run a small evaluation with deliberately correct and incorrect candidate answers, inspect each dataset row's mapped input and confirm the expected scoring difference is recorded.
  2. Implement: Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. Verify: Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets.
  3. Implement: Assemble a versioned evaluation dataset with realistic user queries, approved reference context, expected facts or rubric labels and explicit column mapping to the Foundry evaluation workflow. Verify: Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.
  4. Implement: Assemble a versioned evaluation dataset with realistic user queries, approved reference context, expected facts or rubric labels and explicit column mapping to the Foundry evaluation workflow. Verify: Run a small evaluation with deliberately correct and incorrect candidate answers, inspect each dataset row's mapped input and confirm the expected scoring difference is recorded.

Answer & Explanations

Correct answer: D

A.

The test would be relevant: Run a small evaluation with deliberately correct and incorrect candidate answers, inspect each dataset row's mapped input and confirm the expected scoring difference is recorded. But the offered implementation—Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness.—addresses “Measure response relevance and coherence against the original user intent”. The action needed here is: Assemble a versioned evaluation dataset with realistic user queries, approved reference context, expected facts or rubric labels and explicit column mapping to the Foundry evaluation workflow.

B.

This action/test pair addresses “Run safety and harmful-content evaluations on generative applications and agents”. Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets. The case instead calls for: Assemble a versioned evaluation dataset with realistic user queries, approved reference context, expected facts or rubric labels and explicit column mapping to the Foundry evaluation workflow.

C.

The action is applicable: Assemble a versioned evaluation dataset with realistic user queries, approved reference context, expected facts or rubric labels and explicit column mapping to the Foundry evaluation workflow. But its proposed test—Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.—addresses “Measure response relevance and coherence against the original user intent”. This requirement is confirmed instead by: Run a small evaluation with deliberately correct and incorrect candidate answers, inspect each dataset row's mapped input and confirm the expected scoring difference is recorded.

D.

The question is set in this situation: A university research chatbot is being evaluated on a repository of course policies and must prove each benchmark answer uses the intended evidence record. Assemble a versioned evaluation dataset with realistic user queries, approved reference context, expected facts or rubric labels and explicit column mapping to the Foundry evaluation workflow. Confirm it through this test: Run a small evaluation with deliberately correct and incorrect candidate answers, inspect each dataset row's mapped input and confirm the expected scoring difference is recorded. An evaluator needs aligned inputs, ground truth and predicted outputs to produce interpretable scores; mixing mismatched question and reference rows can make a high score meaningless.

Question 431

A corporate legal assistant provides eloquent contract summaries but occasionally states obligations that do not appear in the retrieved agreements. Which Azure AI operations action best addresses this requirement?

  1. Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.
  2. Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness.
  3. Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant.
  4. Enable approved request tracing with correlation identifiers for the application, model inference, retrieval and external tool steps, and compare p50 and p95 timings to isolate slow stages.

Answer & Explanations

Correct answer: A

A.

Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources. Verify the result with this test: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

B.

Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. This addresses “Measure response relevance and coherence against the original user intent”, where the governing principle is: Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision. The stated case instead calls for: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

C.

Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. This addresses “Run safety and harmful-content evaluations on generative applications and agents”, where the governing principle is: Task-quality metrics cannot substitute for content safety checks; a response can be relevant and grounded while still revealing confidential data or producing prohibited instructions. The stated case instead calls for: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

D.

Enable approved request tracing with correlation identifiers for the application, model inference, retrieval and external tool steps, and compare p50 and p95 timings to isolate slow stages. This addresses “Trace end-to-end agent latency across model and tool calls”, where the governing principle is: End-to-end response time aggregates multiple stages, so blaming the model endpoint solely from high page latency ignores slow search indexes or failing downstream tools. The stated case instead calls for: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

Question 432

A corporate legal assistant provides eloquent contract summaries but occasionally states obligations that do not appear in the retrieved agreements. The engineer proposes the following change: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. Which mechanism explains why it works?

  1. Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision.
  2. A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.
  3. End-to-end response time aggregates multiple stages, so blaming the model endpoint solely from high page latency ignores slow search indexes or failing downstream tools.
  4. Task-quality metrics cannot substitute for content safety checks; a response can be relevant and grounded while still revealing confidential data or producing prohibited instructions.

Answer & Explanations

Correct answer: B

A.

Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision. That explains “Measure response relevance and coherence against the original user intent”, not the mechanism behind the described change. The relevant principle is: A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

B.

A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources. The corresponding implementation is: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

C.

End-to-end response time aggregates multiple stages, so blaming the model endpoint solely from high page latency ignores slow search indexes or failing downstream tools. That explains “Trace end-to-end agent latency across model and tool calls”, not the mechanism behind the described change. The relevant principle is: A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

D.

Task-quality metrics cannot substitute for content safety checks; a response can be relevant and grounded while still revealing confidential data or producing prohibited instructions. That explains “Run safety and harmful-content evaluations on generative applications and agents”, not the mechanism behind the described change. The relevant principle is: A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

Question 433

A corporate legal assistant provides eloquent contract summaries but occasionally states obligations that do not appear in the retrieved agreements. After implementing this change—Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.—which test most directly validates its intended outcome?

  1. Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets.
  2. Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.
  3. Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.
  4. Reproduce a staged slow external tool, inspect the trace waterfall and verify its duration accounts for the p95 latency increase before applying a narrowly targeted fix.

Answer & Explanations

Correct answer: C

A.

Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets. That would help test “Run safety and harmful-content evaluations on generative applications and agents”, not the outcome required in this scenario. The relevant acceptance check is: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

B.

Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them. That would help test “Measure response relevance and coherence against the original user intent”, not the outcome required in this scenario. The relevant acceptance check is: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

C.

Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them. This check is decisive because a language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

D.

Reproduce a staged slow external tool, inspect the trace waterfall and verify its duration accounts for the p95 latency increase before applying a narrowly targeted fix. That would help test “Trace end-to-end agent latency across model and tool calls”, not the outcome required in this scenario. The relevant acceptance check is: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

Question 434

A corporate legal assistant provides eloquent contract summaries but occasionally states obligations that do not appear in the retrieved agreements. Which proposed shortcut most directly risks convincing hallucinations passing a language-quality check without evidence verification and should therefore be rejected?

  1. Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate.
  2. Rate a support assistant as fully successful whenever it quotes authentic documentation even when its response never addresses the customer's actual question.
  3. Approve an agent solely because its average groundedness score is high and omit all harmful-content and confidential-data leakage evaluation cases.
  4. Double model token throughput whenever an agent is slow without inspecting whether the actual delay occurs in Azure AI Search or an external database tool.

Answer & Explanations

Correct answer: A

A.

Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate. The foreseeable failure is convincing hallucinations passing a language-quality check without evidence verification. The safer response is: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

B.

Rate a support assistant as fully successful whenever it quotes authentic documentation even when its response never addresses the customer's actual question. That shortcut risks unhelpful and off-target assistant output despite accurate retrieved reference material. Here the specific shortcut is: Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate.

C.

Approve an agent solely because its average groundedness score is high and omit all harmful-content and confidential-data leakage evaluation cases. That shortcut risks unsafe or privacy-violating assistant behavior passing an incomplete quality gate. Here the specific shortcut is: Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate.

D.

Double model token throughput whenever an agent is slow without inspecting whether the actual delay occurs in Azure AI Search or an external database tool. That shortcut risks expensive capacity changes that do not resolve agent response latency. Here the specific shortcut is: Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate.

Question 435

A corporate legal assistant provides eloquent contract summaries but occasionally states obligations that do not appear in the retrieved agreements. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. Verify: Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.
  2. Implement: Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. Verify: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.
  3. Implement: Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. Verify: Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets.
  4. Implement: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. Verify: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

Answer & Explanations

Correct answer: D

A.

The action is applicable: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. But its proposed test—Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.—addresses “Measure response relevance and coherence against the original user intent”. This requirement is confirmed instead by: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

B.

The test would be relevant: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them. But the offered implementation—Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness.—addresses “Measure response relevance and coherence against the original user intent”. The action needed here is: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

C.

This action/test pair addresses “Run safety and harmful-content evaluations on generative applications and agents”. Configure supported risk and safety evaluators for prohibited content, unsafe instructions and sensitive data exposure, using approved adversarial fixtures and documented thresholds before releasing an AI assistant. Run a benign authorized question and a controlled disallowed prompt against the staging deployment, review safety evaluator outputs and ensure the unsafe result blocks release without storing real secrets. The case instead calls for: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

D.

Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. Confirm it through this test: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them. A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

Question 436

A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Which Azure AI operations action best addresses this requirement?

  1. Assemble a versioned evaluation dataset with realistic user queries, approved reference context, expected facts or rubric labels and explicit column mapping to the Foundry evaluation workflow.
  2. Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.
  3. Instrument prompt and completion token usage, model deployment pricing or quota, retries and per-request attribution so GenAIOps can set budgets and improve inefficient prompts.
  4. Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness.

Answer & Explanations

Correct answer: B

A.

Assemble a versioned evaluation dataset with realistic user queries, approved reference context, expected facts or rubric labels and explicit column mapping to the Foundry evaluation workflow. This addresses “Construct a representative generative AI evaluation dataset with mapped fields”, where the governing principle is: An evaluator needs aligned inputs, ground truth and predicted outputs to produce interpretable scores; mixing mismatched question and reference rows can make a high score meaningless. The stated case instead calls for: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

B.

The question is set in this situation: A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources. Verify the result with this test: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

C.

Instrument prompt and completion token usage, model deployment pricing or quota, retries and per-request attribution so GenAIOps can set budgets and improve inefficient prompts. This addresses “Track token consumption and model inference cost by feature and deployment”, where the governing principle is: Token-based charges depend on input and output usage and repeated requests; a low count of web page visits can still generate large costs from oversized context and retries. The stated case instead calls for: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

D.

The question is set in this situation: A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. This addresses “Measure response relevance and coherence against the original user intent”, where the governing principle is: Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision. The stated case instead calls for: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

Question 437

A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. The engineer proposes the following change: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. Which mechanism explains why it works?

  1. Token-based charges depend on input and output usage and repeated requests; a low count of web page visits can still generate large costs from oversized context and retries.
  2. Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision.
  3. An evaluator needs aligned inputs, ground truth and predicted outputs to produce interpretable scores; mixing mismatched question and reference rows can make a high score meaningless.
  4. A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

Answer & Explanations

Correct answer: D

A.

Token-based charges depend on input and output usage and repeated requests; a low count of web page visits can still generate large costs from oversized context and retries. That explains “Track token consumption and model inference cost by feature and deployment”, not the mechanism behind the described change. The relevant principle is: A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

B.

The question is set in this situation: A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Relevance measures alignment to the user request, while coherence describes internal intelligibility; a perfectly grounded paragraph can still fail by not answering the requested decision. That explains “Measure response relevance and coherence against the original user intent”, not the mechanism behind the described change. The relevant principle is: A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

C.

An evaluator needs aligned inputs, ground truth and predicted outputs to produce interpretable scores; mixing mismatched question and reference rows can make a high score meaningless. That explains “Construct a representative generative AI evaluation dataset with mapped fields”, not the mechanism behind the described change. The relevant principle is: A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

D.

The question is set in this situation: A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources. The corresponding implementation is: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

Question 438

A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. After implementing this change—Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.—which test most directly validates its intended outcome?

  1. Run a fixed sample of short and long requests, record measured tokens and unit cost by deployment and verify alerts respond to an approved budget threshold.
  2. Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.
  3. Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.
  4. Run a small evaluation with deliberately correct and incorrect candidate answers, inspect each dataset row's mapped input and confirm the expected scoring difference is recorded.

Answer & Explanations

Correct answer: B

A.

Run a fixed sample of short and long requests, record measured tokens and unit cost by deployment and verify alerts respond to an approved budget threshold. That would help test “Track token consumption and model inference cost by feature and deployment”, not the outcome required in this scenario. The relevant acceptance check is: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

B.

The question is set in this situation: A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them. This check is decisive because a language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

C.

The question is set in this situation: A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them. That would help test “Measure response relevance and coherence against the original user intent”, not the outcome required in this scenario. The relevant acceptance check is: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

D.

Run a small evaluation with deliberately correct and incorrect candidate answers, inspect each dataset row's mapped input and confirm the expected scoring difference is recorded. That would help test “Construct a representative generative AI evaluation dataset with mapped fields”, not the outcome required in this scenario. The relevant acceptance check is: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

Question 439

A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Which proposed shortcut most directly risks convincing hallucinations passing a language-quality check without evidence verification and should therefore be rejected?

  1. Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate.
  2. Estimate generative AI cost using only the number of employees with access while ignoring model tokens, repeated retrieval context and retry behavior.
  3. Feed an evaluation tool a CSV of unlabeled questions and randomly ordered answers while assuming column names automatically supply trustworthy references.
  4. Rate a support assistant as fully successful whenever it quotes authentic documentation even when its response never addresses the customer's actual question.

Answer & Explanations

Correct answer: A

A.

The question is set in this situation: A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate. The foreseeable failure is convincing hallucinations passing a language-quality check without evidence verification. The safer response is: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

B.

Estimate generative AI cost using only the number of employees with access while ignoring model tokens, repeated retrieval context and retry behavior. That shortcut risks unexpected AI service bills after ignoring prompt length and token-volume drivers. Here the specific shortcut is: Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate.

C.

Feed an evaluation tool a CSV of unlabeled questions and randomly ordered answers while assuming column names automatically supply trustworthy references. That shortcut risks misleading assistant quality results caused by incorrectly aligned evaluation data. Here the specific shortcut is: Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate.

D.

The question is set in this situation: A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Rate a support assistant as fully successful whenever it quotes authentic documentation even when its response never addresses the customer's actual question. That shortcut risks unhelpful and off-target assistant output despite accurate retrieved reference material. Here the specific shortcut is: Approve a legal research assistant merely because its generated paragraphs read naturally and assume polished writing proves every citation is accurate.

Question 440

A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. Verify: Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them.
  2. Implement: Assemble a versioned evaluation dataset with realistic user queries, approved reference context, expected facts or rubric labels and explicit column mapping to the Foundry evaluation workflow. Verify: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.
  3. Implement: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. Verify: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.
  4. Implement: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. Verify: Run a small evaluation with deliberately correct and incorrect candidate answers, inspect each dataset row's mapped input and confirm the expected scoring difference is recorded.

Answer & Explanations

Correct answer: C

A.

This action/test pair addresses “Measure response relevance and coherence against the original user intent”. Evaluate generated answers for whether they address the specific question, maintain coherent reasoning and avoid irrelevant digressions using agreed rubric metrics alongside factual correctness. Create two test outputs using the same true evidence, one that answers the user's target and one that discusses only a related policy, then confirm the relevance rubric separates them. The case instead calls for: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

B.

The test would be relevant: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them. But the offered implementation—Assemble a versioned evaluation dataset with realistic user queries, approved reference context, expected facts or rubric labels and explicit column mapping to the Foundry evaluation workflow.—addresses “Construct a representative generative AI evaluation dataset with mapped fields”. The action needed here is: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency.

C.

The question is set in this situation: A city service bot cites local regulations and needs an evaluation metric that flags fabricated penalties even when its sentences are well written. Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. Confirm it through this test: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them. A language model can produce clear grammatical prose while inventing facts absent from the retrieval context; fluency is not proof that statements are supported by sources.

D.

The action is applicable: Score an AI response for supported factual claims against its retrieved source text and citation evidence, distinguishing groundedness from coherence or linguistic fluency. But its proposed test—Run a small evaluation with deliberately correct and incorrect candidate answers, inspect each dataset row's mapped input and confirm the expected scoring difference is recorded.—addresses “Construct a representative generative AI evaluation dataset with mapped fields”. This requirement is confirmed instead by: Include one fluent but deliberately unsupported assertion and one awkward but source-supported answer in a controlled benchmark, then verify groundedness and fluency metrics differentiate them.

Operationalizing Machine Learning and Generative AI Solutions (AI-300) Practice Tests