Practice Exams:

Microsoft AI-300 Dumps

EXAM: Operationalizing Machine Learning and Generative AI Solutions
Verified By Experts
Premium Bundle
$49.99

AI-300 Premium ETE File


  • Premium File 213 Questions & Answers
Whats Included:
  • Latest Questions
  • 100% Accurate Answers
  • Fast Exam Updates

Microsoft AI-300 Practice Test 57

Question 560

A healthcare benefits assistant compares two RAG pipelines and must protect medical-policy answer correctness while experimenting with query expansion. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment. Verify: Compare at least two chunking strategies on a labeled test query set, compute evidence recall and citation precision and verify the selected chunks support complete answers.
  2. Implement: Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment. Verify: Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen.
  3. Implement: Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language. Verify: Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines.
  4. Implement: Adjust document chunk size, semantic boundaries and overlap based on the target question types and retriever behavior, preserving source identifiers to enable citation verification. Verify: Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen.

Answer & Explanations

Correct answer: B

A.

The action is applicable: Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment. But its proposed test—Compare at least two chunking strategies on a labeled test query set, compute evidence recall and citation precision and verify the selected chunks support complete answers.—addresses “Tune RAG chunk boundaries and overlap using real retrieval relevance tests”. This requirement is confirmed instead by: Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen.

B.

The question is set in this situation: A healthcare benefits assistant compares two RAG pipelines and must protect medical-policy answer correctness while experimenting with query expansion. Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment. Confirm it through this test: Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen. Improved vector similarity scores alone may not mean better final answers; chunking, hybrid ranking and prompt context selection affect end-to-end usefulness and citation quality.

C.

This action/test pair addresses “Combine semantic vector retrieval with keyword retrieval for exact identifiers”. Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language. Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines. The case instead calls for: Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment.

D.

The test would be relevant: Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen. But the offered implementation—Adjust document chunk size, semantic boundaries and overlap based on the target question types and retriever behavior, preserving source identifiers to enable citation verification.—addresses “Tune RAG chunk boundaries and overlap using real retrieval relevance tests”. The action needed here is: Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment.

Question 561

A customer service assistant needs a consistent structured response tone but its product pricing changes daily in a database, requiring a choice between behavior tuning and retrieval. Which Azure AI operations action best addresses this requirement?

  1. Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.
  2. Produce task-specific synthetic examples only from approved nonsecret sources, label their origin and quality, deduplicate against evaluation sets and review potentially unsafe or fabricated samples before training.
  3. Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language.
  4. Register the tuned model and dataset provenance, deploy a candidate version with controlled traffic, monitor task success and safety metrics and retain an approved rollback path to its previous model.

Answer & Explanations

Correct answer: A

A.

Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization. Verify the result with this test: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

B.

Produce task-specific synthetic examples only from approved nonsecret sources, label their origin and quality, deduplicate against evaluation sets and review potentially unsafe or fabricated samples before training. This addresses “Generate synthetic fine-tuning data with provenance and leakage safeguards”, where the governing principle is: Synthetic data can amplify biases and memorize sensitive prompts; generating many samples is not proof that examples are accurate or independent of a future held-out benchmark. The stated case instead calls for: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

C.

Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language. This addresses “Combine semantic vector retrieval with keyword retrieval for exact identifiers”, where the governing principle is: Embeddings help match meaning but may blur precise identifiers; lexical retrieval finds exact tokens, and hybrid ranking can cover both cases without depending on one retrieval method. The stated case instead calls for: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

D.

Register the tuned model and dataset provenance, deploy a candidate version with controlled traffic, monitor task success and safety metrics and retain an approved rollback path to its previous model. This addresses “Manage fine-tuned model versions with staged rollout and regression monitoring”, where the governing principle is: A tuned model can drift in application performance after user mix changes, and deleting the base deployment before the new release is proven removes an operational recovery option. The stated case instead calls for: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

Question 562

A customer service assistant needs a consistent structured response tone but its product pricing changes daily in a database, requiring a choice between behavior tuning and retrieval. The engineer proposes the following change: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. Which mechanism explains why it works?

  1. A tuned model can drift in application performance after user mix changes, and deleting the base deployment before the new release is proven removes an operational recovery option.
  2. Embeddings help match meaning but may blur precise identifiers; lexical retrieval finds exact tokens, and hybrid ranking can cover both cases without depending on one retrieval method.
  3. Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.
  4. Synthetic data can amplify biases and memorize sensitive prompts; generating many samples is not proof that examples are accurate or independent of a future held-out benchmark.

Answer & Explanations

Correct answer: C

A.

A tuned model can drift in application performance after user mix changes, and deleting the base deployment before the new release is proven removes an operational recovery option. That explains “Manage fine-tuned model versions with staged rollout and regression monitoring”, not the mechanism behind the described change. The relevant principle is: Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

B.

Embeddings help match meaning but may blur precise identifiers; lexical retrieval finds exact tokens, and hybrid ranking can cover both cases without depending on one retrieval method. That explains “Combine semantic vector retrieval with keyword retrieval for exact identifiers”, not the mechanism behind the described change. The relevant principle is: Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

C.

Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization. The corresponding implementation is: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

D.

Synthetic data can amplify biases and memorize sensitive prompts; generating many samples is not proof that examples are accurate or independent of a future held-out benchmark. That explains “Generate synthetic fine-tuning data with provenance and leakage safeguards”, not the mechanism behind the described change. The relevant principle is: Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

Question 563

A customer service assistant needs a consistent structured response tone but its product pricing changes daily in a database, requiring a choice between behavior tuning and retrieval. After implementing this change—Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.—which test most directly validates its intended outcome?

  1. Inspect a random sample and the full automated source-provenance ledger, verify no reserved evaluation fixture is in training and check for protected terms before launching a fine-tune.
  2. Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.
  3. Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines.
  4. Canary a fixed fraction of permissible traffic to the fine-tuned candidate, compare errors and behavioral metrics by version and rehearse rerouting all traffic to the stable deployment.

Answer & Explanations

Correct answer: B

A.

Inspect a random sample and the full automated source-provenance ledger, verify no reserved evaluation fixture is in training and check for protected terms before launching a fine-tune. That would help test “Generate synthetic fine-tuning data with provenance and leakage safeguards”, not the outcome required in this scenario. The relevant acceptance check is: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

B.

Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics. This check is decisive because fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

C.

Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines. That would help test “Combine semantic vector retrieval with keyword retrieval for exact identifiers”, not the outcome required in this scenario. The relevant acceptance check is: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

D.

Canary a fixed fraction of permissible traffic to the fine-tuned candidate, compare errors and behavioral metrics by version and rehearse rerouting all traffic to the stable deployment. That would help test “Manage fine-tuned model versions with staged rollout and regression monitoring”, not the outcome required in this scenario. The relevant acceptance check is: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

Question 564

A customer service assistant needs a consistent structured response tone but its product pricing changes daily in a database, requiring a choice between behavior tuning and retrieval. Which proposed shortcut most directly risks stale factual answers and unnecessary training cost after using fine-tuning to replace current knowledge access and should therefore be rejected?

  1. Use embedding similarity alone for exact invoice IDs and assume semantically related documents cannot outrank a record with the correct identifier.
  2. Replace the only production foundation model with a just-finished fine-tune and discard its preceding version before observing live quality or policy compliance.
  3. Feed unreviewed production customer chats and copied evaluation answers into a synthetic fine-tuning generator without privacy screening or train-test separation.
  4. Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval.

Answer & Explanations

Correct answer: D

A.

Use embedding similarity alone for exact invoice IDs and assume semantically related documents cannot outrank a record with the correct identifier. That shortcut risks wrong retrieved records for exact business identifiers despite plausible semantic matches. Here the specific shortcut is: Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval.

B.

Replace the only production foundation model with a just-finished fine-tune and discard its preceding version before observing live quality or policy compliance. That shortcut risks unrecoverable generative AI regressions caused by tuning without controlled promotion. Here the specific shortcut is: Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval.

C.

Feed unreviewed production customer chats and copied evaluation answers into a synthetic fine-tuning generator without privacy screening or train-test separation. That shortcut risks privacy exposure and inflated tuning metrics from data contamination. Here the specific shortcut is: Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval.

D.

Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval. The foreseeable failure is stale factual answers and unnecessary training cost after using fine-tuning to replace current knowledge access. The safer response is: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

Question 565

A customer service assistant needs a consistent structured response tone but its product pricing changes daily in a database, requiring a choice between behavior tuning and retrieval. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. Verify: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.
  2. Implement: Register the tuned model and dataset provenance, deploy a candidate version with controlled traffic, monitor task success and safety metrics and retain an approved rollback path to its previous model. Verify: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.
  3. Implement: Produce task-specific synthetic examples only from approved nonsecret sources, label their origin and quality, deduplicate against evaluation sets and review potentially unsafe or fabricated samples before training. Verify: Inspect a random sample and the full automated source-provenance ledger, verify no reserved evaluation fixture is in training and check for protected terms before launching a fine-tune.
  4. Implement: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. Verify: Canary a fixed fraction of permissible traffic to the fine-tuned candidate, compare errors and behavioral metrics by version and rehearse rerouting all traffic to the stable deployment.

Answer & Explanations

Correct answer: A

A.

Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. Confirm it through this test: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics. Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

B.

The test would be relevant: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics. But the offered implementation—Register the tuned model and dataset provenance, deploy a candidate version with controlled traffic, monitor task success and safety metrics and retain an approved rollback path to its previous model.—addresses “Manage fine-tuned model versions with staged rollout and regression monitoring”. The action needed here is: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

C.

This action/test pair addresses “Generate synthetic fine-tuning data with provenance and leakage safeguards”. Produce task-specific synthetic examples only from approved nonsecret sources, label their origin and quality, deduplicate against evaluation sets and review potentially unsafe or fabricated samples before training. Inspect a random sample and the full automated source-provenance ledger, verify no reserved evaluation fixture is in training and check for protected terms before launching a fine-tune. The case instead calls for: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

D.

The action is applicable: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. But its proposed test—Canary a fixed fraction of permissible traffic to the fine-tuned candidate, compare errors and behavioral metrics by version and rehearse rerouting all traffic to the stable deployment.—addresses “Manage fine-tuned model versions with staged rollout and regression monitoring”. This requirement is confirmed instead by: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

Question 566

A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Which Azure AI operations action best addresses this requirement?

  1. Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language.
  2. Register the tuned model and dataset provenance, deploy a candidate version with controlled traffic, monitor task success and safety metrics and retain an approved rollback path to its previous model.
  3. Produce task-specific synthetic examples only from approved nonsecret sources, label their origin and quality, deduplicate against evaluation sets and review potentially unsafe or fabricated samples before training.
  4. Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

Answer & Explanations

Correct answer: D

A.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language. This addresses “Combine semantic vector retrieval with keyword retrieval for exact identifiers”, where the governing principle is: Embeddings help match meaning but may blur precise identifiers; lexical retrieval finds exact tokens, and hybrid ranking can cover both cases without depending on one retrieval method. The stated case instead calls for: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

B.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Register the tuned model and dataset provenance, deploy a candidate version with controlled traffic, monitor task success and safety metrics and retain an approved rollback path to its previous model. This addresses “Manage fine-tuned model versions with staged rollout and regression monitoring”, where the governing principle is: A tuned model can drift in application performance after user mix changes, and deleting the base deployment before the new release is proven removes an operational recovery option. The stated case instead calls for: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

C.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Produce task-specific synthetic examples only from approved nonsecret sources, label their origin and quality, deduplicate against evaluation sets and review potentially unsafe or fabricated samples before training. This addresses “Generate synthetic fine-tuning data with provenance and leakage safeguards”, where the governing principle is: Synthetic data can amplify biases and memorize sensitive prompts; generating many samples is not proof that examples are accurate or independent of a future held-out benchmark. The stated case instead calls for: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

D.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization. Verify the result with this test: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

Question 567

A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. The engineer proposes the following change: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. Which mechanism explains why it works?

  1. Synthetic data can amplify biases and memorize sensitive prompts; generating many samples is not proof that examples are accurate or independent of a future held-out benchmark.
  2. Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.
  3. Embeddings help match meaning but may blur precise identifiers; lexical retrieval finds exact tokens, and hybrid ranking can cover both cases without depending on one retrieval method.
  4. A tuned model can drift in application performance after user mix changes, and deleting the base deployment before the new release is proven removes an operational recovery option.

Answer & Explanations

Correct answer: B

A.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Synthetic data can amplify biases and memorize sensitive prompts; generating many samples is not proof that examples are accurate or independent of a future held-out benchmark. That explains “Generate synthetic fine-tuning data with provenance and leakage safeguards”, not the mechanism behind the described change. The relevant principle is: Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

B.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization. The corresponding implementation is: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

C.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Embeddings help match meaning but may blur precise identifiers; lexical retrieval finds exact tokens, and hybrid ranking can cover both cases without depending on one retrieval method. That explains “Combine semantic vector retrieval with keyword retrieval for exact identifiers”, not the mechanism behind the described change. The relevant principle is: Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

D.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. A tuned model can drift in application performance after user mix changes, and deleting the base deployment before the new release is proven removes an operational recovery option. That explains “Manage fine-tuned model versions with staged rollout and regression monitoring”, not the mechanism behind the described change. The relevant principle is: Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

Question 568

A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. After implementing this change—Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.—which test most directly validates its intended outcome?

  1. Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines.
  2. Inspect a random sample and the full automated source-provenance ledger, verify no reserved evaluation fixture is in training and check for protected terms before launching a fine-tune.
  3. Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.
  4. Canary a fixed fraction of permissible traffic to the fine-tuned candidate, compare errors and behavioral metrics by version and rehearse rerouting all traffic to the stable deployment.

Answer & Explanations

Correct answer: C

A.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines. That would help test “Combine semantic vector retrieval with keyword retrieval for exact identifiers”, not the outcome required in this scenario. The relevant acceptance check is: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

B.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Inspect a random sample and the full automated source-provenance ledger, verify no reserved evaluation fixture is in training and check for protected terms before launching a fine-tune. That would help test “Generate synthetic fine-tuning data with provenance and leakage safeguards”, not the outcome required in this scenario. The relevant acceptance check is: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

C.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics. This check is decisive because fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

D.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Canary a fixed fraction of permissible traffic to the fine-tuned candidate, compare errors and behavioral metrics by version and rehearse rerouting all traffic to the stable deployment. That would help test “Manage fine-tuned model versions with staged rollout and regression monitoring”, not the outcome required in this scenario. The relevant acceptance check is: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

Question 569

A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Which proposed shortcut most directly risks stale factual answers and unnecessary training cost after using fine-tuning to replace current knowledge access and should therefore be rejected?

  1. Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval.
  2. Use embedding similarity alone for exact invoice IDs and assume semantically related documents cannot outrank a record with the correct identifier.
  3. Feed unreviewed production customer chats and copied evaluation answers into a synthetic fine-tuning generator without privacy screening or train-test separation.
  4. Replace the only production foundation model with a just-finished fine-tune and discard its preceding version before observing live quality or policy compliance.

Answer & Explanations

Correct answer: A

A.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval. The foreseeable failure is stale factual answers and unnecessary training cost after using fine-tuning to replace current knowledge access. The safer response is: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

B.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Use embedding similarity alone for exact invoice IDs and assume semantically related documents cannot outrank a record with the correct identifier. That shortcut risks wrong retrieved records for exact business identifiers despite plausible semantic matches. Here the specific shortcut is: Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval.

C.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Feed unreviewed production customer chats and copied evaluation answers into a synthetic fine-tuning generator without privacy screening or train-test separation. That shortcut risks privacy exposure and inflated tuning metrics from data contamination. Here the specific shortcut is: Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval.

D.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Replace the only production foundation model with a just-finished fine-tune and discard its preceding version before observing live quality or policy compliance. That shortcut risks unrecoverable generative AI regressions caused by tuning without controlled promotion. Here the specific shortcut is: Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval.

Question 570

A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Produce task-specific synthetic examples only from approved nonsecret sources, label their origin and quality, deduplicate against evaluation sets and review potentially unsafe or fabricated samples before training. Verify: Inspect a random sample and the full automated source-provenance ledger, verify no reserved evaluation fixture is in training and check for protected terms before launching a fine-tune.
  2. Implement: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. Verify: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.
  3. Implement: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. Verify: Canary a fixed fraction of permissible traffic to the fine-tuned candidate, compare errors and behavioral metrics by version and rehearse rerouting all traffic to the stable deployment.
  4. Implement: Register the tuned model and dataset provenance, deploy a candidate version with controlled traffic, monitor task success and safety metrics and retain an approved rollback path to its previous model. Verify: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

Answer & Explanations

Correct answer: B

A.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. This action/test pair addresses “Generate synthetic fine-tuning data with provenance and leakage safeguards”. Produce task-specific synthetic examples only from approved nonsecret sources, label their origin and quality, deduplicate against evaluation sets and review potentially unsafe or fabricated samples before training. Inspect a random sample and the full automated source-provenance ledger, verify no reserved evaluation fixture is in training and check for protected terms before launching a fine-tune. The case instead calls for: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

B.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. Confirm it through this test: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics. Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

C.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. The action is applicable: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. But its proposed test—Canary a fixed fraction of permissible traffic to the fine-tuned candidate, compare errors and behavioral metrics by version and rehearse rerouting all traffic to the stable deployment.—addresses “Manage fine-tuned model versions with staged rollout and regression monitoring”. This requirement is confirmed instead by: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.

D.

The question is set in this situation: A law office wants better contract clause classification and must decide whether curated supervised examples help more than simply expanding a retrieval index. The test would be relevant: Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics. But the offered implementation—Register the tuned model and dataset provenance, deploy a candidate version with controlled traffic, monitor task success and safety metrics and retain an approved rollback path to its previous model.—addresses “Manage fine-tuned model versions with staged rollout and regression monitoring”. The action needed here is: Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.

Operationalizing Machine Learning and Generative AI Solutions (AI-300) Practice Tests