Practice Exams:

Microsoft AI-300 Dumps

EXAM: Operationalizing Machine Learning and Generative AI Solutions
Verified By Experts
Premium Bundle
$49.99

AI-300 Premium ETE File


  • Premium File 213 Questions & Answers
Whats Included:
  • Latest Questions
  • 100% Accurate Answers
  • Fast Exam Updates

Microsoft AI-300 Practice Test 54

Question 530

A regulatory team searches policy clauses using formal section numbers as well as colloquial descriptions and needs hybrid retrieval to avoid a wrong citation. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language. Verify: Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen.
  2. Implement: Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language. Verify: Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines.
  3. Implement: Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment. Verify: Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines.
  4. Implement: Adjust document chunk size, semantic boundaries and overlap based on the target question types and retriever behavior, preserving source identifiers to enable citation verification. Verify: Compare at least two chunking strategies on a labeled test query set, compute evidence recall and citation precision and verify the selected chunks support complete answers.

Answer & Explanations

Correct answer: B

A.

The action is applicable: Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language. But its proposed test—Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen.—addresses “Evaluate RAG changes using labeled relevance and controlled A/B traffic”. This requirement is confirmed instead by: Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines.

B.

The question is set in this situation: A regulatory team searches policy clauses using formal section numbers as well as colloquial descriptions and needs hybrid retrieval to avoid a wrong citation. Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language. Confirm it through this test: Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines. Embeddings help match meaning but may blur precise identifiers; lexical retrieval finds exact tokens, and hybrid ranking can cover both cases without depending on one retrieval method.

C.

The test would be relevant: Evaluate paired natural-language and exact-ID queries against a labeled document corpus, compare top-k recall and reranking quality with vector-only or keyword-only baselines. But the offered implementation—Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment.—addresses “Evaluate RAG changes using labeled relevance and controlled A/B traffic”. The action needed here is: Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language.

D.

This action/test pair addresses “Tune RAG chunk boundaries and overlap using real retrieval relevance tests”. Adjust document chunk size, semantic boundaries and overlap based on the target question types and retriever behavior, preserving source identifiers to enable citation verification. Compare at least two chunking strategies on a labeled test query set, compute evidence recall and citation precision and verify the selected chunks support complete answers. The case instead calls for: Use a supported hybrid search strategy combining vector similarity and lexical term matching with calibrated merging or reranking when queries contain exact part numbers, legal clauses or conceptual language.

Question 531

A multi-tenant HR assistant must retrieve each company's employee handbook but prevent an unrelated tenant's handbook from appearing as context. Which Azure AI operations action best addresses this requirement?

  1. Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment.
  2. Register the tuned model and dataset provenance, deploy a candidate version with controlled traffic, monitor task success and safety metrics and retain an approved rollback path to its previous model.
  3. Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.
  4. Adjust document chunk size, semantic boundaries and overlap based on the target question types and retriever behavior, preserving source identifiers to enable citation verification.

Answer & Explanations

Correct answer: C

A.

Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment. This addresses “Evaluate RAG changes using labeled relevance and controlled A/B traffic”, where the governing principle is: Improved vector similarity scores alone may not mean better final answers; chunking, hybrid ranking and prompt context selection affect end-to-end usefulness and citation quality. The stated case instead calls for: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

B.

Register the tuned model and dataset provenance, deploy a candidate version with controlled traffic, monitor task success and safety metrics and retain an approved rollback path to its previous model. This addresses “Manage fine-tuned model versions with staged rollout and regression monitoring”, where the governing principle is: A tuned model can drift in application performance after user mix changes, and deleting the base deployment before the new release is proven removes an operational recovery option. The stated case instead calls for: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

C.

Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents. Verify the result with this test: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

D.

Adjust document chunk size, semantic boundaries and overlap based on the target question types and retriever behavior, preserving source identifiers to enable citation verification. This addresses “Tune RAG chunk boundaries and overlap using real retrieval relevance tests”, where the governing principle is: Very small chunks can omit necessary context while overly large chunks dilute embedding specificity or waste model context; fixed arbitrary chunk sizes are not universally optimal. The stated case instead calls for: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

Question 532

A multi-tenant HR assistant must retrieve each company's employee handbook but prevent an unrelated tenant's handbook from appearing as context. The engineer proposes the following change: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. Which mechanism explains why it works?

  1. Very small chunks can omit necessary context while overly large chunks dilute embedding specificity or waste model context; fixed arbitrary chunk sizes are not universally optimal.
  2. Improved vector similarity scores alone may not mean better final answers; chunking, hybrid ranking and prompt context selection affect end-to-end usefulness and citation quality.
  3. A tuned model can drift in application performance after user mix changes, and deleting the base deployment before the new release is proven removes an operational recovery option.
  4. A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

Answer & Explanations

Correct answer: D

A.

Very small chunks can omit necessary context while overly large chunks dilute embedding specificity or waste model context; fixed arbitrary chunk sizes are not universally optimal. That explains “Tune RAG chunk boundaries and overlap using real retrieval relevance tests”, not the mechanism behind the described change. The relevant principle is: A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

B.

Improved vector similarity scores alone may not mean better final answers; chunking, hybrid ranking and prompt context selection affect end-to-end usefulness and citation quality. That explains “Evaluate RAG changes using labeled relevance and controlled A/B traffic”, not the mechanism behind the described change. The relevant principle is: A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

C.

A tuned model can drift in application performance after user mix changes, and deleting the base deployment before the new release is proven removes an operational recovery option. That explains “Manage fine-tuned model versions with staged rollout and regression monitoring”, not the mechanism behind the described change. The relevant principle is: A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

D.

A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents. The corresponding implementation is: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

Question 533

A multi-tenant HR assistant must retrieve each company's employee handbook but prevent an unrelated tenant's handbook from appearing as context. After implementing this change—Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.—which test most directly validates its intended outcome?

  1. Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen.
  2. Compare at least two chunking strategies on a labeled test query set, compute evidence recall and citation precision and verify the selected chunks support complete answers.
  3. Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.
  4. Canary a fixed fraction of permissible traffic to the fine-tuned candidate, compare errors and behavioral metrics by version and rehearse rerouting all traffic to the stable deployment.

Answer & Explanations

Correct answer: C

A.

Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen. That would help test “Evaluate RAG changes using labeled relevance and controlled A/B traffic”, not the outcome required in this scenario. The relevant acceptance check is: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

B.

Compare at least two chunking strategies on a labeled test query set, compute evidence recall and citation precision and verify the selected chunks support complete answers. That would help test “Tune RAG chunk boundaries and overlap using real retrieval relevance tests”, not the outcome required in this scenario. The relevant acceptance check is: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

C.

Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt. This check is decisive because a vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

D.

Canary a fixed fraction of permissible traffic to the fine-tuned candidate, compare errors and behavioral metrics by version and rehearse rerouting all traffic to the stable deployment. That would help test “Manage fine-tuned model versions with staged rollout and regression monitoring”, not the outcome required in this scenario. The relevant acceptance check is: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

Question 534

A multi-tenant HR assistant must retrieve each company's employee handbook but prevent an unrelated tenant's handbook from appearing as context. Which proposed shortcut most directly risks hallucinated or cross-tenant answers from inappropriate retrieval filtering and should therefore be rejected?

  1. Always pass the highest-ranked vector result into the model even when its score is weak or it belongs to another customer's document collection.
  2. Promote a new retriever merely because its embedding average cosine similarity increased without measuring factual answer quality or citation correctness.
  3. Split every contract into one giant vector chunk or tiny unrelated fragments and assume either extreme always maximizes accurate evidence retrieval.
  4. Replace the only production foundation model with a just-finished fine-tune and discard its preceding version before observing live quality or policy compliance.

Answer & Explanations

Correct answer: A

A.

Always pass the highest-ranked vector result into the model even when its score is weak or it belongs to another customer's document collection. The foreseeable failure is hallucinated or cross-tenant answers from inappropriate retrieval filtering. The safer response is: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

B.

Promote a new retriever merely because its embedding average cosine similarity increased without measuring factual answer quality or citation correctness. That shortcut risks user-facing answer regressions after choosing retrieval settings from a single internal similarity score. Here the specific shortcut is: Always pass the highest-ranked vector result into the model even when its score is weak or it belongs to another customer's document collection.

C.

Split every contract into one giant vector chunk or tiny unrelated fragments and assume either extreme always maximizes accurate evidence retrieval. That shortcut risks unsupported answers after retrieval chunks drop or bury the crucial policy context. Here the specific shortcut is: Always pass the highest-ranked vector result into the model even when its score is weak or it belongs to another customer's document collection.

D.

Replace the only production foundation model with a just-finished fine-tune and discard its preceding version before observing live quality or policy compliance. That shortcut risks unrecoverable generative AI regressions caused by tuning without controlled promotion. Here the specific shortcut is: Always pass the highest-ranked vector result into the model even when its score is weak or it belongs to another customer's document collection.

Question 535

A multi-tenant HR assistant must retrieve each company's employee handbook but prevent an unrelated tenant's handbook from appearing as context. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Adjust document chunk size, semantic boundaries and overlap based on the target question types and retriever behavior, preserving source identifiers to enable citation verification. Verify: Compare at least two chunking strategies on a labeled test query set, compute evidence recall and citation precision and verify the selected chunks support complete answers.
  2. Implement: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. Verify: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.
  3. Implement: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. Verify: Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen.
  4. Implement: Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment. Verify: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

Answer & Explanations

Correct answer: B

A.

This action/test pair addresses “Tune RAG chunk boundaries and overlap using real retrieval relevance tests”. Adjust document chunk size, semantic boundaries and overlap based on the target question types and retriever behavior, preserving source identifiers to enable citation verification. Compare at least two chunking strategies on a labeled test query set, compute evidence recall and citation precision and verify the selected chunks support complete answers. The case instead calls for: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

B.

Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. Confirm it through this test: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt. A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

C.

The action is applicable: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. But its proposed test—Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen.—addresses “Evaluate RAG changes using labeled relevance and controlled A/B traffic”. This requirement is confirmed instead by: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

D.

The test would be relevant: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt. But the offered implementation—Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment.—addresses “Evaluate RAG changes using labeled relevance and controlled A/B traffic”. The action needed here is: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

Question 536

A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Which Azure AI operations action best addresses this requirement?

  1. Pin the embedding deployment version and vector dimension assumptions, then re-embed and stage a compatible search index before switching retrieval queries to a changed model.
  2. Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment.
  3. Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples.
  4. Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

Answer & Explanations

Correct answer: D

A.

Pin the embedding deployment version and vector dimension assumptions, then re-embed and stage a compatible search index before switching retrieval queries to a changed model. This addresses “Version embedding models and rebuild incompatible vector indexes safely”, where the governing principle is: Embedding models can produce different vector spaces and dimensions; mixing old document embeddings with new query embeddings can degrade semantic relevance or produce schema errors. The stated case instead calls for: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

B.

The question is set in this situation: A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment. This addresses “Evaluate RAG changes using labeled relevance and controlled A/B traffic”, where the governing principle is: Improved vector similarity scores alone may not mean better final answers; chunking, hybrid ranking and prompt context selection affect end-to-end usefulness and citation quality. The stated case instead calls for: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

C.

Before customizing a model, evaluate prompt changes and retrieval grounding for knowledge-dependent questions, reserving supported fine-tuning for an identifiable behavioral or formatting gap with curated examples. This addresses “Choose supervised fine-tuning only when retrieval and prompting cannot meet the task”, where the governing principle is: Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization. The stated case instead calls for: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

D.

The question is set in this situation: A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents. Verify the result with this test: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

Question 537

A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. The engineer proposes the following change: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. Which mechanism explains why it works?

  1. Improved vector similarity scores alone may not mean better final answers; chunking, hybrid ranking and prompt context selection affect end-to-end usefulness and citation quality.
  2. Embedding models can produce different vector spaces and dimensions; mixing old document embeddings with new query embeddings can degrade semantic relevance or produce schema errors.
  3. A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.
  4. Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization.

Answer & Explanations

Correct answer: C

A.

The question is set in this situation: A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Improved vector similarity scores alone may not mean better final answers; chunking, hybrid ranking and prompt context selection affect end-to-end usefulness and citation quality. That explains “Evaluate RAG changes using labeled relevance and controlled A/B traffic”, not the mechanism behind the described change. The relevant principle is: A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

B.

Embedding models can produce different vector spaces and dimensions; mixing old document embeddings with new query embeddings can degrade semantic relevance or produce schema errors. That explains “Version embedding models and rebuild incompatible vector indexes safely”, not the mechanism behind the described change. The relevant principle is: A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

C.

The question is set in this situation: A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents. The corresponding implementation is: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

D.

Fine-tuning modifies model behavior and does not automatically incorporate up-to-date source documents; frequently changing facts are usually better served through governed retrieval than model-weight memorization. That explains “Choose supervised fine-tuning only when retrieval and prompting cannot meet the task”, not the mechanism behind the described change. The relevant principle is: A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

Question 538

A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. After implementing this change—Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.—which test most directly validates its intended outcome?

  1. Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen.
  2. Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics.
  3. Build a parallel index using the candidate embedding model, run labeled retrieval tests and verify all vectors match the expected dimensions before switching the application.
  4. Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

Answer & Explanations

Correct answer: D

A.

The question is set in this situation: A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen. That would help test “Evaluate RAG changes using labeled relevance and controlled A/B traffic”, not the outcome required in this scenario. The relevant acceptance check is: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

B.

Compare the base model with a RAG pipeline and one small supported fine-tuned candidate on identical held-out tasks, including citation freshness and response format metrics. That would help test “Choose supervised fine-tuning only when retrieval and prompting cannot meet the task”, not the outcome required in this scenario. The relevant acceptance check is: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

C.

Build a parallel index using the candidate embedding model, run labeled retrieval tests and verify all vectors match the expected dimensions before switching the application. That would help test “Version embedding models and rebuild incompatible vector indexes safely”, not the outcome required in this scenario. The relevant acceptance check is: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

D.

The question is set in this situation: A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt. This check is decisive because a vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

Question 539

A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Which proposed shortcut most directly risks hallucinated or cross-tenant answers from inappropriate retrieval filtering and should therefore be rejected?

  1. Always pass the highest-ranked vector result into the model even when its score is weak or it belongs to another customer's document collection.
  2. Update only the query-side embedding model while retaining an incompatible document-vector index and assume vector similarity values remain meaningful.
  3. Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval.
  4. Promote a new retriever merely because its embedding average cosine similarity increased without measuring factual answer quality or citation correctness.

Answer & Explanations

Correct answer: A

A.

The question is set in this situation: A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Always pass the highest-ranked vector result into the model even when its score is weak or it belongs to another customer's document collection. The foreseeable failure is hallucinated or cross-tenant answers from inappropriate retrieval filtering. The safer response is: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

B.

Update only the query-side embedding model while retaining an incompatible document-vector index and assume vector similarity values remain meaningful. That shortcut risks retrieval failure or degraded RAG accuracy after uncoordinated embedding changes. Here the specific shortcut is: Always pass the highest-ranked vector result into the model even when its score is weak or it belongs to another customer's document collection.

C.

Fine-tune a base model on last year's tax rules and assume it automatically answers newly enacted regulations more reliably than live retrieval. That shortcut risks stale factual answers and unnecessary training cost after using fine-tuning to replace current knowledge access. Here the specific shortcut is: Always pass the highest-ranked vector result into the model even when its score is weak or it belongs to another customer's document collection.

D.

The question is set in this situation: A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Promote a new retriever merely because its embedding average cosine similarity increased without measuring factual answer quality or citation correctness. That shortcut risks user-facing answer regressions after choosing retrieval settings from a single internal similarity score. Here the specific shortcut is: Always pass the highest-ranked vector result into the model even when its score is weak or it belongs to another customer's document collection.

Question 540

A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Which combined implementation and verification plan is the best match for this workload?

  1. Implement: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. Verify: Build a parallel index using the candidate embedding model, run labeled retrieval tests and verify all vectors match the expected dimensions before switching the application.
  2. Implement: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. Verify: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.
  3. Implement: Pin the embedding deployment version and vector dimension assumptions, then re-embed and stage a compatible search index before switching retrieval queries to a changed model. Verify: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.
  4. Implement: Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment. Verify: Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen.

Answer & Explanations

Correct answer: B

A.

The action is applicable: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. But its proposed test—Build a parallel index using the candidate embedding model, run labeled retrieval tests and verify all vectors match the expected dimensions before switching the application.—addresses “Version embedding models and rebuild incompatible vector indexes safely”. This requirement is confirmed instead by: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt.

B.

The question is set in this situation: A technical support bot needs to decline questions unsupported by its documentation rather than invent an answer based on the nearest unrelated article. Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent. Confirm it through this test: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt. A vector nearest neighbor result is not inherently relevant or authorized; filters restrict scope while thresholds help reject poor matches rather than forcing an answer from unrelated documents.

C.

The test would be relevant: Test an in-scope query, an out-of-scope query and a cross-tenant query; inspect retrieved document IDs and verify weak or unauthorized results do not feed the answer prompt. But the offered implementation—Pin the embedding deployment version and vector dimension assumptions, then re-embed and stage a compatible search index before switching retrieval queries to a changed model.—addresses “Version embedding models and rebuild incompatible vector indexes safely”. The action needed here is: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

D.

This action/test pair addresses “Evaluate RAG changes using labeled relevance and controlled A/B traffic”. Assess retrieval recall, precision, answer groundedness and user-task success on fixed queries, then compare a candidate RAG configuration with a stable baseline using approved A/B assignment. Route matched user cohorts to candidate and baseline retrieval versions, inspect relevance and grounding metrics, ensure comparable latency and roll back when key safety or answer thresholds worsen. The case instead calls for: Apply evaluated similarity thresholds, top-k limits and tenant or document metadata filters to the RAG retrieval stage, and return a supported no-evidence response when authorized relevant context is absent.

Operationalizing Machine Learning and Generative AI Solutions (AI-300) Practice Tests