Operational Efficiency and Optimization for GenAI Applications

205. A team enables prompt caching but sees no cost reduction, and the prompt's shared instruction block sits after the variable user content. Which change is required?

1. A Reduce the length of the static instruction block so it fits the cache.
2. B Increase the cache retention period so entries survive longer between requests.
3. C Move the static content to the start of the prompt, because only a prefix that is identical across requests can be reused.
4. D Raise the number of concurrent requests so the cache is exercised more often.

206. An assistant answers many semantically similar questions each day. Which solution most reduces inference cost?

1. A Increase the maximum output tokens so answers are more complete.
2. B Route every request to a larger model for better answers.
3. C Add a semantic cache that returns a stored response when an incoming query is sufficiently similar to a previous one.
4. D Raise the temperature so responses vary between similar questions.

207. Context supplied to the model has grown as retrieval was tuned, and cost has risen with it. Which approach is appropriate?

1. A Measure answer quality against the amount of context supplied and prune to the smallest amount that holds quality.
2. B Reduce the maximum output tokens so responses are shorter.
3. C Switch to a model with a larger context window.
4. D Raise the retrieval similarity threshold until fewer passages qualify.

208. A time-sensitive feature must minimise the delay before the user sees the first word of a response. Which combination of steps meets these requirements? (Select TWO.)

1. A Increase the maximum output tokens so the response is more complete.
2. B Enable response streaming so the first tokens are delivered as they are generated.
3. C Raise the temperature so the model begins generating sooner.
4. D Select a latency-optimized model configuration appropriate to the task.
5. E Increase the number of retrieved passages so the model has more context.

209. Retrieval latency dominates the end-to-end response time of a RAG application. Which approach is appropriate?

1. A Increase the application's request timeout so slow retrievals complete.
2. B Increase the number of candidates retrieved so fewer follow-up queries are needed.
3. C Switch to a larger generation model so the response completes sooner.
4. D Profile the retrieval path and optimize the index configuration and query preprocessing against measured latency and recall.

210. A workflow makes several independent model calls that currently run one after another. Which solution meets these requirements?

1. A Combine all the calls into one prompt so only a single invocation is needed.
2. B Increase each call's maximum output tokens so fewer calls are required.
3. C Increase the workflow's timeout so the sequential calls have time to complete.
4. D Issue the independent calls in parallel and combine the results, keeping any dependent calls in sequence.

211. A team must detect a sudden change in token consumption that indicates a prompt or retrieval regression. Which solution meets these requirements?

1. A Track total monthly spend and review it at the end of each month.
2. B Track the number of invocations per day and alarm on a static threshold.
3. C Track the application's error rate and alarm when it rises.
4. D Track tokens per request as a metric and alarm on an anomaly detection band so a burst or drift is detected.

212. An investigation requires the full prompt and completion for a specific past interaction. Which solution meets these requirements?

1. A Enable AWS X-Ray so the invocation is traced across service boundaries.
2. B Enable AWS CloudTrail so the invocation API calls are recorded.
3. C Record the request identifier returned by each invocation in the application log.
4. D Enable model invocation logging so requests and responses are recorded with the model and configuration identifiers.

213. A vector store's retrieval quality must be monitored so degradation is detected before users report poor answers. Which solution meets these requirements?

1. A Track the number of documents ingested each day.
2. B Track the vector store's storage size and alarm when it grows unexpectedly.
3. C Run a recurring set of known questions against the index and track retrieval hit rate over time.
4. D Track the average query latency of the vector store.

214. An application's context grows through a conversation until cost per request becomes unacceptable. Which approach is appropriate?

1. A Truncate the oldest turns without regard to their content.
2. B Switch to a model with a larger context window.
3. C Prune the context by summarizing older turns while retaining facts the conversation depends on.
4. D Start a new conversation once the cost threshold is reached.

215. An application's provisioned throughput is underused outside business hours. Which approach is appropriate?

1. A Batch all out-of-hours requests until business hours resume.
2. B Retain the provisioned capacity continuously for predictable performance.
3. C Match committed capacity to the sustained period and serve out-of-hours demand on demand.
4. D Increase the provisioned capacity so utilization percentages appear lower.

216. A caching strategy must avoid returning a stale answer when the underlying documents change. Which approach is appropriate?

1. A Invalidate the cached entries whose source documents changed, keying the cache so that mapping is possible.
2. B Disable caching for any question that might relate to changeable content.
3. C Refresh the entire cache on a nightly schedule.
4. D Set a short expiry on every cache entry regardless of source.

217. An application must reduce perceived latency for a response that genuinely takes several seconds to generate. Which approach is appropriate?

1. A Increase the model's maximum output tokens.
2. B Increase the application's request timeout.
3. C Reduce the number of retrieved passages.
4. D Stream the response so the user sees output while generation continues.

218. A team must decide between a lower temperature and a higher one for a factual question-answering application. Which consideration applies?

1. A A lower temperature produces more deterministic output, which suits factual answers where variation is undesirable.
2. B Temperature should be set to the model's maximum for all applications.
3. C Temperature has no effect on output when retrieval is used.
4. D A higher temperature improves factual accuracy by exploring more possibilities.

219. A retrieval step is fast in isolation but slow within the application's request path. Which cause should be investigated first?

1. A Whether the generation model is slower than expected.
2. B Whether the index contains too many documents.
3. C Whether connection setup, network path, or serialization between the application and the index is adding overhead.
4. D Whether the embedding model has changed.

220. An application must handle a burst of concurrent model invocations without exceeding its throughput allocation. Which approach is appropriate?

1. A Reduce the number of tokens in each request.
2. B Invoke the model for every request as it arrives and retry on throttling.
3. C Limit concurrency at the application layer and queue the excess, so the allocation is used fully but not exceeded.
4. D Increase the request timeout so throttled requests eventually succeed.

221. A dashboard must show whether a GenAI application is delivering business value rather than only operating correctly. Which metric belongs on it?

1. A The proportion of requests that returned without error.
2. B The application's average response latency.
3. C The number of model invocations per hour.
4. D The rate at which users complete the task the application exists to support.

222. A team must detect when a model update changes an application's behaviour on inputs that previously worked. Which approach is appropriate?

1. A Run a golden dataset through the application after each update and compare outputs against the recorded baseline.
2. B Monitor the application's error rate after each update.
3. C Review user feedback in the week following each update.
4. D Compare the new model's benchmark scores against the previous version's.

223. Vector store operational health must be monitored so degradation is detected before it affects answers. Which combination of signals is appropriate? (Select TWO.)

1. A The number of ingestion jobs run each day.
2. B Retrieval hit rate against a recurring set of known questions.
3. C The storage cost of the index each month.
4. D Query latency at the percentile the application targets.
5. E The total number of vectors stored.

224. An agent's tool usage must be monitored to establish whether it is calling tools appropriately. Which signal is most informative?

1. A The total number of tool calls per day.
2. B The average latency of each tool.
3. C The number of tools registered with the agent.
4. D The distribution of tool calls per task compared against an established baseline.

225. A request's token count must be estimated before invocation to enforce a per-request budget. Which approach is appropriate?

1. A Skip estimation and check the bill later.
2. B Count words and assume one token each.
3. C Use the model's tokenizer or a token counting API to measure the prompt, and reject or trim requests exceeding the budget.
4. D Estimate tokens as characters divided by four.

226. A tiered model strategy must be validated before rollout. Which validation is appropriate?

1. A Compare the models' list prices.
2. B Roll out and monitor complaints.
3. C Evaluate the cheaper tier on the query class it will handle against the quality bar, measuring both quality and cost per query.
4. D Assume simpler queries need less capability.

227. Batching must be applied to a workload of many independent short requests to improve throughput. Which consideration applies?

1. A Batching always reduces latency.
2. B Batching requires requests to be related.
3. C Batching raises throughput at the cost of per-request latency, so it suits workloads that tolerate delay rather than interactive ones.
4. D Batching has no effect on cost.

228. Predictable queries must be answered faster than the model can generate them. Which approach is appropriate?

1. A Use a faster network.
2. B Reduce the response length for all queries.
3. C Pre-compute responses for the predictable queries and serve them from a cache, invoking the model only for novel ones.
4. D Increase the model's throughput.

229. A hybrid search implementation must weight lexical and semantic results appropriately. Which approach is appropriate?

1. A Tune the combination weights against a labelled query set, since the optimal balance depends on the corpus and query style.
2. B Weight them equally always.
3. C Use lexical results only.
4. D Use semantic results only.

230. Top-p and temperature parameters must be selected for a creative writing feature. Which configuration is appropriate?

1. A A higher temperature and top-p to allow varied, less predictable output, validated against quality expectations.
2. B Top-p of zero.
3. C The defaults without evaluation.
4. D Temperature zero for determinism.

231. A GenAI workload's capacity must be planned in terms of token processing rather than request count. Which rationale applies?

1. A Requests vary enormously in token count, so token throughput is the resource actually consumed and the correct planning unit.
2. B Tokens are irrelevant to capacity.
3. C Request count is always proportional to tokens.
4. D Capacity should be planned per user.

232. API call profiling reveals that most latency is in a post-processing step rather than the model. Which action is appropriate?

1. A Reduce the prompt length.
2. B Increase provisioned throughput.
3. C Switch to a faster model.
4. D Optimize the post-processing step, since the model is not the bottleneck.

233. A GenAI application's observability must cover business impact as well as operational health. Which metric belongs in the business view?

1. A Conversion or resolution rate for the workflow the feature supports.
2. B Error rate.
3. C Token count per request.
4. D Endpoint latency.

234. Response drift must be detected in a deployed GenAI application. Which approach is appropriate?

1. A Periodically run a fixed input set and compare outputs against the recorded baseline for semantic change.
2. B Monitor error rate.
3. C Monitor request volume.
4. D Monitor latency.

235. Forensic traceability must allow reconstruction of exactly what a GenAI system did for a given request. Which data must be logged?

1. A The request timestamp only.
2. B The user identifier only.
3. C The response only.
4. D The request identifier, prompt version, model identifier, retrieved sources, tool calls, and response, correlated together.

236. Vector store index optimization must be automated to maintain performance as the corpus grows. Which approach is appropriate?

1. A Rebuild the index on every document change.
2. B Schedule index maintenance routines and monitor query latency and recall, triggering reoptimization when they degrade.
3. C Never optimize after initial build.
4. D Rebuild the index manually when users complain.

237. A hallucination must be detected in production without a reference answer. Which technique applies?

1. A Compare the response against retrieved context for grounding, and use output diffing across repeated runs to detect inconsistency.
2. B Check the response for grammar errors.
3. C Ask the model if it hallucinated.
4. D Check the response length.

238. A GenAI application's cost must be attributed to the features that drive it. Which approach is appropriate?

1. A Tag each invocation with the feature that made it and aggregate token spend by feature.
2. B Divide the total cost by the number of features.
3. C Record the total monthly cost.
4. D Estimate each feature's share from its usage frequency.

239. A GenAI application's cost must be forecast before a traffic increase. Which approach is appropriate?

1. A Scale the current bill by the expected traffic multiple.
2. B Assume cost grows with the number of users.
3. C Use the model provider's list price per request.
4. D Model the expected invocations and average tokens per invocation, and price them at the current rates.

240. A GenAI application must reduce cost for requests that repeat a long static preamble. Which approach is appropriate?

1. A Enable prompt caching with the static content placed at the start of the prompt so the prefix can be reused.
2. B Shorten the preamble.
3. C Switch to a cheaper model.
4. D Reduce the number of requests.

241. A GenAI application's end-to-end latency must be reduced where retrieval dominates. Which approach is appropriate?

1. A Optimize the retrieval stage specifically, since that is where the time is spent.
2. B Reduce the maximum output tokens.
3. C Switch to a faster generation model.
4. D Increase the application's concurrency.

242. A GenAI application must handle a burst without degrading latency for existing requests. Which approach is appropriate?

1. A Reduce the response length for all requests.
2. B Process every request immediately.
3. C Bound concurrency and queue the excess, so in-flight requests are not starved.
4. D Increase the timeout for all requests.

243. A GenAI application's throughput must be increased without raising cost per request. Which approach is appropriate?

1. A Increase the number of application instances.
2. B Increase the model's provisioned throughput.
3. C Batch compatible requests where latency permits, so throughput rises without additional per-request cost.
4. D Switch to a larger model.

244. A GenAI application must reduce the time to first token for interactive use. Which approach is appropriate?

1. A Increase the model's maximum output tokens.
2. B Stream the response and minimize the work performed before generation begins.
3. C Reduce the total response length.
4. D Increase the retrieval result count.

245. A GenAI application's performance regression must be detected before users report it. Which approach is appropriate?

1. A Run synthetic requests continuously and alarm on latency percentiles exceeding the target.
2. B Wait for user reports.
3. C Review the latency dashboard each morning.
4. D Compare monthly latency averages.

246. A GenAI application must maintain quality while reducing its model cost. Which approach is appropriate?

1. A Route all requests to the cheapest model.
2. B Alternate between models randomly.
3. C Route each request to the least capable model that meets the quality bar for that request class, validated by evaluation.
4. D Route all requests to the most capable model.

247. A GenAI application's monitoring must distinguish a model problem from an application problem. Which approach is appropriate?

1. A Monitor the model provider's status page.
2. B Monitor the total error rate.
3. C Monitor the application's request volume.
4. D Instrument each stage separately so the failing stage is identified from the telemetry.

248. A GenAI application's quality must be monitored continuously in production. Which approach is appropriate?

1. A Sample production interactions, score them against the quality dimensions, and track the scores over time.
2. B Review user complaints.
3. C Monitor the application's error rate.
4. D Rely on the pre-deployment evaluation.

249. A GenAI application's guardrail effectiveness must be monitored. Which approach is appropriate?

1. A Track the number of requests blocked.
2. B Track both the block rate and the rate of blocks later judged incorrect, so both failure directions are visible.
3. C Track the number of requests allowed.
4. D Track the guardrail's latency.

250. A GenAI application's token consumption must be monitored for anomalies. Which approach is appropriate?

1. A Publish tokens per request as a metric and alarm on deviation from the established pattern.
2. B Count the number of requests per day.
3. C Alarm when the total daily tokens exceed a fixed threshold.
4. D Review the monthly bill.

251. A GenAI agent's behaviour must be monitored for unexpected tool usage. Which approach is appropriate?

1. A Record the total tool invocations.
2. B Limit the agent to a single tool.
3. C Review the agent's logs weekly.
4. D Record each tool invocation with its task context and alarm on patterns that deviate from the established baseline.

252. A GenAI application's observability data must support investigating a specific user complaint. Which approach is appropriate?

1. A Retain records for interactions that produced errors.
2. B Retain the most recent hour of interactions.
3. C Retain per-interaction records keyed by an identifier the user can supply, within the applicable retention policy.
4. D Retain aggregate metrics only.