Engineering Decision Skills: What I Want AI to Do in Reviews, Planning, and Retrospectives

Conclusion: hold production rollout; proceed with integration testing in isolation. A 5% rollout limits the number of orders exposed, but the one-coupon-per-order rule must still hold. The material does not explain how that rule survives timeouts, retries, and rollback.

Blocker one: requests with unknown outcomes enter ordinary retries. Status: open.

The evidence is that a timeout may occur after a coupon has been issued, and failures retry up to three times. Suppose the coupon service issues a coupon but its response is lost. The worker never records completion, and the next call may issue another coupon. This is a gap between an external side effect and local state. An “already completed” field in the local order table alone does not close it.

Confirm whether the coupon service supports idempotent requests scoped to an order. If it does, both synchronous and asynchronous paths should use the same business identifier. If it does not, define an “outcome unknown” state, reconciliation through queries, and a manual resolution path. Do not treat every timeout as safe to retry. The lookup API can help confirm outcomes, but querying before issuing does not by itself prevent duplicates caused by concurrency or delayed query results.

Blocker two: rollback does not account for queued work and requests in flight. Status: open.

The evidence is “turn off workers and return to synchronous issuance.” Stopping consumers does not withdraw requests already sent to the coupon service. When queued tasks are consumed later, they may also process orders already handled synchronously.

The rollback design must address four things: where new work goes during the switch, how in-flight outcomes are confirmed, how queued tasks are reconciled and handled, and how both paths share deduplication rules. The implementation may choose to pause, drain, or isolate old work. The acceptance condition is that switching paths cannot issue a duplicate coupon or leave an order permanently unfulfilled.

Important open question: how does an order complete after its task reaches the dead-letter queue?

Keep the proposed backlog and failure-count monitoring. Also confirm who takes failed tasks, how issuance is recovered, and how the business can check the final state. Queue durability and what counts as a successful write are not specified either. These are missing review inputs; their absence from the proposal does not establish that the implementation loses messages.

Suggested exercises before launch:

• Issue a coupon, lose the response, and consume the task again. There must still be only one coupon.
• Exit after the coupon service succeeds but before the worker writes local completion. Restarting must not issue a duplicate.
• Enqueue the same order twice, or let it reach both old and new paths. The business rule must hold.
• Roll back with queued and in-flight work, then reconcile each order's final state. Checking that the switch changed is insufficient.

Once these behaviors are specified and exercise records are available, review whether to enable 5% of traffic. The proposal owner and production approver have not been identified; confirm them before rollout.

Make project-level permissions the primary candidate, exports a conditional second item, and defer committing to the full order-service split. First run a brief scope and capacity check to establish whether two items can be promised. All effort figures are supplied estimates, not validated estimates; 12 person-weeks is the stated available capacity.

Confirm two things first. The product lead should establish exactly which permission capabilities are needed in week six and whether they are a purchasing prerequisite. Engineering should separately map the key person's design, review, and integration time. A total effort budget can still produce two projects waiting on one person if capacity by capability is ignored.

Bound permissions by the customer's confirmed requirements. Suggested acceptance checks include permitted role actions, rejected access across projects, and inspectable authorization-change records; validate these against the actual requirements. For exports, first inspect which tasks recur within the roughly 20 weekly requests, then define the first version's scope. Without time per request, the plan cannot claim a number of person-days saved per month.

Bring the direction to discussion without presenting both full scopes as committed. If scope review still yields 11 person-weeks, make the remaining one person-week visible to the product lead. If a dependency or rework exceeds that allowance, reduce export scope or renegotiate acceptance timing first. Do not silently budget overtime.

For the service split, investigate the two deployment conflicts. Compare a split with changes to deployment windows, compatibility practices, or the release process. Reopen the investment decision if evidence shows coupling repeatedly blocking deployment and smaller changes cannot resolve it. Investigation consumes capacity too: estimate it, then authorize it from the available allowance, rather than attaching it to the plan for free.

Revisit the combination if the customer cancels or pushes back acceptance of the permissions work, its scope exceeds the estimate substantially, the key engineer is unavailable when needed, or new deployment evidence changes the priority. Those changes should reopen the plan rather than leave the original schedule in place.

The most useful change is how integration readiness is judged. The material supports this sequence: APIs were reported complete in week two; the first complete integration run happened in week five; amount units and empty-order semantics surfaced then. A green weekly status did not show that end-to-end acceptance had not taken place.

This supports a judgment about the working process: development completion stood in for evidence that delivery risk was under control, without timely proof that both sides understood the interface in the same way. It does not show that this caused the entire two-week delay, nor that no contract checks happened earlier. The material only records when the first complete integration run happened.

The new filter field may have added work, and obtaining production test access may have caused waiting. Their contributions to the critical path cannot currently be calculated. Preserve both the product team's claim that the field was outside the main flow and the developers' claim about changing requirements; neither is established fact. Attributing the full delay would require change records, access-availability timing, and integration-blocker records. Preparing the next project need not wait for all of those.

Try two changes on the next similar project.

First, agree on minimal integration examples at kickoff: one normal order, one with a fractional monetary amount, and one empty order. Keep inputs, units, expected responses, and error semantics together for both teams to confirm. Aim to run the smallest complete path in week two. If production test access is unavailable, use mutually confirmed examples to check the contract while keeping real-environment validation explicitly incomplete. Passing examples does not replace acceptance in the real environment.

Second, report “our development complete” separately from “end-to-end validation complete.” To mark integration ready, attach an accessible environment, test-access status, and the latest example results. If a condition is missing, name the dependency and who must confirm its availability. Both teams need to agree on those owners at kickoff; a retrospective cannot accept commitments on their behalf.

Keep resolving differences found during integration and carrying the work through acceptance. Stop marking a cross-team milestone green solely because APIs are complete. Do not add a comprehensive new approval process for now.

Review the experiment in week two of the next project by checking whether the minimal path has run. At acceptance, check whether differences in units, states, or error semantics still emerge mainly at the end. If the main obstacle instead becomes prolonged lack of external access, adjust dependency preparation and escalation. Do not attribute every delay to late integration.

Keep existing weekend human coverage today; do not reduce service on the basis of this increase yet. What is established is that the displayed value changed from 72% to 91%. Because the denominator and transfer entry point changed together, those values do not yet establish improved task resolution by the bot.

The current metric measures conversations without a transfer. Removing coverage requires knowing whether users can complete their tasks without human help. Abandonment, failure to find the entry point, and seeking help through another channel can all count as success under the current definition.

At least three explanations remain unresolved: the bot handles more problems; the harder-to-find entry point reduces transfers; and excluding welcome-only exits changes the denominator. These may occur together. A few reports justify investigating the entry point, but cannot establish the share affected.

First make the comparison interpretable, then run a limited service trial. If original events remain available, recalculate both periods under one denominator rule, segmented by issue type and weekday versus weekend. If old events cannot be recovered, state that the increase cannot be reconstructed and begin a new baseline. Even if the recalculated rate rises, the effect of the changed entry point still needs investigation.

Start with a frequent task whose completion can be observed, such as checking order status. Define the primary metric as the proportion of conversations attempting that task which return the needed order status and receive user confirmation of resolution. Mark unconfirmed outcomes as unknown. Both system state and user confirmation have limitations, so sample-check them during the trial; silence is not success.

The minimum records are task attempts, returned results, user confirmation, and requests for human help with their outcomes, linked to the same conversation. Begin with one data-quality check: replay a sample of conversations and compare attempts and final states with the event records. A full new dashboard is unnecessary at this stage.

Cover a complete working week and weekend initially, while keeping a clear, usable route to a person. Whether that produces enough evidence depends on task volume and segment distribution; seven days alone guarantees nothing. Track task completion, with unmet requests for human help and repeat requests over a short interval as guardrails. The support lead must confirm the interval and acceptable limits; the current material cannot supply thresholds.

If a conversation shows a task remaining incomplete and human help being blocked, address that failure path and pause expansion of self-service. Discuss reducing coverage only when evidence supports a specific task and time period. Success for one task does not justify removing all weekend coverage.

Release-platform pilot progress and a request for two weeks of engineering support

We recommend assigning an engineer familiar with the old release scripts to the team next week for an estimated two weeks, to adapt and validate rollback before reconsidering expansion. The assignment awaits approval. If support is unavailable this cycle, the team will maintain the current six-service pilot and will not commit to an onboarding date for the remaining two services.

Over four weeks, six of eight planned services have joined the pilot and completed 17 releases. Manual records for the same six services show median manual operation time decreasing from 18 minutes before the pilot to 6 minutes afterward, providing initial evidence of less time spent on operating steps. This excludes release waiting time, and no failure-rate data has been collected. It does not yet establish improved overall release efficiency or reliability. One of the 17 releases still required manual rollback.

Rollback capability is the main obstacle to expansion. The two remaining services use old release scripts without integrated automated rollback. The existing review requires that gap to be closed first. Business teams are frustrated with onboarding speed, but adding services before meeting that condition would exceed the scope the review currently permits.

My responsibilities are integration standards and cross-team coordination; another group built the execution engine. The next stage needs an engineer familiar with the old scripts for adaptation and validation. Two weeks is the current estimate, to be refined once staffing and script differences are confirmed. Suggested acceptance should cover rollback paths for the two pending services and revisit the earlier manual rollback, so that expansion is not based solely on completed onboarding.

The director's decision is whether this staffing support can be provided. If approved, the team will confirm the engineer, scope, and next review date. If it cannot be provided, the team will continue the limited pilot under the existing conclusion and explain to business teams why expansion is paused and what would allow it to resume.