Morning, boss.
Read 1,204 records from six sources between 05:12 and 05:58. One incident overnight, mitigated. Two things need your call.
Compiled 06:00 · Mon 21 Sep
Since 06:00Watching six sources.
EU order writes blocked for 14 minutes
40 checkouts failed · orders_eu · 02:41–02:55
02:41p99 write latency to 30 s · DatadogDatadog monitor “orders_eu write p99” fired: 30.2 s against a 2 s threshold, eu-west-1.
02:44Marcus paged, acknowledged in 40 s · PagerDutyPagerDuty incident 2291 paged Marcus Lind, primary on call. Acknowledged at 02:44:40.
02:54Index build on orders_eu cancelled · Slack #incident-2291Slack #incident-2291, Marcus: “Cancelling the orders_eu index build, it’s holding the write lock.”
02:55Writes recovered, queue drained by 03:02 · DatadogDatadog: write p99 back under 200 ms at 02:55; the queue drained by 03:02.
03:12ORD-1187 opened for the 40 retries · LinearLinear ORD-1187, opened by Marcus: retry the 40 failed checkouts.
Cause: index build took the write lock during EU morning traffic.
Billing migration 0043 runs tonight against a 22 M-row tableTonight
Same index-build pattern as 2291, the case behind Marcus’s off-peak proposal this morning. Tomas can move it to 23:00 UTC with the index built first.
TBTomas Berg, billing · you decide
On-call rota lists two people who have leftThrough Fri
Sam and Ines are still in the platform rota through Friday. Marcus is the only live responder this week.
MLMarcus Lind, SRE · you decide
Deploys / day
0.0
▲ 8% vs Aug
Lead time
0.0 d
▼ 0.3 d
Change failure
0.0%
▼ 0.4 pt
Time to restore
0 min
▲ 4 min · 2291
DeploysWeekend7-day average
Production deploys per day, 30 days; weekends shaded.
0
records read tonight · 05:12–05:58
LLinear312
GGitHub488
DDatadog201
PPagerDuty3
SSlack176
CCalendar24
0tracked · 9 kept · 3 open · 2 slipped
PNSDK-418 fix to betaPriya · moved to Thu 24Moved
ERAtlas scope signedElena · Tue 15Kept
MLRota updatedMarcus · due Fri 18Slipped
ZBoard notes to youZora · Thu 24, draftedOpen
09:30Incident review, 22914 parts · 8 recordsReady
11:00Board prepEvery figure sourcedDrafted
15:00Atlas checkpointScope decision attachedReady
17:00Commitments check14 against what shippedZora
Atlas ships EU-only firstElena · Tue 15 · 3 records
Index builds move off-peakMarcus, proposed · Mon 21, 02:58 · pending your call
SDK 2.0 release held to ThuPriya · Tue 15 · 2 records
Rollback owner named per serviceYou · Fri 18 · draftDraft
Each with who decided and the evidence behind it.
| Project | Owner | Status | Next milestone | Why | Records |
|---|---|---|---|---|---|
| Atlas · EU launch | ER Elena Ruiz | On track | Oct 14 | Scope cut to EU-only on Tue; dependency on billing 0043 cleared once it runs. | 41 |
| SDK 2.0 · mobile | PN Priya Nair | At risk | Sep 24 | Release held on SDK-418; fix moved to Thu, partner demo is Fri. | 27 |
| Billing migration | TB Tomas Berg | Held | Tonight | Step 0043 repeats the 2291 pattern; waiting on your call. | 18 |
| Data platform | ML Marcus Lind | On track | Nov 2 | Off-peak index policy written Mon 02:58; rota gap is the open risk. | 33 |
DKDana KimCTO · you
ERElena RuizEngineering manager, Atlas · 41 records this week
PNPriya NairEngineering manager, mobile · owns SDK-418
TBTomas BergBilling lead · owns migration 0043
MLMarcus LindSRE · primary on call this weekOn call
SOSam OkaforLeft 12 Sep · still in the platform rotaOn rota
Advise, and hold for your callEvery call since 1 Sep · 31 held, 31 decided by youActive
Open a ticket when a written condition is metShadow · would have fired 4 times this monthShadow
Page the on-call on the published ruleShadow · would have fired once, 2291 at 02:42Shadow
Hold a deploy one click from the on-callNot grantedOff
Each rule runs in shadow first and shows up in the brief. Demotion is one message.
The briefCompiled 06:00 local, Monday to FridaySources are read from 05:12 to 05:58, and the brief is compiled at 06:00.
Delivered toDana Kim, by email and Slack DMAnnounced at 06:00 by email and Slack DM, each with the brief’s sub-line and a link straight to it.
SourcesSix connected · read 05:12–05:58
Urgent callsA Slack DM when a deadline is inside two hoursA held call whose deadline falls inside two hours gets its own message.