Scores are across five axes. The backend runtime and observability instrumentation are well-advanced for a prototype; the missing piece is deployment infrastructure and secret management discipline before any real traffic can touch it.
Three provisioned dashboards — Overview, MPP Flow, and Latency Distribution — together give strong operational and business coverage. Gaps are mostly in alerting and DynamoDB-specific metrics rather than missing panels.
Rate, error rate (4xx/5xx split), and duration (p50/p95/p99) tracked per Lambda function and in aggregate. Matches the Google SRE RED model exactly.
Every REST endpoint and every MCP tool has its own heatmap + timeseries pair in the Latency Distribution dashboard — 50+ panels covering all 14 Lambdas and 9 MCP tools.
Stripe webhook duration (p50/p95) and event-type breakdown are tracked. Critical for payment settlement correctness.
Loki logs and Tempo traces are wired in. OTel bridges console to Loki. Spans are emitted per Lambda invocation. Trace sampling is currently 100% — fine at low volume, needs review at scale.
Not tracked. Lambda cold start duration is a key operational signal for API Gateway + Lambda; it cannot be inferred from p50/p95 alone and deserves its own panel.
Not tracked. On-demand mode absorbs small spikes but can still throttle. CloudWatch ThrottledRequests metric should be surfaced in Grafana via the CloudWatch datasource when deployed.
Catalog views → 402 challenge → bids placed → payment confirmed. Conversion rates (view→challenge, bid→payment success) shown as stat panels. This directly maps to the core product metric.
Stripe (fiat) vs Tempo (crypto) split across ad bids and subscriptions. Total all-rails revenue panel exists. Revenue rate timeseries per source.
Created, canceled, revenue, invoices paid — all tracked over time and as 24h stat panels. Subscription revenue by interval (monthly/yearly) is visualised.
Average and p95 payment amounts shown. Payment events (succeeded/failed/refunded) on a single timeseries — important for spotting refund spikes.
MaxAdvertiserSpendUSDCents is a SAM parameter but spend-vs-cap utilisation is not visualised. When the cap goes live in production it should be dashboarded.
No Grafana alert rules exist on any dashboard. No SLO definitions or error-budget burn-rate panels. Dashboards are "observe and react" — not "be paged before customers notice".
Every service that runs locally has a cheap or free AWS equivalent. The entire stack maps to serverless or managed services — no EC2 instances needed.
| Local service (tmuxinator pane) | What it does | AWS replacement | Cost signal |
|---|---|---|---|
localstack — LocalStack DynamoDB |
Emulates DynamoDB at :4566 | AWS DynamoDB (same table names, already PAY_PER_REQUEST; just remove DYNAMODB_ENDPOINT) |
Free tier 25 GB + 25 WCU/RCU free forever |
sam-api — SAM local API |
14 Lambda functions behind sam local start-api on :3001 | Lambda + API Gateway HTTP API via sam deploy. SAM template is already complete and parameterised. |
Free tier 1M invocations/month free forever |
stripe — stripe listen |
Forwards Stripe webhooks to localhost | Configure Stripe dashboard to POST directly to the API Gateway /webhooks/stripe endpoint. No AWS resource needed. |
$0 |
swagger — Swagger UI (watchexec) |
Redocly/Swagger UI on :8082 for API exploration | Build static Redocly bundle → include in Hugo docs site → deploy to S3/CloudFront. Alternatively keep as a developer-only tool. | $0 (static file) |
site-up — Hugo catalog site |
Human-readable catalog at :1317 | hugo build → S3 bucket (static website) + CloudFront distribution. Hugo already produces clean static HTML. |
~$0 CloudFront free tier covers 1 TB/month + 10M requests |
docs-up — Hugo docs site |
ADRs, proposal, prototype at :1318 | Same S3 + CloudFront deployment as catalog site. Use a separate S3 prefix or a second CloudFront behaviour. A single distribution serves both. | ~$0 Shared with catalog CDN |
mcp — MCP server (Go) |
Model Context Protocol HTTP server on :8083 |
Option A (cheapest): Lambda Function URL — package Go binary, cold starts acceptable for a dev-tool. Option B: App Runner — managed, scales to zero, ~$5/month minimum. MVP shortcut: Omit from initial deployment; MCP is a developer tool, not customer-facing. |
$0–$5/mo |
obs — LGTM stack (Grafana, Prometheus, Loki, Tempo, OTelCol) |
Full observability at :3000/:9090/:3100/:3200/:4317/:4318 |
Grafana Cloud free tier — 10,000 series, 50 GB logs, 50 GB traces, 50 GB profiles per month.
Change OTEL_EXPORTER_OTLP_ENDPOINT to the Grafana Cloud OTLP endpoint.
All three existing dashboard JSON files import without changes.
|
$0 Free tier is generous for startup scale |
Ordered by severity. Critical items must be resolved before accepting real payments. Important items should be resolved within the first week of live traffic.
sam deploy workflow — the SAM template has never been deployed
There is no samconfig.toml, no sam deploy task, and no GitHub Actions deployment job. The quality-gates workflow tests code but never ships it.
MppSecretKey defaults to "local-dev-secret-change-me"
If deployed without overriding this parameter, every MPP credential accepted by production would be cryptographically unsecured.
StripeSecretKey, StripeWebhookSecret, and StripeNetworkId all default to ''. Payment processing and webhook verification will fail silently if not injected at deploy time.
MaxAdvertiserSpendUSDCents defaults to 0 (no cap)
In production, a runaway bidder or a bug in the MPP flow could charge an advertiser unbounded amounts. Set a sane ceiling (e.g. 1000000 = $10,000) before going live.
AllowedOrigins defaults to "*"
Fine for local dev but must be set to the actual production domain (e.g. https://madcatalog.com) before launch.
OtelExporterOtlpEndpoint defaults to ''
The Lambda OTel module falls back to http://localhost:4318 which doesn't exist in production. All traces, metrics, and log records are lost. Must point to Grafana Cloud OTLP endpoint.
resolve:ssm-secure:/path/to/param natively — no code change required.
PAY_PER_REQUEST mode but does not enable PITR. A bad deploy or data-corruption bug has no recovery path. PITR costs ~$0.20/GB/month — negligible at startup scale.
GF_AUTH_ANONYMOUS_ENABLED=true with OrgRole=Admin is fine for local dev but must not be deployed. Grafana Cloud handles auth automatically; self-hosted Grafana needs auth providers configured.
faas.coldstart attribute.
madcatalog.com + api.madcatalog.com + docs.madcatalog.commadcatalog.com) and docs site (docs.madcatalog.com or /docs behaviour). hugo build in CI uploads to S3.api.madcatalog.com) → 14 Lambda functions. Deployed via sam deploy. Throttling already configured in SAM template (burst 200, rate 100 rps).madcatalog-ads, madcatalog-bids, madcatalog-advertisers, madcatalog-subscriptions. Same tables as LocalStack. PITR enabled.POST api.madcatalog.com/webhooks/stripe. No forwarder. Stripe Restricted Key limited to the exact permissions required./madcatalog/prod/stripe-secret-key, /madcatalog/prod/stripe-webhook-secret, /madcatalog/prod/mpp-secret-key, /madcatalog/prod/stripe-network-id. Resolved by SAM at deploy time.Assumes low initial traffic (hundreds to low thousands of requests per day). All AWS free tier limits are perpetual unless noted. Estimates are conservative — actual costs will likely be lower.
| Service | Usage assumption | Monthly cost | Note |
|---|---|---|---|
| AWS Lambda | <100K invocations/month | $0 | 1M requests + 400K GB-seconds free forever |
| API Gateway HTTP API | <100K requests/month | $0 | $1.00/million; first 300M/month discounted |
| DynamoDB on-demand | <1M reads + <200K writes/month | $0 | 25 GB storage + 25 WCU + 25 RCU free forever |
| DynamoDB PITR | <1 GB of data | ~$0 | $0.20/GB/month; negligible at <1 GB |
| S3 (static sites) | <500 MB storage | $0 | 5 GB storage + 20K PUTs + 200K GETs free (12 months) |
| CloudFront | <10 GB transfer/month | $0 | 1 TB/month + 10M requests free forever |
| ACM (TLS certs) | 1 wildcard cert | $0 | Free when used with CloudFront or API Gateway |
| SSM Parameter Store | ~6 standard SecureString params | $0 | Standard parameters are free; Advanced is $0.05/param/month |
| Route 53 | 1 hosted zone, ~10K queries/month | ~$0.50 | $0.50/zone/month + $0.40/million queries |
| Grafana Cloud | Free tier | $0 | 10K series, 50 GB logs, 50 GB traces per month |
| GitHub Actions (CI/CD) | <2,000 minutes/month | $0 | 2,000 free minutes/month on public repos; 500 MB artifacts storage |
| Total estimated monthly cost | ~$0.50 | Dominated by Route 53 hosted zone fee | |
Ordered from unblocked-now to needs-runtime-feedback. Each step is independently deployable. The whole sequence can be completed in a single day.
STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET, MPP_SECRET_KEY,
STRIPE_NETWORK_ID as SecureString under /madcatalog/prod/.
Update SAM parameters to use resolve:ssm-secure:/madcatalog/prod/stripe-secret-key syntax.
IAM role for the deployer needs ssm:GetParameter on those paths.
samconfig.toml with production defaults
Set AllowedOrigins=https://madcatalog.com, MaxAdvertiserSpendUSDCents=1000000,
OtelExporterOtlpEndpoint=https://otlp-gateway-prod-us-east-0.grafana.net/otlp,
TempoTestnet=false for production. Keep DynamoDBEndpoint empty (uses real AWS).
sam build && sam deploy --guided
The SAM template is complete and already passes sam validate --lint.
First deploy uses --guided to populate samconfig.toml.
Subsequent deploys use sam deploy (no flags). Note the API Gateway URL from Outputs.
hugo build outputs static HTML. Create an S3 bucket, CloudFront distribution,
ACM cert, and Route 53 records. Use aws s3 sync public/ s3://madcatalog-site.
CloudFront cache invalidation on deploy: aws cloudfront create-invalidation --paths "/*".
https://api.madcatalog.com/webhooks/stripe as a webhook endpoint.
Select events: payment_intent.succeeded, payment_intent.payment_failed,
customer.subscription.*, invoice.*. Copy the signing secret to SSM.
OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS
(Bearer token) in samconfig.toml and redeploy. Import the three dashboard JSON files.
Add Prometheus, Loki, and Tempo data sources from the Grafana Cloud connection wizard.
aws dynamodb update-continuous-backups --table-name madcatalog-ads --point-in-time-recovery-specification PointInTimeRecoveryEnabled=true (repeat for each table).
Alternatively add PointInTimeRecoverySpecification to each table in the SAM template and redeploy.
.github/workflows/deploy.yml triggered on push to main.
Steps: checkout → setup-node → setup-sam → sam build → sam deploy
(using OIDC role, no long-lived AWS keys) → hugo build → aws s3 sync
→ CloudFront invalidation.
The MCP server is a developer tool. It adds no value to end users or advertisers. Skip it for the initial launch; deploy when you need Claude to operate against production data.
Add WAF after you see real traffic patterns. At launch, API Gateway throttling (already configured) is sufficient. WAF costs ~$5/month base plus $0.60/million requests.
Set limits once you have a baseline. Premature limits cause unnecessary 429s during legitimate traffic spikes. The DynamoDB on-demand mode absorbs reasonable burst natively.
Leave TempoTestnet=true until Tempo mainnet integration is validated. Real money should not flow through an untested crypto payment path. Stripe fiat path is production-ready.
Generated 2026-06-09 · madcatalog production readiness audit