IoT Weather Station: What Went Wrong (and How I Fixed It)
This IoT Weather Station is a real-world telemetry pipeline built around a physical Ambient WS-2902D weather station, Raspberry Pi 3, SDR receiver, AWS IoT Core, Lambda, Amazon Timestream, DynamoDB, API Gateway, S3, CloudFront, and WAF. Below are the main engineering challenges I worked through while turning raw sensor data into a stable, cost-aware, production-style AWS dashboard.
Telemetry Source Validation
One of the most important lessons from this project was that cloud architecture is only as trustworthy as the telemetry entering it. My Ambient WS-2902D weather station uses a WH65B sensor array, but the Raspberry Pi receiver was originally decoding the signal as Fineoffset-WH24. The pipeline still produced data, but rainfall totals slowly drifted from the Ambient console.
The correction was made at the edge. By running rtl_433 with -s 250k, the sensor was correctly decoded as Fineoffset-WH65B. Once the source device was identified correctly, rainfall values aligned with the physical weather station and the downstream AWS pipeline became much more reliable.
This reinforced a practical engineering lesson: before blaming the cloud services, API, database, or dashboard, validate the physical source, decoder settings, units, and payload shape. Bad telemetry at the edge becomes bad architecture everywhere else.
IoT Transport & Auth
Using X.509 certificates for MQTT over TLS proved brittle on the Raspberry Pi and added operational overhead for certificate management and rotation.
I switched to MQTT over WebSockets with SigV4 authentication, which simplified device authentication, reduced maintenance struggles, and made the outbound network path more reliable across typical home and consumer networks.
Lambda Ingest vs. Read Paths
I separated ingestion and read responsibilities into two Lambda functions instead of trying to make one function handle both paths.
wx_ingest handles the write path only: AWS IoT Core Rule → Lambda → Amazon Timestream. Its job is intentionally narrow: validate the normalized telemetry payload, write a single multi-measure record to the weather_MQTT_data_mm Timestream table, and emit freshness signals for monitoring.
get_weather_data handles the read path only: API Gateway → Lambda → Timestream → JSON response. It is responsible for query-window selection, safe row limits, rollup lookups, and returning live or aggregated weather data to the dashboard.
Splitting those responsibilities made the system easier to reason about, easier to secure with least-privilege IAM, and easier to troubleshoot. Write-path failures, query problems, and dashboard data issues are now isolated to the part of the system actually responsible for them.
IAM
Timestream calls initially failed with AccessDeniedException, including an
explicit deny caused by a temporary pause policy that was still attached to the role.
The fix required cleaning up IAM attachments in Terraform and reapplying a true
least-privilege model. Each execution path was granted only the Timestream actions it
actually needed: DescribeEndpoints, Query, and
WriteRecords.
API Gateway Confusion
Early experimentation left me with multiple overlapping HTTP APIs, some of which had no active routes or stages. This caused confusion when troubleshooting frontend errors and verifying deployments.
I standardized on a single HTTP API (v2) with a defined POST /weather
route, explicit CORS configuration for https://c-comp.net, and a verified stage URL
wired into the frontend. That eliminated ambiguity and made deployments and debugging predictable.
Caching & Stale Data
Charts and tables for longer time ranges originally depended too heavily on live Timestream queries, which increased latency and raised the risk of unnecessary query cost. I moved longer-range views to a scheduled precompute path: EventBridge → Lambda (wx_precompute) → S3 JSON → CloudFront.
Short dashboard windows remain live through API Gateway and get_weather_data, while longer windows are served from precomputed JSON behind CloudFront. This keeps the dashboard responsive, reduces repeated Timestream reads, and makes the cost profile easier to control.
Terraform State & Imports
Because CloudFront was originally created manually, I imported it into Terraform and mirrored the existing behaviors (static site, /data/*, JSON, etc.). Keeping the IaC aligned with actual AWS resources prevented “drift” surprises.
Cost Control
My Timestream costs climbed very quickly during my live-query testing. The redesign keeps short windows (1h/24h) as on-demand queries (cheap), and pushes 7d/30d to precomputed JSON in S3 (cheapest). CloudFront shields S3 and prevents repeated API hits. Logs are sampled and metric filters drive alerts.
Telemetry Accuracy & Rainfall YTD Debugging
Rainfall YTD State Management
Rainfall required special handling because the weather station reports rainfall as a cumulative counter, not as a clean daily or yearly total ready for display. That meant the dashboard could not simply trust one field and call it “YTD rainfall.” The backend needed a stable way to calculate yearly rainfall from the station's cumulative telemetry model.
The final design uses Amazon Timestream for corrected rainfall telemetry and DynamoDB for durable baseline state. The weather-station-state table stores the yearly baseline using pk = WeatherStation and sk = rain_baseline#2026, with the baseline value stored as baseline_rain_in.
Rain YTD is now calculated as:
latest corrected rain_in from Timestream - DynamoDB baseline_rain_in
This moved the rainfall calculation out of fragile frontend logic and into a backend model that is easier to validate, explain, and maintain. It also mirrors a real telemetry pattern: cumulative counters often need a trusted baseline before they can produce accurate period-based totals.
The bigger lesson was that telemetry validation has to happen end to end. I had to confirm the sensor identity, decoder settings, payload units, Lambda normalization, Timestream records, DynamoDB baseline state, API response, and final dashboard display. Once the source telemetry was corrected and the baseline model was in place, the dashboard Rain YTD aligned with the Ambient console instead of drifting over time.
Outcome
The system now has stable ingestion, a responsive API, cost-optimized long-range graphs backed by CloudFront, and a rainfall model that stays aligned with the station's cumulative telemetry instead of drifting over time. The application now behaves in a production-like manner, with predictable data freshness and controlled operating costs.
The work here was not theoretical - these were real failures that were traced and fixed through tighter IAM, clearer API wiring, smarter caching, and Terraform that accurately reflects the deployed platform.