IoT Weather Station — Technical Overview

Industrial teams rely on trustworthy telemetry from field devices to make operational decisions and detect issues quickly. In petrochemical refinery operations, engineers continuously monitor signals such as flow rates, pressures, temperatures, and equipment status to ensure safe and efficient processes.

The challenge is rarely the sensor itself; it is the surrounding system. Secure ingestion, reliable processing, data freshness guarantees, historical analysis, operational visibility, and predictable costs all matter. Gaps in telemetry, delayed ingestion, or unreliable pipelines can directly increase operational risks.

This project models those problem areas using a physical weather station as a safe, low-cost stand-in for industrial instrumentation. While the measurements are meteorological, the system behavior is realistic: frequent telemetry updates, intermittent dropouts, noisy data, and the need to support both near real-time monitoring and historical analysis.

The telemetry pipeline is implemented on AWS using AWS IoT Core for device ingestion, AWS Lambda for processing and aggregation, Amazon Timestream for time-series storage, Amazon S3 and CloudFront for low-cost historical access, and API Gateway for controlled live reads. The design reflects production-style tradeoffs around reliability, observability, and cost containment rather than focusing on the sensors themselves.

Quick demo video

I will be recording a short ~5 minute walkthrough explaining the user flow and my key design choices.
 Watch on YouTube (coming soon)

AWS Architecture

View Architecture Diagram (PDF)
  • Weather station: Ambient WS-2902D weather station with WH65B sensor array. The station provides real physical telemetry, including temperature, humidity, wind, rainfall, and device power status.
  • Edge device: Raspberry Pi 3 with RTL-SDR running rtl_433 -s 250k. The adjusted sample rate allows the WH65B sensor array to be decoded correctly before the data enters AWS, keeping the telemetry model consistent from the start.
  • Device ingestion: Python publisher → MQTT → AWS IoT Core. The publisher normalizes sensor readings into JSON and sends the telemetry securely into AWS for processing.
  • Ingest processing: AWS IoT Core Rule WeatherDataRule → Lambda (wx_ingest) → Amazon Timestream. The ingest function validates and writes weather telemetry into the Timestream table weather_MQTT_data_mm.
  • Live reads (1h / 24h): API Gateway HTTP API → Lambda (get_weather_data) → Amazon Timestream → JSON. Used for short time windows where freshness and low latency matter.
  • Rainfall baseline: DynamoDB table weather-station-state. Stores the yearly rainfall baseline used to calculate dashboard Rain YTD from the latest validated rainfall reading.
  • Precomputed history (7d / 30d): EventBridge → Lambda (wx_precompute) and Lambda (wx_rollups_daily) → JSON written to S3. Reduces live database query volume and keeps longer-range dashboard views cost-efficient.
  • Frontend delivery: Static dashboard hosted on S3 + CloudFront, with AWS WAF in front of CloudFront. CloudFront serves the dashboard and cached historical JSON while WAF adds a protective edge layer.
  • Caching strategy: CloudFront ordered behavior for /data/*. Edge caching is used to serve precomputed history while preventing repeated backend reads.
  • Infrastructure: Entire stack is defined and managed with Terraform; Lambda runtime Python 3.12 (arm64) for cost efficiency and consistency.

Data Paths

  • Write path: Ambient weather station → Raspberry Pi / SDR → Python publisher → AWS IoT Core → IoT Rule → Lambda → Timestream. Optimized for reliable ingestion of continuous telemetry with minimal coupling to downstream consumers.
  • Read path (short windows): Browser → CloudFront / WAF → API Gateway → Lambda → Timestream. Used only for recent data (1h / 24h) where freshness and low latency matter.
  • Read path (long windows): Browser → CloudFront → S3 (precomputed JSON). Used for historical views (7d / 30d) to avoid repeated database queries and stabilize cost.

Rainfall Data Model

The station reports rainfall as a running cumulative counter, not as a dashboard-ready year-to-date value. The dashboard calculates Rain YTD by comparing the latest corrected rainfall value in Timestream against a yearly baseline stored in DynamoDB.

rain_ytd_in = latest corrected rain_in - DynamoDB baseline_rain_in

The baseline record is stored in weather-station-state using pk = WeatherStation and sk = rain_baseline#2026. This keeps the rainfall calculation stable without relying on hardcoded offsets.

A key operational lesson from this project was that accurate telemetry starts before the data reaches AWS. Updating the rtl_433 configuration to use -s 250k allowed the WH65B sensor array to be decoded correctly, aligning the dashboard Rain YTD calculation with the Ambient weather station's console display.

Cost Controls

  • Precompute large ranges to S3 (cheap storage + edge caching) instead of live database queries
  • EventBridge every 30 minutes (not per‑request)
  • CloudFront cache policy for /data/* to limit backend reads while keeping data reasonably fresh
  • Targeted Timestream queries (binned windows; short ranges only via API)
  • Frontend refresh guardrail (the 1h table): The dashboard auto-refreshes only for the 1-hour weather data view and pauses after ~3 minutes (3 polls). Users can manually refresh to continue, which prevents idle tabs from driving unnecessary API/Lambda/Timestream query costs.

Alarms

  • Ingestion liveness — IoT IncomingMessages sum == 0 over 5 minutes → notify (early indicator that the edge device or network stalled)
  • Lambda errors — Notify via email on any errors with ingestion or Lambda
  • API health — HTTP API 5xx rate alarm → notify
  • Edge efficiency — CloudFront cache hit rate dip → review caching headers/patterns

Operations & Monitoring

  • Structured Lambda logs: query strings, row counts, and write results for fast triage
  • Least-privilege IAM: separate roles for API read, ingest, and precompute workflows
  • Known behavior: charts show full 7/30-day frames; they only render days with available data
  • Rainfall validation: CloudWatch logs and dashboard comparisons were used to verify that the corrected telemetry model aligned with the Ambient console reporting.

Security

  • SigV4 WebSocket auth for device → IoT Core
  • CORS minimal; HTTP API restricted to POST for data queries

Why This Design

  • Fast UX: Long graphs are served as static JSON at the edge, while short views come from a low-latency API
  • Low cost: Timestream is used for recent operational reads; S3 + CloudFront handle longer-range history cheaply
  • Telemetry realism: The design preserves real sensor behavior, including cumulative rainfall counters, and derives dashboard metrics from validated telemetry instead of flattening everything into simple snapshots
  • Simple ops: CloudWatch alarms and structured logs make it easier to spot ingest failures, stale data, and rollup regressions quickly

Tech Stack

Ambient WS-2902D / WH65B sensor array, Raspberry Pi 3, RTL-SDR, rtl_433, Python publisher, AWS IoT Core, Lambda (Python 3.12, arm64), Amazon Timestream, DynamoDB, EventBridge, API Gateway (HTTP API), S3 static hosting, CloudFront, AWS WAF, Terraform Cloud.

 

Back to Dashboard Weather Station Challenges