Core Concepts

Understanding these fundamental concepts will help you make the most of Ona’s energy forecasting platform. This section explains how our system works, key terminology, and the principles behind accurate energy predictions.

How Ona Works

The Forecasting Pipeline

graph LR
    A[Historical Data] --> B[Data Preprocessing]
    B --> C[Feature Engineering]
    C --> D[Model Training]
    D --> E[Model Validation]
    E --> F[Deployment]
    F --> G[Real-time Forecasting]
    G --> H[Results Delivery]

1. Data Ingestion & Preprocessing

• Automatic data validation and cleaning
• Gap filling and interpolation
• Outlier detection and correction
• Timezone normalization

2. Feature Engineering

• Weather data integration (solar irradiance, temperature, wind)
• Calendar features (day of week, season, holidays)
• Lag features and rolling statistics
• Site-specific characteristics

3. Model Training

• Custom model per site/device
• Ensemble methods (XGBoost, LSTM, Prophet)
• Cross-validation and hyperparameter tuning
• Continuous learning from new data

4. Forecasting & Delivery

• Real-time predictions with confidence intervals
• Multiple horizon forecasts (1-hour to 7-day ahead)
• Automatic model updates and retraining
• Results via API, email, or webhook

Data Architecture

Regional Deployment

Ona operates in multiple regions for data sovereignty and performance:

Benefits:

• Low Latency: Regional processing reduces response times
• Data Sovereignty: Data stays within regulatory boundaries
• High Availability: Multi-region redundancy
• Compliance: GDPR, CCPA, and local data protection laws

Data Flow

CSV Upload → Lambda@Edge → Regional API Gateway → 
Preprocessing Service → S3 Data Lake → SageMaker Training → 
Model Registry → Inference Service → Results Storage

Forecasting Types

Historical Data Training

Purpose: Train custom models using your site’s historical data

Requirements:

• Minimum 12 months of data (36 months recommended)
• Regular intervals (15/30/60 minutes)
• <5% missing data points

Process:

1. Upload via /upload_historical endpoint
2. Trigger training via /train endpoint
3. Model validation and backtesting
4. Model deployment for live forecasting

Nowcast Forecasting

Purpose: Real-time forecasting using recent data

Use Cases:

• Day-ahead energy trading
• Grid balancing and dispatch
• Demand response optimization

Features:

• 1-48 hour forecast horizons
• 15-minute to 1-hour intervals
• Confidence intervals and uncertainty quantification
• Weather-based adjustments

Long-term Forecasting

Purpose: Strategic planning and capacity optimization

Horizons:

• Weekly (1-4 weeks ahead)
• Monthly (1-12 months ahead)
• Seasonal (quarterly forecasts)

Applications:

• Resource planning
• Maintenance scheduling
• Financial modeling
• Contract negotiations

Model Types & Algorithms

Ensemble Methods

Ona uses multiple algorithms to maximize accuracy:

XGBoost (Gradient Boosting):

• Excellent for complex, non-linear patterns
• Handles weather correlations well
• Fast training and inference
• Good for short-term forecasts (1-24 hours)

LSTM Neural Networks:

• Captures long-term temporal dependencies
• Learns seasonal patterns automatically
• Handles missing data gracefully
• Best for multi-day forecasts

Prophet (Time Series):

• Robust to outliers and missing data
• Excellent for seasonal decomposition
• Handles holidays and special events
• Good baseline model

Weather-Aware Models:

• Integrate real-time weather data
• Solar irradiance and cloud forecasting
• Temperature and wind speed correlations
• Geographic and topographic factors

Model Selection Logic

def select_model(forecast_horizon, data_quality, use_case):
    if forecast_horizon <= 24:  # Hours
        if data_quality > 0.95:
            return "XGBoost + Weather"
        else:
            return "LSTM + Interpolation"
    
    elif forecast_horizon <= 168:  # 1 week
        return "LSTM + Prophet Ensemble"
    
    else:  # Long-term
        return "Prophet + Seasonal Decomposition"

Data Quality & Preprocessing

Data Validation Pipeline

1. Format Validation

• Timestamp parsing and timezone handling
• Numeric value validation
• Column mapping and standardization
• File integrity checks

2. Quality Assessment

quality_metrics = {
    "completeness": missing_data_percentage,
    "consistency": interval_regularity_score, 
    "accuracy": outlier_detection_score,
    "freshness": data_recency_hours
}

3. Automatic Corrections

• Gap Filling: Linear interpolation for <2 hour gaps
• Outlier Removal: Statistical outliers beyond 3 standard deviations
• Smoothing: Kalman filtering for noisy sensor data
• Normalization: Power/energy unit conversions

Data Enrichment

Weather Integration:

• Satellite-based solar irradiance data
• Numerical weather prediction models
• Historical weather pattern analysis
• Microclimate adjustments

Calendar Features:

• National and regional holidays
• Daylight saving time transitions
• School holidays and business cycles
• Cultural and religious events

Accuracy & Performance Metrics

Standard Metrics

Mean Absolute Error (MAE):

MAE = (1/n) * Σ|actual - predicted|

• Easy to interpret in original units
• Robust to outliers
• Typical values: 8-15% for solar, 5-12% for load

Root Mean Square Error (RMSE):

RMSE = √[(1/n) * Σ(actual - predicted)²]

• Penalizes large errors more heavily
• Good for optimization objectives
• Always ≥ MAE

Mean Absolute Percentage Error (MAPE):

MAPE = (100/n) * Σ|((actual - predicted)/actual)|

• Scale-independent comparison
• Easy stakeholder communication
• Can be unstable near zero values

Advanced Metrics

Forecast Skill Score:

Skill = 1 - (MAE_model / MAE_baseline)

• Compares against simple baseline (persistence, climatology)
• Values > 0 indicate model adds value
• Industry standard for forecast evaluation

Pinball Loss (Quantile Accuracy):

• Evaluates prediction intervals
• Measures both coverage and sharpness
• Critical for risk management and trading

Benchmark Performance

Integration Patterns

Batch Processing

Use Case: Daily forecast generation for large portfolios

# Example batch workflow
def daily_batch_process():
    sites = get_active_sites()
    
    for site in sites:
        # Upload latest data
        upload_nowcast_data(site)
        
        # Generate forecasts
        forecast = generate_forecast(site, horizon='24h')
        
        # Store results
        store_forecast(site, forecast)
        
        # Send alerts if needed  
        check_forecast_alerts(site, forecast)

Real-time Streaming

Use Case: Live grid operations and trading

# Real-time processing
def process_real_time_data(site_data):
    # Validate incoming data
    validated_data = validate_stream(site_data)
    
    # Update running models
    update_online_model(validated_data)
    
    # Generate immediate forecast
    forecast = predict_next_interval(validated_data)
    
    # Publish to downstream systems
    publish_forecast(forecast)

Event-Driven Architecture

Use Case: Webhook-based forecast delivery

{
  "event": "forecast_ready",
  "site_id": "SE123456789", 
  "forecast_time": "2024-01-15T08:00:00Z",
  "horizon_hours": 24,
  "accuracy_metrics": {
    "mae": 0.12,
    "rmse": 0.18,
    "skill_score": 0.35
  },
  "download_url": "https://s3.amazonaws.com/ona-forecasts/..."
}

Error Handling & Reliability

Common Error Scenarios

1. Data Quality Issues

• Symptom: High forecast errors or model training failures
• Causes: Missing data, sensor malfunctions, format changes
• Solution: Automatic data validation and client notifications

2. Model Performance Degradation

• Symptom: Gradually increasing forecast errors
• Causes: Concept drift, equipment changes, seasonal shifts
• Solution: Automated retraining triggers and performance monitoring

3. API Availability

• Symptom: Request timeouts or 5xx errors
• Causes: High load, infrastructure issues, maintenance
• Solution: Regional failover, request queuing, status notifications

Reliability Features

Multi-Region Failover:

• Automatic routing to healthy regions
• Data replication across regions
• <1 minute failover time

Graceful Degradation:

• Fallback to simpler models if complex models fail
• Historical average as last resort
• Clear uncertainty indicators

Monitoring & Alerting:

• Real-time accuracy tracking
• Data quality dashboards
• Automated issue notifications
• SLA monitoring and reporting

Security & Compliance

Data Protection

Encryption:

• TLS 1.3 for data in transit
• AES-256 encryption for data at rest
• Key rotation every 90 days

Access Control:

• API key-based authentication
• Role-based access control (RBAC)
• IP whitelisting available
• Audit logging for all access

Data Retention:

• Customer data: Retained per contract terms
• Model artifacts: 2 years for reproducibility
• Logs: 90 days for debugging
• Secure deletion on request

Compliance Standards

Certifications:

• SOC 2 Type II compliance
• ISO 27001 information security
• GDPR data protection (EU)
• CCPA privacy compliance (California)

Industry Standards:

• IEEE 1547 grid interconnection
• IEC 61850 power system communication
• NERC CIP critical infrastructure protection

Next Steps

Now that you understand how Ona works:

1. Start building: API Reference for detailed endpoints
2. See examples: Use Cases for real implementations
3. Try the SDK: SDK Documentation for easier integration
4. Get support: Contact us for technical questions

Get Help & Stay Updated

© 2025 Asoba Corporation. All rights reserved.