c77_dbh/README.md

# c77_dbh - Postgres Database Health Extension

**c77_dbh** is a PostgreSQL extension designed to monitor and maintain database health, providing actionable insights for database administrators and non-technical users alike. It offers a suite of functions and tables to track vacuum performance, index efficiency, materialized view refresh health, and TOAST table overhead—all packaged as a self-contained, drop-in solution for Postgres environments.

The extension pushes logic into the database itself, minimizing reliance on external frameworks, and outputs JSON-formatted data ideal for dashboards. It dynamically adjusts thresholds based on your database’s behavior and adapts to optional Postgres extensions (e.g., `pgstattuple`, `pg_stat_statements`) for enhanced precision.

## Features

- **Vacuum Health Monitoring**: Tracks table bloat, vacuum history, and TOAST overhead with dynamic thresholds.
- **Index Health Analysis**: Monitors index usage and bloat, excluding or isolating TOAST indexes as needed.
- **Materialized View Refresh Tracking**: Measures refresh performance with event triggers, identifying bottlenecks (I/O, CPU, RAM).
- **Human-Readable Insights**: Provides status (Red/Yellow/Green), severity scores, insights, and actions for each metric.
- **Dashboard-Ready**: JSON outputs structured for easy integration into graphical interfaces.
- **Resource Awareness**: Gracefully falls back when optional extensions are unavailable.

## Installation

### Prerequisites
- PostgreSQL 13 or higher (tested up to 15 as of March 2025).
- Optional extensions for enhanced functionality:
  - `pgstattuple`: Precise bloat estimates for tables and indexes.
  - `pg_stat_statements`: Detailed query and refresh performance metrics.
  - `pg_cron`: Automate periodic stat updates (recommended).

### Steps
1. **Clone the Repository**:
   ```bash
   git clone https://git.jctr3.com/trogers1884/c77_dbh.git
   cd c77_dbh
   ```

2. **Build and Install**:
   ```bash
   make
   sudo make install
   ```
   (Note: Assumes a standard Postgres extension build process; adjust paths if needed.)

3. **Enable in Database**:
   ```sql
   CREATE EXTENSION c77_dbh;
   ```

4. **Grant Permissions** (optional, adjust roles as needed):
   ```sql
   GRANT SELECT ON ALL TABLES IN SCHEMA public TO public;
   GRANT ALL ON ALL TABLES IN SCHEMA public TO your_admin_role;
   ```

## Usage

### Core Components
- **Tables**:
  - `c77_dbh_vacuum_stats`: Stores table vacuum and bloat stats.
  - `c77_dbh_index_stats`: Tracks index size, usage, and bloat.
  - `c77_dbh_mv_stats`: Records materialized view refresh metrics.

- **Functions**:
  - `c77_dbh_update_vacuum_stats()`: Updates vacuum stats for all tables.
  - `c77_dbh_update_index_stats()`: Updates index stats.
  - `c77_dbh_get_vacuum_health()`: Returns JSON with table health status.
  - `c77_dbh_get_index_health()`: JSON for non-TOAST index health.
  - `c77_dbh_get_toast_health()`: JSON for TOAST index health.
  - `c77_dbh_get_mv_health()`: JSON for materialized view refresh health.
  - `c77_dbh_get_mv_details(mv_name text)`: Detailed JSON for a specific MV.

- **Event Triggers**: Automatically track materialized view creation, alteration, and refreshes.

### Example Workflow
1. **Initialize Stats**:
   ```sql
   SELECT c77_dbh_update_vacuum_stats();
   SELECT c77_dbh_update_index_stats();
   ```

2. **Query Health**:
   ```sql
   SELECT * FROM c77_dbh_get_vacuum_health();
   SELECT * FROM c77_dbh_get_index_health();
   SELECT * FROM c77_dbh_get_mv_health();
   ```

3. **Automate Updates** (with `pg_cron`):
   ```sql
   SELECT cron.schedule('update_vacuum_stats', '0 * * * *', $$SELECT c77_dbh_update_vacuum_stats()$$);
   SELECT cron.schedule('update_index_stats', '0 * * * *', $$SELECT c77_dbh_update_index_stats()$$);
   ```

4. **Sample Output** (from `c77_dbh_get_vacuum_health()`):
   ```json
   [
     {
       "metric": "Table Vacuum Health",
       "table_name": "sales.orders",
       "status": "Red",
       "severity": 5,
       "insight": "Table bloat is critically high; TOAST overhead significant.",
       "action": "Run VACUUM FULL; review large fields.",
       "table_size": "128 MB",
       "bloat_estimate": "65%",
       "details": {"toast_bloat": "40%", "median_bloat": "20%"}
     }
   ]
   ```

### Dashboard Integration
- Use the JSON outputs to populate a dashboard:
  - **Vacuum Health**: Pie chart of statuses, table of critical issues.
  - **Index Health**: Bar chart of usage categories, recommendations.
  - **MV Refresh**: Progress bars per view, bottleneck alerts.
- Link to detail pages using `details` fields or separate functions like `c77_dbh_get_toast_health()`.

## Configuration

- **Frequency**: Adjust update schedules based on database activity (hourly for high-write systems, daily for stable ones).
- **Permissions**: Default setup grants `SELECT` to `public` and full access to an admin role (e.g., `homestead`). Modify as needed.
- **Extensions**: Install `pgstattuple` and `pg_stat_statements` for best results; fallbacks work without them.

## Limitations

- TOAST bloat estimates are crude without `pgstattuple`.
- Materialized view bottleneck analysis is less precise without `pg_stat_statements`.
- Dynamic thresholds rely on sufficient historical data; new databases may need time to stabilize.

## Future Development

- Query performance monitoring with `pg_stat_statements`.
- Automated maintenance actions (e.g., trigger `VACUUM` on Red status).
- Integration with external frameworks (e.g., Laravel package).

## Contributing

Contributions are welcome! Fork the repo, submit pull requests, or open issues at `https://git.jctr3.com/trogers1884/c77_dbh`.

## License

MIT License – see `LICENSE` file for details.

## Credits

Developed by [Your Name] with assistance from xAI’s Grok. Inspired by Dr. W. Edwards Deming’s statistical quality control principles.