c77_dbh/README.md

c77_dbh - Postgres Database Health Extension

c77_dbh is a PostgreSQL extension designed to monitor and maintain database health, providing actionable insights for database administrators and non-technical users alike. It offers a suite of functions and tables to track vacuum performance, index efficiency, materialized view refresh health, and TOAST table overhead—all packaged as a self-contained, drop-in solution for Postgres environments.

The extension pushes logic into the database itself, minimizing reliance on external frameworks, and outputs JSON-formatted data ideal for dashboards. It dynamically adjusts thresholds based on your database’s behavior and adapts to optional Postgres extensions (e.g., pgstattuple, pg_stat_statements) for enhanced precision.

Features

• Vacuum Health Monitoring: Tracks table bloat, vacuum history, and TOAST overhead with dynamic thresholds.
• Index Health Analysis: Monitors index usage and bloat, excluding or isolating TOAST indexes as needed.
• Materialized View Refresh Tracking: Measures refresh performance with event triggers, identifying bottlenecks (I/O, CPU, RAM).
• Human-Readable Insights: Provides status (Red/Yellow/Green), severity scores, insights, and actions for each metric.
• Dashboard-Ready: JSON outputs structured for easy integration into graphical interfaces.
• Resource Awareness: Gracefully falls back when optional extensions are unavailable.

Installation

Prerequisites

• PostgreSQL 13 or higher (tested up to 15 as of March 2025).
• Optional extensions for enhanced functionality:
  • pgstattuple: Precise bloat estimates for tables and indexes.
  • pg_stat_statements: Detailed query and refresh performance metrics.
  • pg_cron: Automate periodic stat updates (recommended).

Steps

1. Clone the Repository:

   git clone https://git.jctr3.com/trogers1884/c77_dbh.git
   cd c77_dbh
   

2. Build and Install:

   make
   sudo make install
   

   (Note: Assumes a standard Postgres extension build process; adjust paths if needed.)

3. Enable in Database:

   CREATE EXTENSION c77_dbh;
   

4. Grant Permissions (optional, adjust roles as needed):

   GRANT SELECT ON ALL TABLES IN SCHEMA public TO public;
   GRANT ALL ON ALL TABLES IN SCHEMA public TO your_admin_role;
   

Usage

Core Components

• Tables:

  • c77_dbh_vacuum_stats: Stores table vacuum and bloat stats.
  • c77_dbh_index_stats: Tracks index size, usage, and bloat.
  • c77_dbh_mv_stats: Records materialized view refresh metrics.

• Functions:

  • c77_dbh_update_vacuum_stats(): Updates vacuum stats for all tables.
  • c77_dbh_update_index_stats(): Updates index stats.
  • c77_dbh_get_vacuum_health(): Returns JSON with table health status.
  • c77_dbh_get_index_health(): JSON for non-TOAST index health.
  • c77_dbh_get_toast_health(): JSON for TOAST index health.
  • c77_dbh_get_mv_health(): JSON for materialized view refresh health.
  • c77_dbh_get_mv_details(mv_name text): Detailed JSON for a specific MV.

• Event Triggers: Automatically track materialized view creation, alteration, and refreshes.

Example Workflow

1. Initialize Stats:

   SELECT c77_dbh_update_vacuum_stats();
   SELECT c77_dbh_update_index_stats();
   

2. Query Health:

   SELECT * FROM c77_dbh_get_vacuum_health();
   SELECT * FROM c77_dbh_get_index_health();
   SELECT * FROM c77_dbh_get_mv_health();
   

3. Automate Updates (with pg_cron):

   SELECT cron.schedule('update_vacuum_stats', '0 * * * *', $$SELECT c77_dbh_update_vacuum_stats()$$);
   SELECT cron.schedule('update_index_stats', '0 * * * *', $$SELECT c77_dbh_update_index_stats()$$);
   

4. Sample Output (from c77_dbh_get_vacuum_health()):

   [
     {
       "metric": "Table Vacuum Health",
       "table_name": "sales.orders",
       "status": "Red",
       "severity": 5,
       "insight": "Table bloat is critically high; TOAST overhead significant.",
       "action": "Run VACUUM FULL; review large fields.",
       "table_size": "128 MB",
       "bloat_estimate": "65%",
       "details": {"toast_bloat": "40%", "median_bloat": "20%"}
     }
   ]
   

Dashboard Integration

• Use the JSON outputs to populate a dashboard:
  • Vacuum Health: Pie chart of statuses, table of critical issues.
  • Index Health: Bar chart of usage categories, recommendations.
  • MV Refresh: Progress bars per view, bottleneck alerts.
• Link to detail pages using details fields or separate functions like c77_dbh_get_toast_health().

Configuration

• Frequency: Adjust update schedules based on database activity (hourly for high-write systems, daily for stable ones).
• Permissions: Default setup grants SELECT to public and full access to an admin role (e.g., homestead). Modify as needed.
• Extensions: Install pgstattuple and pg_stat_statements for best results; fallbacks work without them.

Limitations

• TOAST bloat estimates are crude without pgstattuple.
• Materialized view bottleneck analysis is less precise without pg_stat_statements.
• Dynamic thresholds rely on sufficient historical data; new databases may need time to stabilize.

Future Development

• Query performance monitoring with pg_stat_statements.
• Automated maintenance actions (e.g., trigger VACUUM on Red status).
• Integration with external frameworks (e.g., Laravel package).

Contributing

Contributions are welcome! Fork the repo, submit pull requests, or open issues at https://git.jctr3.com/trogers1884/c77_dbh.

License

MIT License – see LICENSE file for details.

Credits

Developed by [Your Name] with assistance from xAI’s Grok. Inspired by Dr. W. Edwards Deming’s statistical quality control principles.