commit 61898e6debe80573db667f329be51917a14ee0dd Author: trogers1884 Date: Wed Mar 26 10:48:15 2025 -0500 Add Dependency Map diff --git a/dependency-map.md b/dependency-map.md new file mode 100644 index 0000000..c00e2ef --- /dev/null +++ b/dependency-map.md @@ -0,0 +1,153 @@ +# PostgreSQL Function Dependency Map + +## Overview + +This document maps the dependencies between the PostgreSQL functions in the `config` schema. The functions are organized into two main subsystems: + +1. **Table Analysis Subsystem**: Functions for analyzing tables to identify optimal keys, partitioning strategies, and data quality issues +2. **Materialized View Management Subsystem**: Functions for creating, monitoring, and maintaining materialized views + +## Table Analysis Subsystem + +### Main Entry Point +- `config.grok_analyze_table_fitness` - Orchestrates the complete table analysis process + +### Dependency Hierarchy + +``` +grok_analyze_table_fitness +├── grok_calculate_sample_size +├── grok_create_temp_table +├── grok_analyze_column_stats +├── grok_identify_order_by_candidates +├── grok_analyze_column_combinations +├── grok_calculate_dqi +└── grok_assemble_result +``` + +### Function Relationships + +1. `grok_analyze_table_fitness` + - Calls `grok_calculate_sample_size` to determine appropriate sample size + - Calls `grok_create_temp_table` to create a temporary copy of the source table + - Calls `grok_analyze_column_stats` for each column to analyze its characteristics + - Calls `grok_identify_order_by_candidates` to find columns suitable for ordering + - Calls `grok_analyze_column_combinations` to identify potential composite keys + - Calls `grok_calculate_dqi` to calculate the Data Quality Index + - Calls `grok_assemble_result` to prepare the final results and clean up + +2. `grok_analyze_column_stats` + - No dependencies on other functions + - Results are used by `grok_analyze_column_combinations`, `grok_identify_order_by_candidates`, and `grok_calculate_dqi` + +3. `grok_calculate_dqi` + - Uses data from `grok_analyze_column_stats` + - No direct function dependencies + +4. `grok_create_temp_table` + - No dependencies on other functions + - Creates temporary tables used by other analysis functions + +## Materialized View Management Subsystem + +### Main Entry Points +- `grok_create_optimized_matv` - Creates an optimized materialized view system +- `grok_manage_matv_health` - Monitors and maintains materialized view health + +### Dependency Hierarchy for Creation + +``` +grok_create_optimized_matv +├── grok_generate_column_lists (not explicitly called but similar functionality) +├── grok_generate_synthetic_key_and_hash (not explicitly called but similar functionality) +└── grok_create_indexes (not explicitly called but similar functionality) +``` + +### Dependency Hierarchy for Health Management + +``` +grok_manage_matv_health +├── grok_check_matv_mismatches +├── grok_estimate_matv_refresh_time +└── grok_perform_matv_action (indirectly) + +grok_perform_matv_action +└── (No function dependencies) + +grok_assemble_matv_health_result +└── grok_estimate_matv_refresh_time +``` + +### Function Relationships + +1. `grok_create_optimized_matv` + - Has similar functionality to `grok_generate_column_lists` but doesn't call it directly + - Has similar functionality to `grok_generate_synthetic_key_and_hash` but doesn't call it directly + - Has similar functionality to `grok_create_indexes` but doesn't call it directly + - Creates a complete materialized view system (source view, materialized view, and read views) + +2. `grok_manage_matv_health` + - Calls `grok_check_matv_mismatches` to detect inconsistencies + - Calls `grok_estimate_matv_refresh_time` to estimate refresh times + - Contains embedded functionality similar to `grok_perform_matv_action` + +3. `grok_perform_matv_action` + - No direct function dependencies + - Performs maintenance actions on materialized views + +4. `grok_assemble_matv_health_result` + - Calls `grok_estimate_matv_refresh_time` to get refresh time estimates + - Formats health check results + +5. `grok_check_matv_mismatches` + - No direct function dependencies + - Performs content hash comparison between source and materialized views + +6. `grok_validate_matv_inputs` + - No direct function dependencies + - Validates materialized view and source view existence + +7. `grok_set_validation_params` + - No direct function dependencies + - Configures validation parameters for health checks + +## Utility Functions + +1. `grok_calculate_sample_size` + - Called by `grok_analyze_table_fitness` + - Called by `grok_calculate_matv_sample_size` (though the result is unused) + +2. `grok_calculate_matv_sample_size` + - Calls `grok_calculate_sample_size` but doesn't use the result + - Used for materialized view validation sampling + +3. `grok_estimate_matv_refresh_time` + - Called by `grok_assemble_matv_health_result` + - Called by `grok_manage_matv_health` + - Estimates materialized view refresh times + +4. `grok_validate_order_by_columns` + - No direct function dependencies + - Validates timestamp-like columns for ordering + +## Integration Points + +The two subsystems integrate at these key points: + +1. **Table Analysis → Materialized View Creation**: + - Analysis results from `grok_analyze_table_fitness` can inform parameters for `grok_create_optimized_matv` + - Recommended partition columns and order-by columns can be used directly + +2. **Materialized View Management**: + - Both `grok_create_indexes` and `grok_create_optimized_matv` create similar index structures + - `grok_assemble_matv_result` and `grok_assemble_matv_health_result` format related outputs + +## External Dependencies + +These functions depend on external database objects: + +1. **Table Fitness Audit Table**: + - `config.table_fitness_audit` - Stores table analysis results + +2. **Materialized View Statistics Tables**: + - `public.c77_dbh_matv_stats` - Stores materialized view refresh statistics