diff --git a/README.md b/README.md new file mode 100644 index 0000000..392c6e5 --- /dev/null +++ b/README.md @@ -0,0 +1,168 @@ +# Azure DevOps → GitHub Processor (GUI) + +Azure DevOps -> GitHub Processor is a small, focused GUI tool that you can use to perform the following tasks: + +1. Fetches Azure DevOps work items using a query you specify and extracts documentation change information from them. +1. Creates a GitHub issues using the information and assigns the issue to GitHub Copilot, which analyzes the request, creates a branch, updates the article to fix the issue, and submits a pull request (PR) on behalf of you. + + The following diagram shows you the high-level steps performed by this solution: + + ![High-level diagram of the Azure DevOps -> GitHub Processor solution](./media/flow-diagram.png) + + The PR is assigned to the author of the article based on the author metadata in the article. The author reviews off the PR and signs on it if the changes done by Copilot look good. + +The `devops_to_github.py` file in the `Devops_to_GitHub_Python/latest` folder has all the code behind this tool. + +## Prerequisites +Here are the prerequisites to use the tool: + +- Python 3.10 or newer (the code uses modern union types like `dict | None`) +- Install packages used in the code, such as `requests` and `tkinter`, which is used for the GUI. +- GitHub Copilot must be enabled at the organization level or for the specific repository. +- You must have admin access to the repository to assign issues to Copilot. + +### Azure DevOps prerequisites + +- **Azure DevOps Query URL**: A shared query URL containing documentation update work items. You can use this [sample query](https://dev.azure.com/msft-skilling/Content/_queries/query/ab92a364-1376-43d5-a6da-510aa40a65d1/) for testing purposes. + + Work items must contain the following fields in their description: + + - **Nature of Request**: Must include **Modify existing docs**. + - **Link to doc**: Must be a URL to a Learn article. + - **Text to change**: Current text that needs to update. Can be blank. + - **Proposed new text**: Suggested replacement text. This text can be instructional such as "Remove last two links from the Related content section". In this example, the GitHub Copilot doesn't replace old text with new text. It just removes those two links. +- **Azure DevOps Personal Access Token (PAT)**: Token with work item read, write, and manage permissions. The write permission is used to write back the link to GitHub issue in the DevOps work item. For instructions on creating a token, see [Authenticate to DevOps with personal access tokens](https://learn.microsoft.com/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows#create-a-pat). + +### GitHub prerequisites +- **GitHub Personal Access Token**: A **classic** token with **repo**, **admin:org**, and **copilot** permissions. For instructions on creating a token, see [Create a personal access token (classic)](https://docs.github.com/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic) + +## Configuration and tokens management +The app first looks for tokens in the **launch.json** file and then in the **.env** file. We recommend that you use the **.env** approach. + +Create a `.env` file in the repository root and add: + +``` +AZURE_DEVOPS_QUERY=your_azure-devops-query-url +AZURE_DEVOPS_PAT=your_azure_devops_pat_here +GITHUB_PAT=your_github_pat_here +``` + +**Security reminder**: Never commit secrets. Add `.env` to `.gitignore` if you plan to store tokens locally. + +### launch.json + +Create or edit `.vscode/launch.json` and add the values into the `env` object for the configuration you run. Example snippet: + +```json +{ + "configurations": [ + { + "name": "Run devops_to_github", + "type": "python", + "request": "launch", + "program": "${workspaceFolder}/POC/latest/devops_to_github.py", + "console": "integratedTerminal", + "env": { + "AZURE_DEVOPS_QUERY": "", + "AZURE_DEVOPS_PAT": "", + "GITHUB_PAT": "" + } + } + ] +} + +``` +- If you use a VS Code launch configuration, do NOT put secrets in `.vscode/launch.json`. + - Instead use an env file and reference it via `"envFile": "${workspaceFolder}/.env"` or keep only placeholders in a committed `launch.json.example`. +- For CI and shared environments, use secret stores (GitHub Actions secrets, Azure Key Vault, etc.)—do not commit tokens into the repo. + +**Warning:** Never commit PATs, tokens, or other secrets. If a secret is accidentally committed, rotate it immediately. + +## How to run the app +Use one of the following methods: + +- From a terminal (PowerShell on Windows), run the following command: `python .\POC\latest\devops_to_github.py` +- Or run the `Run`/`Debug` configuration in VS Code if you configured `launch.json` (or) `.env`. + +## Using the app (overview) +1. Enter or verify the Azure DevOps Query URL and the PATs (or rely on loaded config from `launch.json` / `.env`). +1. Toggle **Dry Run**" to simulate operations (recommended for first runs). +1. Select **Fetch Work Items** to execute the Azure query and parse work items (this invokes `start_fetch_work_items` -> `fetch_work_items`). + + ![UI after fetching work items from Azure DevOps.](./media/fetch-work-items.png) +1. Navigate items with **← Previous** / **Next →**. Each item shows the parsed fields (`nature_of_request`, `mydoc_url`, `text_to_change`, `new_text`), detected GitHub repo (owner/repo), and `ms.author` when available. +1. Edit the **proposed new text** inline by selecting the ✏️ Edit control, make changes, then **Save**. +1. Select **🚀 CREATE ISSUE** to create an issue in the detected GitHub repo. On success you will get a hyperlink dialog with the created issue URL. + + ![UI after creating a GitHub issue.](./media/issue-created.png) + + Here's what the tool does when you select **Create issue** + - Uses GitHub GraphQL API to fetch the repository ID and call `createIssue`. + - Attempts to locate a suggested actor named like Copilot and assign the new issue to that actor. + - Includes an `AB#` line in the issue body to trace back to the Azure work item. + - If available, calls Azure DevOps API to add a hyperlink back to the Azure work item referencing the new GitHub issue URL. +1. Select the link to navigate to the GitHub issue to do the following tasks. In your Azure DevOps work item, you should see a link to the GitHub issue in the **GitHub issue** section as well. + - Review the details + - Notice that the GitHub issue is assigned to a Copilot + - The Copilot has created a pull request. + - A link back to Azure DevOps work item + + ![GitHub issue UI.](./media/github-issue-copilot-pr.png) +1. Select the pull request to review changes, and sign off on the PR after a successful build and the changes look good to you. + + ![GitHub Pull Request UI.](./media/github-pull-request.png) + +## Dry-run and logging +Dry-run prints GraphQL payloads and simulates the behavior without performing remote mutations. Inspect the "Processing Log" tab to see payloads and debug messages. + +## Notes, limitations & troubleshooting +- The UI currently focuses on creating issues. There is a GraphQL `createPullRequest` function implemented, but the PR option is not exposed in the GUI at the moment. +- The work item description parsing is opinionated and expects fields like "Nature of Request:", "Link to Doc:", "Text to Change:", and "Proposed new text:". If parsing fails, the work item will be skipped. +- Document URL extraction relies on the document's HTML metadata (`original_content_git_url` or similar). If the metadata isn't present or the URL is not a GitHub URL, the tool cannot determine owner/repo. +- Common errors: + - Invalid query URL: ensure the query URL uses `_queries/query//` or includes `queryId=` + - Token permission errors: check PAT scopes + - Network timeouts: check VPN/firewall rules and increase timeouts in code if needed + +## Developer notes +- File of interest: `POC/latest/devops_to_github.py` (main GUI + logic) +- Logging is visible in the GUI under "Processing Log". The app is single-process with background threads for network calls. +- Python typing uses modern syntax (requires Python 3.10+). + +## Process flow (diagram) + +The following diagram illustrates the end-to-end processing flow implemented by `POC/latest/devops_to_github.py`. It shows how the application fetches Azure DevOps work items, validates and parses each item, discovers the target GitHub repository from the document metadata, and then creates and (optionally) assigns and links the resulting GitHub issue back to the Azure DevOps work item. + +![Hack2025 processing flow](Devops_to_GitHub_Python/latest/Hack2025_flow.png) + +High-level steps shown in the diagram: + +1. User launches the GUI and provides the Azure DevOps query URL and tokens. +1. The app validates the query URL and executes the query to fetch work item IDs. +1. For each work item the app fetches details and parses the description for the required fields (Nature of Request, Link to Doc, Text to Change, Proposed new text). +1. If the item contains a valid document URL, the tool requests the document HTML and extracts `original_content_git_url` and metadata such as `ms.author`. +1. The tool maps the document metadata to a GitHub `owner/repo`, builds an issue title and body, and uses the GitHub GraphQL API to create the issue (dry-run mode simulates this). +1. If available, Copilot (or another suggested actor) is assigned to the created issue; the tool will also attempt to add a hyperlink back to the Azure work item. +1. The GUI updates the item's processing status and shows a link to the created issue on success. + +## Contributing + +This project welcomes contributions and suggestions. Most contributions require you to agree to a +Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us +the rights to use your contribution. For details, visit [Contributor License Agreements](https://cla.opensource.microsoft.com). + +When you submit a pull request, a CLA bot will automatically determine whether you need to provide +a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions +provided by the bot. You will only need to do this once across all repos using our CLA. + +This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). +For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or +contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. + +## Trademarks + +This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft +trademarks or logos is subject to and must follow +[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/legal/intellectualproperty/trademarks/usage/general). +Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. +Any use of third-party trademarks or logos are subject to those third-party's policies. diff --git a/SETUP.md b/SETUP.md new file mode 100644 index 0000000..22cd43e --- /dev/null +++ b/SETUP.md @@ -0,0 +1,307 @@ +# MicrosoftDocFlow Tool - Setup Guide + +## Overview + +This tool automates the process of converting Azure DevOps work items into GitHub pull requests with AI assistance. It fetches work items, processes documentation changes, and creates pull requests with proper diffs. + +## Quick Start + +### Prerequisites + +- **Python 3.8+** installed on your system +- **Git** installed and configured +- **Azure DevOps** account with work items +- **GitHub** account with repository access +- **AI Provider** account (optional, for enhanced processing) + +### Installation + +1. **Clone/Download the Repository** + + ```bash + git clone https://github.com/yourusername/github_automation.git + cd github_automation + ``` + +2. **Create Virtual Environment** (Recommended) + + ```bash + # Create virtual environment + python -m venv venv + + # Activate virtual environment + # On Windows: + venv\Scripts\activate + + # On macOS/Linux: + source venv/bin/activate + ``` + + To deactivate the environment when done: + + ```bash + deactivate + ``` + +3. **Install Dependencies** + + ```bash + pip install -r application/requirements.txt + ``` + +4. **Run the Application** + + ```bash + python application/app.py + ``` + +### Virtual Environment Management + +**Activating the environment** (when returning to the project): + +- Windows: `venv\Scripts\activate` +- macOS/Linux: `source venv/bin/activate` + +**Deactivating the environment** (when done working): + +```bash +deactivate +``` + +**Why use a virtual environment?** + +- **Isolation**: Keeps project dependencies separate from system Python +- **Clean installs**: Prevents conflicts with other Python projects +- **Reproducible**: Ensures consistent dependency versions +- **Safe updates**: Won't affect other projects when updating packages + +## Project Structure + +The project is organized as follows: + +```text +github_automation/ +├── application/ # Main application +│ ├── app.py # Application entry point +│ ├── requirements.txt # Python dependencies +│ └── app_components/ # Application modules +│ ├── ai_manager.py # AI provider +│ ├── azure_devops_api.py # Azure DevOps API +│ ├── cache_manager.py # Work item caching +│ ├── config_manager.py # Configuration +│ ├── dataverse_api.py # Dataverse API +│ ├── github_api.py # GitHub API client +│ ├── main_gui.py # Main GUI interface +│ ├── settings_dialog.py # Settings dialog +│ ├── utils.py # Utility functions +│ └── work_item_processor.py # Work item +├── media/ # Images and assets +├── README.md # Project overview +├── SETUP.md # This setup guide +└── LICENSE # License information +``` + +## Configuration + +### First-Time Setup + +1. **Launch the application** and click "Settings" button +2. **Configure required fields** in the Settings dialog: + +#### Azure DevOps Configuration (Required) + +- **Query URL**: Your Azure DevOps query URL + - Example: `https://dev.azure.com/yourorg/project/_queries/query/12345678-1234-1234-1234-123456789abc/` + - Get this from Azure DevOps by creating/opening a query and copying the URL +- **Personal Access Token**: Azure DevOps PAT with work item read permissions + - Create at: `https://dev.azure.com/yourorg/_usersSettings/tokens` + - Required scopes: Work Items (Read/Write) + +#### GitHub Configuration (Required) + +- **Personal Access Token**: GitHub PAT for repository access + - Create at: `https://github.com/settings/tokens` + - Required scopes: repo, workflow +- **Target Repository**: Format as `owner/repository` + - Example: `microsoft/fabric-docs` +- **Forked Repository**: Your fork of the target repository + - Example: `yourusername/fabric-docs` +- **Local Repo Path**: Directory where repositories will be/are cloned + - Example: `C:\Users\yourname\repos\` + +#### AI Provider Configuration (Optional) + +- **Provider**: Choose from: + - `none` - GitHub Copilot via a automatic comment on the PR + - `claude` - Anthropic Claude API + - `chatgpt` - OpenAI ChatGPT API + - `github-copilot` - GitHub Models API +- **API Keys**: Provide keys based on your chosen provider + - GitHub Token: Auto-defaults to GitHub PAT if left empty + +### Configuration Tips + +**Start Simple**: Configure only Azure DevOps and GitHub initially +**Test Connection**: Use "Test Connection" button to verify settings +**Cache Benefits**: Work items are cached for faster subsequent loads +**AI Enhancement**: Add AI provider later for automated text processing + +## Usage Guide + +### Basic Workflow + +1. **Load Work Items** + - Click "Fetch Work Items" to load work items from your query + - Items are displayed in the "All Work Items" tab + - Cached items load automatically on subsequent launches + +2. **Select Work Item** + - Navigate to "All Work Items" tab + - **Double-click** any work item to select it as current + - OR click "Set as Current Item" button + - Selected item appears in "Current Work Item" tab + +3. **Review Work Item Details** + - **Work Item ID**: Click to open in Azure DevOps + - **Nature of Request**: Description of required changes + - **Document URL**: Target documentation URL + - **Text to Change**: Current text that needs modification + - **Proposed New Text**: Replacement text (editable) + +4. **Process Changes** + - **With AI**: If AI provider is configured and the AI provider is not `none`, click "Create PR" and the selected AI will be used + - **Manual**: Click "Create PR" or "Create Issue" for manual processing / GitHub Copilot comment + - **Dry Run**: Enable for testing without actual changes (Located in settings) + +### Advanced Features + +#### Work Item Navigation + +- **Next/Previous**: Navigate through multiple work items +- **Edit Mode**: Click "Edit" to modify proposed text + +#### Git Diff Viewer + +- View file changes in the "Git Diff" tab +- Shows before/after comparison +- Automatic diff generation from repository changes + +#### Processing Log + +- Real-time activity logging in "Processing Log" tab +- Track API calls, file operations, and errors +- Detailed workflow visibility + +## Advanced Configuration + +### AI Provider Setup + +#### Claude (Anthropic) + +1. Get API key from [console.anthropic.com](https://console.anthropic.com) +2. Set provider to `claude` +3. Enter API key in settings +4. Cost: ~$0.01-0.05 per work item + +#### ChatGPT (OpenAI) + +1. Get API key from [platform.openai.com/api-keys](https://platform.openai.com/api-keys) +1. Set provider to `chatgpt` +1. Enter API key in settings +1. Cost: ~$0.01-0.05 per work item + +#### GitHub Copilot + +1. Ensure GitHub account has Copilot access +2. Set provider to `github-copilot` +3. GitHub Token auto-defaults to GitHub PAT +4. Uses GitHub Models API + +### Repository Management + +#### Local Repository Setup + +- **Automatic Cloning**: Repositories are cloned automatically to Local Repo Path if AI provider is used +- **Branch Management**: Creates feature branches for each work item +- **Sync Handling**: Keeps repositories updated with upstream changes + +#### Forked Repository Workflow + +1. **Fork** the target repository to your GitHub account +2. **Configure** both target and forked repositories in settings +3. **Automatic**: Tool manages branch creation and PR submission + +### Custom Instructions + +Add custom AI instructions in settings to guide processing: + +```AI Prompt +Focus on technical accuracy and clear documentation. +Ensure all code examples are properly formatted. +Include relevant cross-references to related topics. +``` + +## 🛠️ Troubleshooting + +### Common Issues + +**"No work items found"** + +- Verify Azure DevOps query URL is correct +- Check PAT has work item read permissions +- Ensure query returns results in Azure DevOps + +**"Repository not found"** + +- Verify GitHub repository name format (`owner/repo`) +- Check GitHub PAT has repository access +- Ensure repository exists and is accessible + +**"AI processing failed"** + +- Verify API key is correct and has credits +- Check internet connection +- Review processing log for specific errors + +**"Git operations failed"** + +- Ensure Git is installed and configured +- Check Local Repo Path exists and is writable +- Verify GitHub authentication is working + +### Performance Tips + +**Use Cache**: Let items load from cache for faster startup +**Dry Run**: Test changes before committing +**Batch Processing**: Process multiple items in sequence +**Monitor Logs**: Watch processing log for issues + +## Security Best Practices + +### Token Security + +- **Never commit** `.env` file to version control +- **Rotate tokens** regularly (90 days recommended) +- **Minimum permissions**: Use least privilege principle +- **Secure storage**: Store tokens in secure password manager + +### Repository Access + +- **Fork workflow**: Use personal forks for changes +- **Branch isolation**: Each work item gets separate branch +- **Review process**: All changes go through pull requests + +## Support + +### Getting Help + +1. **Check logs** in Processing Log tab for detailed errors +2. **Test connections** using Settings dialog test button +3. **Review configuration** for missing or incorrect values +4. **Consult documentation** for API-specific issues + +### Common Resources + +- [Azure DevOps PAT Documentation](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate) +- [GitHub PAT Documentation](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) +- [Git Configuration Guide](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup) diff --git a/application/app.py b/application/app.py new file mode 100644 index 0000000..2a658dd --- /dev/null +++ b/application/app.py @@ -0,0 +1,125 @@ +""" +MicrosoftDocFlow v3 +Main application entry point + +This application processes Azure DevOps work items and UUF items, +creating GitHub issues or pull requests with AI assistance. +""" + +import sys +import tkinter as tk +from tkinter import messagebox + +# Import our modular components +try: + from app_components.config_manager import ConfigManager + from app_components.ai_manager import AIManager + from app_components.github_api import GitHubAPI + from app_components.utils import Logger, PRNumberManager, ContentBuilders + from app_components.main_gui import MainGUI +except ImportError as e: + print(f"Error importing application components: {e}") + print("Make sure all files are present in the app_components folder") + sys.exit(1) + + +class AzureDevOpsToGitHubApp: + """Main application class that orchestrates all components""" + + def __init__(self): + """Initialize the application""" + self.root = tk.Tk() + self.root.title("MicrosoftDocFlow v3") + self.root.geometry("1400x1000") + + # Initialize core managers + self.config_manager = ConfigManager() + self.ai_manager = AIManager() + + # Load configuration + self.config = self.config_manager.load_configuration() + + # Initialize dry run state + dry_run_config = self.config.get('DRY_RUN', 'false') + self.dry_run_enabled = str(dry_run_config).lower() in ('true', '1', 'yes', 'on') + + # Initialize main GUI + self.main_gui = MainGUI( + root=self.root, + config_manager=self.config_manager, + ai_manager=self.ai_manager, + app=self + ) + + # Set up AI provider check after GUI is ready + self.root.after(100, self._check_ai_provider_setup) + + def _check_ai_provider_setup(self): + """Check and setup AI providers after GUI initialization""" + try: + ai_provider = self.config.get('AI_PROVIDER', '').strip().lower() + + if not ai_provider or ai_provider in ['none', '']: + return # No AI provider selected + + if ai_provider not in ['chatgpt', 'claude', 'anthropic', 'github-copilot', 'copilot', 'github_copilot']: + return # Unknown provider + + # Check if modules are available and offer installation if needed + self.ai_manager.check_and_install_ai_modules(ai_provider, self.root) + + except Exception as e: + print(f"Error checking AI provider setup: {e}") + + def get_config(self): + """Get current configuration""" + return self.config.copy() + + def update_config(self, new_config): + """Update configuration""" + self.config.update(new_config) + self.config_manager.config = self.config.copy() + + def save_config(self, config_values): + """Save configuration""" + success = self.config_manager.save_configuration(config_values) + if success: + self.config = self.config_manager.get_config() + # Update dry run state + dry_run_config = self.config.get('DRY_RUN', 'false') + self.dry_run_enabled = str(dry_run_config).lower() in ('true', '1', 'yes', 'on') + return success + + def create_github_api(self, token=None, dry_run=None): + """Create a GitHub API instance""" + if token is None: + token = self.config.get('GITHUB_PAT', '') + if dry_run is None: + dry_run = self.dry_run_enabled + + logger = self.main_gui.logger if hasattr(self.main_gui, 'logger') else None + return GitHubAPI(token, logger, dry_run) + + def run(self): + """Start the application""" + try: + self.root.mainloop() + except KeyboardInterrupt: + print("Application interrupted by user") + except Exception as e: + messagebox.showerror("Application Error", f"An unexpected error occurred:\n{str(e)}") + print(f"Application error: {e}") + + +def main(): + """Main entry point""" + try: + app = AzureDevOpsToGitHubApp() + app.run() + except Exception as e: + print(f"Failed to start application: {e}") + sys.exit(1) + + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/application/app_components/__init__.py b/application/app_components/__init__.py new file mode 100644 index 0000000..fe852c6 --- /dev/null +++ b/application/app_components/__init__.py @@ -0,0 +1,33 @@ +""" +Azure DevOps & UUF → GitHub Processor - Application Components +Modular components for the application +""" + +# Version info +__version__ = "3.0.0" +__author__ = "Azure DevOps to GitHub Processor" + +# Export main classes for easier imports +from .config_manager import ConfigManager +from .ai_manager import AIManager +from .github_api import GitHubAPI +from .azure_devops_api import AzureDevOpsAPI +from .dataverse_api import DataverseAPI +from .work_item_processor import WorkItemProcessor +from .settings_dialog import SettingsDialog +from .main_gui import MainGUI +from .utils import Logger, PRNumberManager, ContentBuilders + +__all__ = [ + 'ConfigManager', + 'AIManager', + 'GitHubAPI', + 'AzureDevOpsAPI', + 'DataverseAPI', + 'WorkItemProcessor', + 'SettingsDialog', + 'MainGUI', + 'Logger', + 'PRNumberManager', + 'ContentBuilders' +] \ No newline at end of file diff --git a/application/app_components/ai_manager.py b/application/app_components/ai_manager.py new file mode 100644 index 0000000..18a8cd3 --- /dev/null +++ b/application/app_components/ai_manager.py @@ -0,0 +1,3192 @@ +""" +AI Manager +Handles AI module availability checking, installation, and provider management +Includes AI provider implementations (Claude, ChatGPT) and git operations +""" + +import os +import shutil +import subprocess +import sys +import tempfile +import time +import tkinter as tk +from abc import ABC, abstractmethod +from pathlib import Path +from tkinter import messagebox +from typing import List, Tuple, Optional + + +class Logger: + """Simple logger interface""" + def __init__(self, log_func): + self.log = log_func + + +class AIProvider(ABC): + """Base class for AI providers""" + + def __init__(self, api_key: str, logger: Logger): + self.api_key = api_key + self.logger = logger + + @abstractmethod + def make_change(self, file_content: str, old_text: str, new_text: str, file_path: str, custom_instructions: str = None) -> Optional[str]: + """ + Use AI to make a change in the file content. + + Args: + file_content: Current content of the file + old_text: Text to find and replace + new_text: New text to replace with + file_path: Path to the file (for context) + custom_instructions: Optional custom instructions from user + + Returns: + Updated file content, or None if AI couldn't make the change + """ + pass + + +class ClaudeProvider(AIProvider): + """Claude AI provider using Anthropic API""" + + def make_change(self, file_content: str, old_text: str, new_text: str, file_path: str, custom_instructions: str = None) -> Optional[str]: + """Make smart, targeted changes based on reference text and suggestions + + Args: + file_content: Full file content + old_text: Reference text (what user is talking about - may not be exact) + new_text: Suggested changes (what user wants to see) + file_path: Path to the file being modified + custom_instructions: Optional custom instructions from user + """ + + # Step 1: Try direct string replacement if reference text is exact match + if old_text and old_text.strip() in file_content: + self.logger.log("✅ Making direct string replacement (reference text found exactly)") + updated_content = file_content.replace(old_text.strip(), new_text.strip()) + if updated_content != file_content: + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + self.logger.log(f"✅ Direct replacement successful ({changed_lines} lines changed)") + return updated_content + + # Step 2: Use AI to generate full document with targeted changes + self.logger.log("📝 Using AI to modify the document...") + return self._generate_updated_document(file_content, old_text, new_text, file_path, custom_instructions) + + def _generate_updated_document(self, file_content: str, old_text: str, new_text: str, file_path: str, custom_instructions: str = None) -> Optional[str]: + """Generate updated document content using Claude""" + + try: + import anthropic + client = anthropic.Anthropic(api_key=self.api_key) + + # Build custom instructions text + if custom_instructions and custom_instructions.strip(): + custom_instructions_text = f""" +**Additional Custom Instructions:** +{custom_instructions.strip()} + +""" + else: + custom_instructions_text = "" + + # Handle case where new_text is empty or just guidance + if new_text and new_text.strip() and not new_text.strip().lower().startswith(' [!IMPORTANT] +> OUTPUT REQUIREMENTS: +> - Return ONLY the complete file content - no explanatory text, dialog, or commentary +> - Do NOT add any text before or after the file content +> - Do NOT wrap output in markdown code blocks (```), just return the raw content +> - Return the ENTIRE document - no truncation, no placeholders like [Rest of the document here...] +> - Every single line of the original document must be present in your response +> - Preserve all markdown formatting, links, and code blocks exactly +> - Please ensure the changes align with Microsoft documentation standards +> - Only make changes that fulfill the specified request + +{custom_instructions_text} + +**Current File Content:** +``` +{file_content} +``` + +{guidance_text} + +Return the complete updated file content now (NO explanatory text):""" + + message = client.messages.create( + model="claude-3-5-haiku-20241022", + max_tokens=4096, + temperature=0.1, + messages=[{"role": "user", "content": prompt}] + ) + + updated_content = message.content[0].text.strip() + + # Basic validation - ensure content was actually changed + if updated_content and updated_content != file_content: + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + + self.logger.log(f"✅ Claude document update successful ({changed_lines} lines affected)") + return updated_content + else: + self.logger.log("⚠️ No changes detected in AI response") + return None + + except Exception as e: + self.logger.log(f"❌ Error generating updated document with Claude: {str(e)}") + return None + + def _generate_with_context_window_claude(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Use context window approach with Claude - AI only sees/modifies a small section + + This physically prevents AI from rewriting entire file by only giving it + the relevant section to work with. + """ + try: + import difflib + import anthropic + + # Step 1: Find where the reference text is located + lines = file_content.split('\n') + ref_lines = old_text.split('\n') if old_text else [] + + # Find best matching location for reference text + start_line = 0 + if ref_lines: + matcher = difflib.SequenceMatcher(None, ref_lines, lines) + match = matcher.find_longest_match(0, len(ref_lines), 0, len(lines)) + if match.size > 0: + start_line = match.b + self.logger.log(f"📍 Found reference area at line {start_line + 1}") + else: + self.logger.log("📍 Reference text not found, using beginning of file") + + # Step 2: Extract context window (30 lines before, 30 lines after) + window_before = 30 + window_after = 30 + + window_start = max(0, start_line - window_before) + window_end = min(len(lines), start_line + len(ref_lines) + window_after) + + context_window = lines[window_start:window_end] + self.logger.log(f"📄 Context window: lines {window_start + 1} to {window_end} ({len(context_window)} lines)") + self.logger.log(f" (AI can only modify this section, rest of file is protected)") + + # Step 3: Have AI modify only the context window + context_text = '\n'.join(context_window) + + client = anthropic.Anthropic(api_key=self.api_key) + + prompt = f"""You are helping modify a small section of a documentation file. You can ONLY modify the section provided below. + +File: {file_path} +Section location: Lines {window_start + 1} to {window_end} + +REFERENCE TEXT (what user is referring to): +{old_text} + +SUGGESTED CHANGES (what user wants): +{new_text} + +SECTION TO MODIFY: +``` +{context_text} +``` + +INSTRUCTIONS: +1. Understand the user's INTENT from the reference and suggestions: + - "add/include/incorporate a section" = Add a COMPLETE NEW SECTION with heading and full content + - "update/modify/change X" = Modify existing text X intelligently + - "fix/correct" = Make specific correction only + - Be generous with new content when asked to add something + +2. For ADDING content (sections, paragraphs, examples): + - Create complete, well-written content (not just stubs or brief additions) + - Add proper markdown headers (## Best Practices, ### Example, etc.) + - Place it logically (end of section, before ## Related content, etc.) + - Match the document's writing style and tone + +3. For MODIFYING content: + - Change only what's requested + - Leave everything else exactly as-is + +4. Return the ENTIRE section (all {len(context_window)} lines) with your changes +5. No explanations - just the modified section + +OUTPUT THE COMPLETE MODIFIED SECTION:""" + + message = client.messages.create( + model="claude-3-5-haiku-20241022", + max_tokens=4096, + temperature=0.1, + messages=[{"role": "user", "content": prompt}] + ) + + modified_window = message.content[0].text.strip() + + # Clean up code blocks if AI wrapped it + if modified_window.startswith('```'): + modified_window = '\n'.join(modified_window.split('\n')[1:-1]) + + # Step 4: Replace the context window in the full file + modified_lines = modified_window.split('\n') + result_lines = lines[:window_start] + modified_lines + lines[window_end:] + updated_content = '\n'.join(result_lines) + + # Verify change is minimal + diff = list(difflib.unified_diff(lines, result_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + + self.logger.log(f"✅ Context window approach successful ({changed_lines} lines changed)") + + # Ensure we actually made changes + if updated_content == file_content: + self.logger.log("⚠️ No changes detected, falling back to full-document approach") + return self._generate_updated_document(file_content, old_text, new_text, file_path) + + return updated_content + + except Exception as e: + self.logger.log(f"❌ Error with context window approach: {str(e)}") + self.logger.log("⚠️ Falling back to full-document approach") + return self._generate_updated_document(file_content, old_text, new_text, file_path) + + def _validate_diff_patch(self, diff_patch: str, original_content: str, old_text: str, new_text: str) -> bool: + """Validate that the AI-generated diff is safe and appropriate""" + try: + # Check for common problems + lines = diff_patch.split('\n') + + # Problem 0: Check for proper diff structure + has_hunk_header = any(line.startswith('@@') for line in lines) + if not has_hunk_header: + self.logger.log("❌ Invalid diff: Missing @@ hunk headers") + return False + + # Problem 1: Check for duplicate +++ lines + plus_count = sum(1 for line in lines if line.startswith('+++')) + if plus_count > 1: + self.logger.log("❌ Invalid diff: Multiple +++ lines detected") + return False + + # Problem 2: Check for removal of metadata (title, author, etc.) + for line in lines: + if line.startswith('-') and not line.startswith('---'): + removed_content = line[1:].strip() + # Check if removing metadata + if any(keyword in removed_content.lower() for keyword in ['title:', 'author:', 'description:', 'ms.author:', 'ms.date:']): + self.logger.log(f"❌ Invalid diff: Attempting to remove metadata: {removed_content}") + return False + + # Problem 3: Check if diff is too large (indicates rewrite) + removed_lines = len([line for line in lines if line.startswith('-') and not line.startswith('---')]) + added_lines = len([line for line in lines if line.startswith('+') and not line.startswith('+++')]) + + if removed_lines > 10: # Too many removals for an additive change + self.logger.log(f"❌ Invalid diff: Too many removals ({removed_lines} lines)") + return False + + return True + + except Exception as e: + self.logger.log(f"❌ Error validating diff: {str(e)}") + return False + + def _create_safe_diff(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Create a safer, simpler diff that just adds content without removing anything""" + try: + # Strategy: Find the best location to add the new content and insert it there + lines = file_content.split('\n') + + # Look for common insertion points for adding sections + insertion_point = self._find_safe_insertion_point(lines, old_text, new_text) + + if insertion_point is None: + self.logger.log("⚠️ Could not find safe insertion point") + return None + + # Insert the new content at the found location + new_lines = lines[:insertion_point] + [new_text.strip(), ''] + lines[insertion_point:] + updated_content = '\n'.join(new_lines) + + self.logger.log(f"✅ Created safe diff - inserting content at line {insertion_point}") + return updated_content + + except Exception as e: + self.logger.log(f"❌ Error creating safe diff: {str(e)}") + return None + + def _find_safe_insertion_point(self, lines: list, old_text: str, new_text: str) -> Optional[int]: + """Find the best place to insert new content safely""" + try: + # Look for section headers to insert after + for i, line in enumerate(lines): + # If the old_text contains context about where to insert + if old_text and old_text.lower().strip() in line.lower(): + # Insert after this line + return i + 1 + + # Look for pattern where we should insert a new section + # Insert before conclusion, examples, or other sections + if line.strip().startswith('##') and any(keyword in line.lower() for keyword in ['example', 'conclusion', 'summary', 'next steps']): + return i + + # If no specific location found, insert before the last section + for i in range(len(lines) - 1, -1, -1): + if lines[i].strip().startswith('##'): + return i + + # Last resort: insert at 80% through the document + return int(len(lines) * 0.8) + + except Exception: + return None + + def _apply_diff_patch(self, original_content: str, diff_patch: str, file_path: str) -> Optional[str]: + """Apply a unified diff patch to the original content""" + try: + import tempfile + import subprocess + import os + + # Create temporary files + with tempfile.TemporaryDirectory() as temp_dir: + # Write original content to temp file + original_file = os.path.join(temp_dir, "original.txt") + with open(original_file, 'w', encoding='utf-8') as f: + f.write(original_content) + + # Write diff patch to temp file + patch_file = os.path.join(temp_dir, "changes.patch") + with open(patch_file, 'w', encoding='utf-8') as f: + f.write(diff_patch) + + # Apply patch using git apply (more reliable than patch command) + try: + # First try git apply + subprocess.run(['git', 'apply', '--verbose', patch_file], + cwd=temp_dir, check=True, capture_output=True, text=True) + + # Read the result + with open(original_file, 'r', encoding='utf-8') as f: + return f.read() + + except subprocess.CalledProcessError: + # Fallback to manual patch application + self.logger.log("📝 Git apply failed, trying manual diff application...") + return self._manual_diff_apply(original_content, diff_patch) + + except Exception as e: + self.logger.log(f"⚠️ Patch application failed: {str(e)}") + return self._manual_diff_apply(original_content, diff_patch) + + def _manual_diff_apply(self, original_content: str, diff_patch: str) -> Optional[str]: + """Manually apply a diff patch when git apply fails""" + try: + # Detect original line ending style + has_crlf = '\r\n' in original_content + + original_lines = original_content.split('\n') + result_lines = original_lines.copy() + + # Parse the diff patch + diff_lines = diff_patch.split('\n') + current_original_line = 0 + + i = 0 + while i < len(diff_lines): + line = diff_lines[i] + + # Look for @@ headers + if line.startswith('@@'): + # Extract line numbers: @@ -start,count +start,count @@ + parts = line.split() + if len(parts) >= 3: + old_info = parts[1][1:] # Remove the - + if ',' in old_info: + start_line = int(old_info.split(',')[0]) - 1 # Convert to 0-based + else: + start_line = int(old_info) - 1 + + current_original_line = start_line + i += 1 + continue + + # Skip diff headers (must check before processing -/+ lines) + if line.startswith('---') or line.startswith('+++'): + i += 1 + continue + + # Process diff lines + if line.startswith('-'): + # Remove line + if current_original_line < len(result_lines): + del result_lines[current_original_line] + elif line.startswith('+'): + # Add line + new_line = line[1:] # Remove the + + # If original had CRLF and this line doesn't have \r, add it + if has_crlf and not new_line.endswith('\r'): + new_line = new_line + '\r' + result_lines.insert(current_original_line, new_line) + current_original_line += 1 + elif line.startswith(' '): + # Context line - advance + current_original_line += 1 + + i += 1 + + return '\n'.join(result_lines) + + except Exception as e: + self.logger.log(f"❌ Manual diff application failed: {str(e)}") + return None + + def _detect_change_type(self, old_text: str, new_text: str, file_path: str) -> str: + """Detect the type of change requested""" + old_lower = old_text.lower() + new_lower = new_text.lower() + + # Additive indicators + additive_keywords = ['add', 'include', 'incorporate', 'insert', 'create section', 'new section', 'best practices'] + if any(keyword in old_lower or keyword in new_lower for keyword in additive_keywords): + return "ADDITIVE" + + # Corrective indicators + corrective_keywords = ['correct', 'fix', 'grammar', 'spelling', 'typo', 'misspell', 'wrong', 'error'] + if any(keyword in old_lower or keyword in new_lower for keyword in corrective_keywords): + return "CORRECTIVE" + + # If new text is much longer than old text, likely additive + if len(new_text.strip()) > len(old_text.strip()) * 2: + return "ADDITIVE" + + # If similar length, likely corrective + if abs(len(new_text.strip()) - len(old_text.strip())) < 50: + return "CORRECTIVE" + + return "GENERAL" + + def _handle_additive_change(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Handle additive changes by generating content and inserting it""" + self.logger.log("🔨 Handling additive change - generating new content...") + + try: + import anthropic + client = anthropic.Anthropic(api_key=self.api_key) + + prompt = f"""**Instructions:** + +Task: Add new content to the documentation file as requested. + +Steps to complete: + +1. Generate ONLY the new content that should be added to the documentation file +2. Maintain proper formatting, indentation, and markdown structure +3. Make content standalone - don't reference existing content in the file +4. Use Microsoft documentation standards + +> [!IMPORTANT] +> Only create the new content - do not rewrite or modify existing content. +> Preserve markdown formatting, links, and code blocks as appropriate. +> Please ensure the changes align with Microsoft documentation standards. + +File: {file_path} +Request: {old_text} +Content to add: {new_text} + +Generate only the new content that should be added:""" + + message = client.messages.create( + model="claude-3-5-haiku-20241022", + max_tokens=2048, + temperature=0.1, + messages=[{"role": "user", "content": prompt}] + ) + + new_content = message.content[0].text.strip() + + # Find best insertion point in the file + insertion_point = self._find_insertion_point(file_content, old_text, file_path) + + # Insert the new content + lines = file_content.split('\n') + lines.insert(insertion_point, '\n' + new_content + '\n') + updated_content = '\n'.join(lines) + + # Count actual changes + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+')]) + + self.logger.log(f"✅ Added new content ({changed_lines} lines added)") + return updated_content + + except Exception as e: + self.logger.log(f"❌ Error in additive change: {str(e)}") + return None + + def _handle_corrective_change(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Handle corrective changes by finding and fixing specific issues""" + self.logger.log("🔍 Handling corrective change - finding specific issues...") + + try: + import anthropic + client = anthropic.Anthropic(api_key=self.api_key) + + prompt = f"""**Instructions:** + +Task: Find and fix a specific issue in the documentation file. + +Steps to complete: + +1. Locate the exact text that needs to be corrected in the file +2. Provide the precise replacement text +3. Make minimal changes - fix only what needs to be fixed +4. Maintain existing formatting and structure + +> [!IMPORTANT] +> Only make the specified correction - do not make additional changes. +> Preserve all markdown formatting, links, and code blocks. +> Please ensure the changes align with Microsoft documentation standards. + +Issue: {old_text} +Fix: {new_text} +File: {file_path} + +Return your response in this format: +OLD: [exact text to find] +NEW: [exact replacement text] + +Be very specific - find the minimal text that needs changing. For example: +- If fixing "Microsft" → return OLD: Microsft, NEW: Microsoft +- If fixing grammar → return OLD: [the incorrect phrase], NEW: [corrected phrase] + +Find the exact text to correct:""" + + message = client.messages.create( + model="claude-3-5-haiku-20241022", + max_tokens=1024, + temperature=0.1, + messages=[{"role": "user", "content": f"{prompt}\n\nFile content to search:\n{file_content}"}] + ) + + response = message.content[0].text.strip() + + # Parse the response to extract OLD and NEW + old_match = None + new_match = None + + for line in response.split('\n'): + if line.startswith('OLD:'): + old_match = line[4:].strip() + elif line.startswith('NEW:'): + new_match = line[4:].strip() + + if old_match and new_match and old_match in file_content: + updated_content = file_content.replace(old_match, new_match) + # Count changes + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + self.logger.log(f"✅ Corrective change successful ({changed_lines} lines affected)") + return updated_content + else: + self.logger.log(f"⚠️ Could not find exact text to correct") + return None + + except Exception as e: + self.logger.log(f"❌ Error in corrective change: {str(e)}") + return None + + def _find_insertion_point(self, file_content: str, context: str, file_path: str) -> int: + """Find the best place to insert new content""" + lines = file_content.split('\n') + + # For markdown files, try to find a good section to add after + if file_path.endswith('.md'): + # Look for existing sections + for i, line in enumerate(lines): + if line.startswith('#') and i < len(lines) - 1: + # Insert after this section + continue + + # If no good sections found, add at the end + return len(lines) + + # For other files, add at the end + return len(lines) + + def _handle_general_change(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Handle general changes with enhanced targeting""" + self.logger.log("🎯 Handling general change with enhanced targeting...") + + max_retries = 3 + base_delay = 2 + + for attempt in range(max_retries): + try: + import anthropic + client = anthropic.Anthropic(api_key=self.api_key) + + prompt = f"""**Instructions:** + +Task: Update the documentation file with the specific change requested. + +Steps to complete: + +1. Locate the specific section that needs changing in the file +2. Make ONLY the requested change +3. Maintain the existing formatting, indentation, and markdown structure +4. Preserve everything else exactly as-is +5. Return the complete updated file + +> [!IMPORTANT] +> Only make the specified change - do not rewrite or reorganize content. +> Preserve all markdown formatting, links, and code blocks. +> Please ensure the changes align with Microsoft documentation standards. +> Make the SMALLEST possible change. + +File: {file_path} +Change needed: {old_text} +New content: {new_text} + +Current file content: +{file_content}""" + + message = client.messages.create( + model="claude-3-5-haiku-20241022", + max_tokens=4096, + temperature=0.1, + messages=[{"role": "user", "content": prompt}] + ) + + updated_content = message.content[0].text + + if new_text.strip() in updated_content: + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + + if changed_lines > 30: + self.logger.log(f"⚠️ Change affected {changed_lines} lines - may be too broad") + else: + self.logger.log(f"✅ General change successful ({changed_lines} lines affected)") + + return updated_content + else: + self.logger.log("⚠️ New text not found in result") + return None + + except Exception as e: + if attempt < max_retries - 1: + delay = base_delay * (2 ** attempt) + self.logger.log(f"⚠️ Retry {attempt + 1}/{max_retries} after {delay}s...") + time.sleep(delay) + continue + else: + self.logger.log(f"❌ Error in general change: {str(e)}") + return None + + return None + + +class ChatGPTProvider(AIProvider): + """ChatGPT/GPT-4 provider using OpenAI API""" + + def make_change(self, file_content: str, old_text: str, new_text: str, file_path: str, custom_instructions: str = None) -> Optional[str]: + """Make smart, targeted changes based on reference text and suggestions + + Args: + file_content: Full file content + old_text: Reference text (what user is talking about - may not be exact) + new_text: Suggested changes (what user wants to see) + file_path: Path to file being modified + custom_instructions: Optional custom instructions from user + """ + + # Step 1: Try direct string replacement if reference text is exact match + if old_text and old_text.strip() in file_content: + self.logger.log("✅ Making direct string replacement (reference text found exactly)") + updated_content = file_content.replace(old_text.strip(), new_text.strip()) + if updated_content != file_content: + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + self.logger.log(f"✅ Direct replacement successful ({changed_lines} lines changed)") + return updated_content + + # Step 2: Use AI to generate full document with targeted changes + self.logger.log("📝 Using AI to modify the document...") + return self._generate_updated_document_chatgpt(file_content, old_text, new_text, file_path, custom_instructions) + + def _generate_updated_document_chatgpt(self, file_content: str, old_text: str, new_text: str, file_path: str, custom_instructions: str = None) -> Optional[str]: + """Generate updated document content using ChatGPT""" + + try: + import openai + client = openai.OpenAI(api_key=self.api_key) + + # Build custom instructions text + if custom_instructions and custom_instructions.strip(): + custom_instructions_text = f""" + +**Additional Custom Instructions:** +{custom_instructions.strip()} + +""" + else: + custom_instructions_text = "" + + # Handle blank new_text field with dynamic prompt + if not new_text or not new_text.strip(): + # General improvement request when new text is blank + prompt = f"""**Instructions:** + +Task: Review and improve the documentation file based on the reference context provided. + +Steps to complete: + +1. Review the current file content below +2. Look at the reference context: "{old_text}" +3. Improve the relevant sections based on Microsoft documentation standards +4. Maintain the existing formatting, indentation, and markdown structure +5. Return the complete updated file content + +> [!IMPORTANT] +> OUTPUT REQUIREMENTS: +> - Return ONLY the complete file content - no explanatory text, dialog, or commentary +> - Do NOT add any text before or after the file content +> - Do NOT wrap output in markdown code blocks (```), just return the raw content +> - Return the ENTIRE document - no truncation, no placeholders like [Rest of the document here...] +> - Every single line of the original document must be present in your response +> - Focus on areas related to: {old_text} +> - Preserve all markdown formatting, links, and code blocks exactly +> - Please ensure improvements align with Microsoft documentation standards +> - Only make improvements - do not remove existing content unless it's redundant + +{custom_instructions_text} + +**Current File Content:** +``` +{file_content} +``` + +**Context for improvements:** +``` +{old_text} +``` + +Return the complete updated file content now (NO explanatory text):""" + + else: + # Specific replacement when new text is provided + prompt = f"""**Instructions:** + +Task: Update the documentation file with the changes requested. + +Steps to complete: + +1. Review the current file content below +2. Find the reference text that needs to be updated +3. Replace it with the suggested new content +4. Maintain the existing formatting, indentation, and markdown structure +5. Return the complete updated file content + +> [!IMPORTANT] +> OUTPUT REQUIREMENTS: +> - Return ONLY the complete file content - no explanatory text, dialog, or commentary +> - Do NOT add any text before or after the file content +> - Do NOT wrap output in markdown code blocks (```), just return the raw content +> - Return the ENTIRE document - no truncation, no placeholders like [Rest of the document here...] +> - Every single line of the original document must be present in your response +> - Only replace the specified text - do not make additional changes +> - Preserve all markdown formatting, links, and code blocks exactly +> - If the current text cannot be found exactly, search for similar text +> - Please ensure the changes align with Microsoft documentation standards +> - Do not remove any text unless the reference or suggested guidance indicates to do so + +{custom_instructions_text} + +**Current File Content:** +``` +{file_content} +``` + +**Reference text to find and replace:** +``` +{old_text} +``` + +**Suggested new content:** +``` +{new_text} +``` + +Return the complete updated file content now (NO explanatory text):""" + + response = client.chat.completions.create( + model="gpt-4-turbo-preview", + messages=[ + {"role": "system", "content": "You are a document editor. Return ONLY the complete updated file content - no explanatory text, no dialog, no code blocks, no truncation, no placeholders. Output must be the raw complete file content with requested changes."}, + {"role": "user", "content": prompt} + ], + temperature=0.1 + ) + + updated_content = response.choices[0].message.content.strip() + + # Clean up code blocks if AI wrapped it + if updated_content.startswith('```'): + updated_content = '\n'.join(updated_content.split('\n')[1:-1]) + + # Basic validation - ensure content was actually changed + if updated_content and updated_content != file_content: + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + + self.logger.log(f"✅ ChatGPT document update successful ({changed_lines} lines affected)") + return updated_content + else: + self.logger.log("⚠️ No changes detected in AI response") + return None + + except Exception as e: + self.logger.log(f"❌ Error generating updated document with ChatGPT: {str(e)}") + return None + + def _generate_with_context_window(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Use context window approach - AI only sees/modifies a small section + + This physically prevents AI from rewriting entire file by only giving it + the relevant section to work with. + + Args: + file_content: Full file content + old_text: Reference text (guides where to look) + new_text: Suggestions (what to change to) + """ + try: + import difflib + + # Step 1: Find where the reference text is located + lines = file_content.split('\n') + ref_lines = old_text.split('\n') if old_text else [] + + # Find best matching location for reference text + start_line = 0 + if ref_lines: + matcher = difflib.SequenceMatcher(None, ref_lines, lines) + match = matcher.find_longest_match(0, len(ref_lines), 0, len(lines)) + if match.size > 0: + start_line = match.b + self.logger.log(f"📍 Found reference area at line {start_line + 1}") + else: + self.logger.log("📍 Reference text not found, using beginning of file") + + # Step 2: Extract context window (30 lines before, 30 lines after) + window_before = 30 + window_after = 30 + + window_start = max(0, start_line - window_before) + window_end = min(len(lines), start_line + len(ref_lines) + window_after) + + context_window = lines[window_start:window_end] + self.logger.log(f"📄 Context window: lines {window_start + 1} to {window_end} ({len(context_window)} lines)") + self.logger.log(f" (AI can only modify this section, rest of file is protected)") + + # Step 3: Have AI modify only the context window + context_text = '\n'.join(context_window) + + import openai + client = openai.OpenAI(api_key=self.api_key) + + prompt = f"""You are helping modify a small section of a documentation file. You can ONLY modify the section provided below. + +File: {file_path} +Section location: Lines {window_start + 1} to {window_end} + +REFERENCE TEXT (what user is referring to): +{old_text} + +SUGGESTED CHANGES (what user wants): +{new_text} + +SECTION TO MODIFY: +``` +{context_text} +``` + +INSTRUCTIONS: +1. Understand the user's INTENT from the reference and suggestions: + - "add/include/incorporate a section" = Add a COMPLETE NEW SECTION with heading and full content + - "update/modify/change X" = Modify existing text X intelligently + - "fix/correct" = Make specific correction only + - Be generous with new content when asked to add something + +2. For ADDING content (sections, paragraphs, examples): + - Create complete, well-written content (not just stubs or brief additions) + - Add proper markdown headers (## Best Practices, ### Example, etc.) + - Place it logically (end of section, before ## Related content, etc.) + - Match the document's writing style and tone + +3. For MODIFYING content: + - Change only what's requested + - Leave everything else exactly as-is + +4. Return the ENTIRE section (all {len(context_window)} lines) with your changes +5. No explanations - just the modified section + +OUTPUT THE COMPLETE MODIFIED SECTION:""" + + response = client.chat.completions.create( + model="gpt-4-turbo-preview", + messages=[ + {"role": "system", "content": "You make precise, targeted edits to documentation sections. Return only the modified text, nothing else."}, + {"role": "user", "content": prompt} + ], + temperature=0.1 + ) + + modified_window = response.choices[0].message.content.strip() + + # Clean up code blocks if AI wrapped it + if modified_window.startswith('```'): + modified_window = '\n'.join(modified_window.split('\n')[1:-1]) + + # Step 4: Replace the context window in the full file + modified_lines = modified_window.split('\n') + result_lines = lines[:window_start] + modified_lines + lines[window_end:] + updated_content = '\n'.join(result_lines) + + # Verify change is minimal + diff = list(difflib.unified_diff(lines, result_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + + self.logger.log(f"✅ Context window approach successful ({changed_lines} lines changed)") + + # Ensure we actually made changes + if updated_content == file_content: + self.logger.log("⚠️ No changes detected, falling back to full-document approach") + return self._generate_updated_document_chatgpt(file_content, old_text, new_text, file_path) + + return updated_content + + except Exception as e: + self.logger.log(f"❌ Error with context window approach: {str(e)}") + self.logger.log("⚠️ Falling back to full-document approach") + return self._generate_updated_document_chatgpt(file_content, old_text, new_text, file_path) + + def _validate_diff_patch(self, diff_patch: str, original_content: str, old_text: str, new_text: str) -> bool: + """Validate that the AI-generated diff is safe and appropriate""" + try: + # Check for common problems + lines = diff_patch.split('\n') + + # Problem 0: Check for proper diff structure + has_hunk_header = any(line.startswith('@@') for line in lines) + if not has_hunk_header: + self.logger.log("❌ Invalid diff: Missing @@ hunk headers") + return False + + # Problem 1: Check for duplicate +++ lines + plus_count = sum(1 for line in lines if line.startswith('+++')) + if plus_count > 1: + self.logger.log("❌ Invalid diff: Multiple +++ lines detected") + return False + + # Problem 2: Check for removal of metadata (title, author, etc.) + for line in lines: + if line.startswith('-') and not line.startswith('---'): + removed_content = line[1:].strip() + # Check if removing metadata + if any(keyword in removed_content.lower() for keyword in ['title:', 'author:', 'description:', 'ms.author:', 'ms.date:']): + self.logger.log(f"❌ Invalid diff: Attempting to remove metadata: {removed_content}") + return False + + # Problem 3: Check if diff is too large (indicates rewrite) + removed_lines = len([line for line in lines if line.startswith('-') and not line.startswith('---')]) + added_lines = len([line for line in lines if line.startswith('+') and not line.startswith('+++')]) + + if removed_lines > 10: # Too many removals for an additive change + self.logger.log(f"❌ Invalid diff: Too many removals ({removed_lines} lines)") + return False + + return True + + except Exception as e: + self.logger.log(f"❌ Error validating diff: {str(e)}") + return False + + def _create_safe_diff(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Create a safer, simpler diff that just adds content without removing anything""" + try: + # Strategy: Find the best location to add the new content and insert it there + lines = file_content.split('\n') + + # Look for common insertion points for adding sections + insertion_point = self._find_safe_insertion_point(lines, old_text, new_text) + + if insertion_point is None: + self.logger.log("⚠️ Could not find safe insertion point") + return None + + # Insert the new content at the found location + new_lines = lines[:insertion_point] + [new_text.strip(), ''] + lines[insertion_point:] + updated_content = '\n'.join(new_lines) + + self.logger.log(f"✅ Created safe diff - inserting content at line {insertion_point}") + return updated_content + + except Exception as e: + self.logger.log(f"❌ Error creating safe diff: {str(e)}") + return None + + def _find_safe_insertion_point(self, lines: list, old_text: str, new_text: str) -> Optional[int]: + """Find the best place to insert new content safely""" + try: + # Look for section headers to insert after + for i, line in enumerate(lines): + # If the old_text contains context about where to insert + if old_text and old_text.lower().strip() in line.lower(): + # Insert after this line + return i + 1 + + # Look for pattern where we should insert a new section + # Insert before conclusion, examples, or other sections + if line.strip().startswith('##') and any(keyword in line.lower() for keyword in ['example', 'conclusion', 'summary', 'next steps']): + return i + + # If no specific location found, insert before the last section + for i in range(len(lines) - 1, -1, -1): + if lines[i].strip().startswith('##'): + return i + + # Last resort: insert at 80% through the document + return int(len(lines) * 0.8) + + except Exception: + return None + + def _apply_diff_patch_chatgpt(self, original_content: str, diff_patch: str, file_path: str) -> Optional[str]: + """Apply a unified diff patch to the original content""" + try: + import tempfile + import subprocess + import os + + # Create temporary files + with tempfile.TemporaryDirectory() as temp_dir: + # Write original content to temp file + original_file = os.path.join(temp_dir, "original.txt") + with open(original_file, 'w', encoding='utf-8') as f: + f.write(original_content) + + # Write diff patch to temp file + patch_file = os.path.join(temp_dir, "changes.patch") + with open(patch_file, 'w', encoding='utf-8') as f: + f.write(diff_patch) + + # Apply patch using git apply + try: + subprocess.run(['git', 'apply', '--verbose', patch_file], + cwd=temp_dir, check=True, capture_output=True, text=True) + + # Read the result + with open(original_file, 'r', encoding='utf-8') as f: + return f.read() + + except subprocess.CalledProcessError: + # Fallback to manual patch application + self.logger.log("📝 Git apply failed, trying manual diff application...") + return self._manual_diff_apply_chatgpt(original_content, diff_patch) + + except Exception as e: + self.logger.log(f"⚠️ ChatGPT patch application failed: {str(e)}") + return self._manual_diff_apply_chatgpt(original_content, diff_patch) + + def _manual_diff_apply_chatgpt(self, original_content: str, diff_patch: str) -> Optional[str]: + """Manually apply a diff patch when git apply fails""" + try: + # Detect original line ending style + has_crlf = '\r\n' in original_content + + original_lines = original_content.split('\n') + result_lines = original_lines.copy() + + # Parse the diff patch + diff_lines = diff_patch.split('\n') + current_original_line = 0 + + i = 0 + while i < len(diff_lines): + line = diff_lines[i] + + # Look for @@ headers + if line.startswith('@@'): + # Extract line numbers: @@ -start,count +start,count @@ + parts = line.split() + if len(parts) >= 3: + old_info = parts[1][1:] # Remove the - + if ',' in old_info: + start_line = int(old_info.split(',')[0]) - 1 # Convert to 0-based + else: + start_line = int(old_info) - 1 + + current_original_line = start_line + i += 1 + continue + + # Skip diff headers (must check before processing -/+ lines) + if line.startswith('---') or line.startswith('+++'): + i += 1 + continue + + # Process diff lines + if line.startswith('-'): + # Remove line + if current_original_line < len(result_lines): + del result_lines[current_original_line] + elif line.startswith('+'): + # Add line + new_line = line[1:] # Remove the + + # If original had CRLF and this line doesn't have \r, add it + if has_crlf and not new_line.endswith('\r'): + new_line = new_line + '\r' + result_lines.insert(current_original_line, new_line) + current_original_line += 1 + elif line.startswith(' '): + # Context line - advance + current_original_line += 1 + + i += 1 + + return '\n'.join(result_lines) + + except Exception as e: + self.logger.log(f"❌ ChatGPT manual diff application failed: {str(e)}") + return None + + def _detect_change_type(self, old_text: str, new_text: str, file_path: str) -> str: + """Detect the type of change requested""" + old_lower = old_text.lower() + new_lower = new_text.lower() + + # Additive indicators + additive_keywords = ['add', 'include', 'incorporate', 'insert', 'create section', 'new section', 'best practices'] + if any(keyword in old_lower or keyword in new_lower for keyword in additive_keywords): + return "ADDITIVE" + + # Corrective indicators + corrective_keywords = ['correct', 'fix', 'grammar', 'spelling', 'typo', 'misspell', 'wrong', 'error'] + if any(keyword in old_lower or keyword in new_lower for keyword in corrective_keywords): + return "CORRECTIVE" + + # If new text is much longer than old text, likely additive + if len(new_text.strip()) > len(old_text.strip()) * 2: + return "ADDITIVE" + + # If similar length, likely corrective + if abs(len(new_text.strip()) - len(old_text.strip())) < 50: + return "CORRECTIVE" + + return "GENERAL" + + def _handle_additive_change_chatgpt(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Handle additive changes using ChatGPT""" + self.logger.log("🔨 ChatGPT handling additive change - generating new content...") + + try: + import openai + client = openai.OpenAI(api_key=self.api_key) + + prompt = f"""**Instructions:** + +Task: Add new content to the documentation file as requested. + +Steps to complete: + +1. Generate ONLY the new content that should be added to the documentation file +2. Maintain proper formatting, indentation, and markdown structure +3. Make content standalone - don't reference existing content in the file +4. Use Microsoft documentation standards + +> [!IMPORTANT] +> Only create the new content - do not rewrite or modify existing content. +> Preserve markdown formatting, links, and code blocks as appropriate. +> Please ensure the changes align with Microsoft documentation standards. + +File: {file_path} +Request: {old_text} +Content to add: {new_text} + +Generate only the new content that should be added:""" + + response = client.chat.completions.create( + model="gpt-4-turbo-preview", + messages=[ + {"role": "system", "content": "You are a content generator. Generate only new content, never rewrite existing content."}, + {"role": "user", "content": prompt} + ], + temperature=0.1 + ) + + new_content = response.choices[0].message.content.strip() + + # Find best insertion point and insert + insertion_point = self._find_insertion_point(file_content, old_text, file_path) + lines = file_content.split('\n') + lines.insert(insertion_point, '\n' + new_content + '\n') + updated_content = '\n'.join(lines) + + # Count actual changes + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+')]) + + self.logger.log(f"✅ ChatGPT added new content ({changed_lines} lines added)") + return updated_content + + except Exception as e: + self.logger.log(f"❌ Error in ChatGPT additive change: {str(e)}") + return None + + def _handle_corrective_change_chatgpt(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Handle corrective changes using ChatGPT""" + self.logger.log("🔍 ChatGPT handling corrective change - finding specific issues...") + + try: + import openai + client = openai.OpenAI(api_key=self.api_key) + + prompt = f"""**Instructions:** + +Task: Find and fix a specific issue in the documentation file. + +Steps to complete: + +1. Locate the exact text that needs to be corrected in the file +2. Provide the precise replacement text +3. Make minimal changes - fix only what needs to be fixed +4. Maintain existing formatting and structure + +> [!IMPORTANT] +> Only make the specified correction - do not make additional changes. +> Preserve all markdown formatting, links, and code blocks. +> Please ensure the changes align with Microsoft documentation standards. + +Issue: {old_text} +Fix: {new_text} +File: {file_path} + +Return your response in this format: +OLD: [exact text to find] +NEW: [exact replacement text] + +Be very specific - find the minimal text that needs changing. For example: +- If fixing "Microsft" → return OLD: Microsft, NEW: Microsoft +- If fixing grammar → return OLD: [the incorrect phrase], NEW: [corrected phrase] + +File content to search: +{file_content} + +Find the exact text to correct:""" + + response = client.chat.completions.create( + model="gpt-4-turbo-preview", + messages=[ + {"role": "system", "content": "You are a precise error detector. Find exact text that needs correction."}, + {"role": "user", "content": prompt} + ], + temperature=0.1 + ) + + response_text = response.choices[0].message.content.strip() + + # Parse OLD and NEW + old_match = None + new_match = None + + for line in response_text.split('\n'): + if line.startswith('OLD:'): + old_match = line[4:].strip() + elif line.startswith('NEW:'): + new_match = line[4:].strip() + + if old_match and new_match and old_match in file_content: + updated_content = file_content.replace(old_match, new_match) + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + self.logger.log(f"✅ ChatGPT corrective change successful ({changed_lines} lines affected)") + return updated_content + else: + self.logger.log(f"⚠️ ChatGPT could not find exact text to correct") + return None + + except Exception as e: + self.logger.log(f"❌ Error in ChatGPT corrective change: {str(e)}") + return None + + def _handle_general_change_chatgpt(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Handle general changes using ChatGPT with enhanced targeting""" + self.logger.log("🎯 ChatGPT handling general change with enhanced targeting...") + + max_retries = 3 + base_delay = 2 + + for attempt in range(max_retries): + try: + import openai + client = openai.OpenAI(api_key=self.api_key) + + prompt = f"""You are helping make a specific text change in a documentation file. + +File: {file_path} +Change needed: {old_text} +New content: {new_text} + +CRITICAL: Make the SMALLEST possible change. Do not rewrite or reorganize content. + +Your task: +1. Find the specific section that needs changing +2. Make ONLY that change +3. Preserve everything else exactly as-is +4. Return the complete updated file + +Current file content: +{file_content}""" + + response = client.chat.completions.create( + model="gpt-4-turbo-preview", + messages=[ + {"role": "system", "content": "You are a precise file editor. Make minimal targeted changes only."}, + {"role": "user", "content": prompt} + ], + temperature=0.1 + ) + + updated_content = response.choices[0].message.content + + if new_text.strip() in updated_content: + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + + if changed_lines > 30: + self.logger.log(f"⚠️ ChatGPT change affected {changed_lines} lines - may be too broad") + else: + self.logger.log(f"✅ ChatGPT general change successful ({changed_lines} lines affected)") + + return updated_content + else: + self.logger.log("⚠️ ChatGPT: New text not found in result") + return None + + except Exception as e: + if attempt < max_retries - 1: + delay = base_delay * (2 ** attempt) + self.logger.log(f"⚠️ ChatGPT retry {attempt + 1}/{max_retries} after {delay}s...") + time.sleep(delay) + continue + else: + self.logger.log(f"❌ Error in ChatGPT general change: {str(e)}") + return None + + return None + + def _find_insertion_point(self, file_content: str, context: str, file_path: str) -> int: + """Find the best place to insert new content""" + lines = file_content.split('\n') + + # For markdown files, try to find a good section to add after + if file_path.endswith('.md'): + # Look for existing sections + for i, line in enumerate(lines): + if line.startswith('#') and i < len(lines) - 1: + # Insert after this section + continue + + # If no good sections found, add at the end + return len(lines) + + # For other files, add at the end + return len(lines) + + +class GitHubCopilotProvider(AIProvider): + """GitHub Copilot provider using GitHub Models API""" + + def make_change(self, file_content: str, old_text: str, new_text: str, file_path: str, custom_instructions: str = None) -> Optional[str]: + """Use diff-based approach for surgical edits""" + + # Step 1: Always try direct string replacement first (most accurate) + if old_text and old_text.strip() in file_content: + self.logger.log("✅ Making direct string replacement (most precise)") + updated_content = file_content.replace(old_text.strip(), new_text.strip()) + if updated_content != file_content: + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + self.logger.log(f"✅ Direct replacement successful ({changed_lines} lines changed)") + return updated_content + + # Step 2: Use AI to generate full document with targeted changes + self.logger.log("📝 Using GitHub Copilot to modify the document...") + return self._generate_updated_document_copilot(file_content, old_text, new_text, file_path, custom_instructions) + + def _generate_updated_document_copilot(self, file_content: str, old_text: str, new_text: str, file_path: str, custom_instructions: str = None) -> Optional[str]: + """Generate updated document content using GitHub Copilot""" + + try: + import requests + + url = "https://models.inference.ai.azure.com/chat/completions" + headers = { + "Authorization": f"Bearer {self.api_key}", + "Content-Type": "application/json", + } + + # Build custom instructions text + if custom_instructions and custom_instructions.strip(): + custom_instructions_text = f""" + +**Additional Custom Instructions:** +{custom_instructions.strip()} + +""" + else: + custom_instructions_text = "" + + # Handle blank new_text field with dynamic prompt + if not new_text or not new_text.strip(): + # General improvement request when new text is blank + prompt = f"""**Instructions:** + +Task: Review and improve the documentation file based on the reference context provided. + +Steps to complete: + +1. Review the current file content below +2. Look at the reference context: "{old_text}" +3. Improve the relevant sections based on Microsoft documentation standards +4. Maintain the existing formatting, indentation, and markdown structure +5. Return the complete updated file content + +> [!IMPORTANT] +> OUTPUT REQUIREMENTS: +> - Return ONLY the complete file content - no explanatory text, dialog, or commentary +> - Do NOT add any text before or after the file content +> - Do NOT wrap output in markdown code blocks (```), just return the raw content +> - Return the ENTIRE document - no truncation, no placeholders like [Rest of the document here...] +> - Every single line of the original document must be present in your response +> - Focus on areas related to: {old_text} +> - Preserve all markdown formatting, links, and code blocks exactly +> - Please ensure improvements align with Microsoft documentation standards +> - Only make improvements - do not remove existing content unless it's redundant + +{custom_instructions_text} + +**Current File Content:** +``` +{file_content} +``` + +**Context for improvements:** +``` +{old_text} +``` + +Return the complete updated file content now (NO explanatory text):""" + + else: + # Specific replacement when new text is provided + prompt = f"""**Instructions:** + +Task: Update the documentation file with the changes requested. + +Steps to complete: + +1. Review the current file content below +2. Find the reference text that needs to be updated +3. Replace it with the suggested new content +4. Maintain the existing formatting, indentation, and markdown structure +5. Return the complete updated file content + +> [!IMPORTANT] +> OUTPUT REQUIREMENTS: +> - Return ONLY the complete file content - no explanatory text, dialog, or commentary +> - Do NOT add any text before or after the file content +> - Do NOT wrap output in markdown code blocks (```), just return the raw content +> - Return the ENTIRE document - no truncation, no placeholders like [Rest of the document here...] +> - Every single line of the original document must be present in your response +> - Only replace the specified text - do not make additional changes +> - Preserve all markdown formatting, links, and code blocks exactly +> - If the current text cannot be found exactly, search for similar text +> - Please ensure the changes align with Microsoft documentation standards +> - Do not remove any text unless the reference or suggested guidance indicates to do so + +{custom_instructions_text} + +**Current File Content:** +``` +{file_content} +``` + +**Reference text to find and replace:** +``` +{old_text} +``` + +**Suggested new content:** +``` +{new_text} +``` + +Return the complete updated file content now (NO explanatory text):""" + + data = { + "messages": [ + {"role": "system", "content": "You are a document editor. Return ONLY the complete updated file content - no explanatory text, no dialog, no code blocks, no truncation, no placeholders. Output must be the raw complete file content with requested changes."}, + {"role": "user", "content": prompt} + ], + "model": "gpt-4o", + "temperature": 0.1, + "max_tokens": 4096 + } + + response = requests.post(url, headers=headers, json=data, timeout=60) + response.raise_for_status() + + result = response.json() + updated_content = result['choices'][0]['message']['content'].strip() + + # Clean up code blocks if AI wrapped it + if updated_content.startswith('```'): + updated_content = '\n'.join(updated_content.split('\n')[1:-1]) + + # Basic validation - ensure content was actually changed + if updated_content and updated_content != file_content: + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + + self.logger.log(f"✅ GitHub Copilot document update successful ({changed_lines} lines affected)") + return updated_content + else: + self.logger.log("⚠️ No changes detected in AI response") + return None + + except Exception as e: + self.logger.log(f"❌ Error generating updated document with GitHub Copilot: {str(e)}") + return None + + def _validate_diff_patch(self, diff_patch: str, original_content: str, old_text: str, new_text: str) -> bool: + """Validate that the AI-generated diff is safe and appropriate""" + try: + # Check for common problems + lines = diff_patch.split('\n') + + # Problem 0: Check for proper diff structure + has_hunk_header = any(line.startswith('@@') for line in lines) + if not has_hunk_header: + self.logger.log("❌ Invalid diff: Missing @@ hunk headers") + return False + + # Problem 1: Check for duplicate +++ lines + plus_count = sum(1 for line in lines if line.startswith('+++')) + if plus_count > 1: + self.logger.log("❌ Invalid diff: Multiple +++ lines detected") + return False + + # Problem 2: Check for removal of metadata (title, author, etc.) + for line in lines: + if line.startswith('-') and not line.startswith('---'): + removed_content = line[1:].strip() + # Check if removing metadata + if any(keyword in removed_content.lower() for keyword in ['title:', 'author:', 'description:', 'ms.author:', 'ms.date:']): + self.logger.log(f"❌ Invalid diff: Attempting to remove metadata: {removed_content}") + return False + + # Problem 3: Check if diff is too large (indicates rewrite) + removed_lines = len([line for line in lines if line.startswith('-') and not line.startswith('---')]) + added_lines = len([line for line in lines if line.startswith('+') and not line.startswith('+++')]) + + if removed_lines > 10: # Too many removals for an additive change + self.logger.log(f"❌ Invalid diff: Too many removals ({removed_lines} lines)") + return False + + return True + + except Exception as e: + self.logger.log(f"❌ Error validating diff: {str(e)}") + return False + + def _create_safe_diff(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Create a safer, simpler diff that just adds content without removing anything""" + try: + # Strategy: Find the best location to add the new content and insert it there + lines = file_content.split('\n') + + # Look for common insertion points for adding sections + insertion_point = self._find_safe_insertion_point(lines, old_text, new_text) + + if insertion_point is None: + self.logger.log("⚠️ Could not find safe insertion point") + return None + + # Insert the new content at the found location + new_lines = lines[:insertion_point] + [new_text.strip(), ''] + lines[insertion_point:] + updated_content = '\n'.join(new_lines) + + self.logger.log(f"✅ Created safe diff - inserting content at line {insertion_point}") + return updated_content + + except Exception as e: + self.logger.log(f"❌ Error creating safe diff: {str(e)}") + return None + + def _find_safe_insertion_point(self, lines: list, old_text: str, new_text: str) -> Optional[int]: + """Find the best place to insert new content safely""" + try: + # Look for section headers to insert after + for i, line in enumerate(lines): + # If the old_text contains context about where to insert + if old_text and old_text.lower().strip() in line.lower(): + # Insert after this line + return i + 1 + + # Look for pattern where we should insert a new section + # Insert before conclusion, examples, or other sections + if line.strip().startswith('##') and any(keyword in line.lower() for keyword in ['example', 'conclusion', 'summary', 'next steps']): + return i + + # If no specific location found, insert before the last section + for i in range(len(lines) - 1, -1, -1): + if lines[i].strip().startswith('##'): + return i + + # Last resort: insert at 80% through the document + return int(len(lines) * 0.8) + + except Exception: + return None + + def _apply_diff_patch_copilot(self, original_content: str, diff_patch: str, file_path: str) -> Optional[str]: + """Apply a unified diff patch to the original content""" + try: + import tempfile + import subprocess + import os + + # Create temporary files + with tempfile.TemporaryDirectory() as temp_dir: + # Write original content to temp file + original_file = os.path.join(temp_dir, "original.txt") + with open(original_file, 'w', encoding='utf-8') as f: + f.write(original_content) + + # Write diff patch to temp file + patch_file = os.path.join(temp_dir, "changes.patch") + with open(patch_file, 'w', encoding='utf-8') as f: + f.write(diff_patch) + + # Apply patch using git apply + try: + subprocess.run(['git', 'apply', '--verbose', patch_file], + cwd=temp_dir, check=True, capture_output=True, text=True) + + # Read the result + with open(original_file, 'r', encoding='utf-8') as f: + return f.read() + + except subprocess.CalledProcessError: + # Fallback to manual patch application + self.logger.log("📝 Git apply failed, trying manual diff application...") + return self._manual_diff_apply_copilot(original_content, diff_patch) + + except Exception as e: + self.logger.log(f"⚠️ GitHub Copilot patch application failed: {str(e)}") + return self._manual_diff_apply_copilot(original_content, diff_patch) + + def _manual_diff_apply_copilot(self, original_content: str, diff_patch: str) -> Optional[str]: + """Manually apply a diff patch when git apply fails""" + try: + # Detect original line ending style + has_crlf = '\r\n' in original_content + + original_lines = original_content.split('\n') + result_lines = original_lines.copy() + + # Parse the diff patch + diff_lines = diff_patch.split('\n') + current_original_line = 0 + + i = 0 + while i < len(diff_lines): + line = diff_lines[i] + + # Look for @@ headers + if line.startswith('@@'): + # Extract line numbers: @@ -start,count +start,count @@ + parts = line.split() + if len(parts) >= 3: + old_info = parts[1][1:] # Remove the - + if ',' in old_info: + start_line = int(old_info.split(',')[0]) - 1 # Convert to 0-based + else: + start_line = int(old_info) - 1 + + current_original_line = start_line + i += 1 + continue + + # Skip diff headers (must check before processing -/+ lines) + if line.startswith('---') or line.startswith('+++'): + i += 1 + continue + + # Process diff lines + if line.startswith('-'): + # Remove line + if current_original_line < len(result_lines): + del result_lines[current_original_line] + elif line.startswith('+'): + # Add line + new_line = line[1:] # Remove the + + # If original had CRLF and this line doesn't have \r, add it + if has_crlf and not new_line.endswith('\r'): + new_line = new_line + '\r' + result_lines.insert(current_original_line, new_line) + current_original_line += 1 + elif line.startswith(' '): + # Context line - advance + current_original_line += 1 + + i += 1 + + return '\n'.join(result_lines) + + except Exception as e: + self.logger.log(f"❌ GitHub Copilot manual diff application failed: {str(e)}") + return None + + def _detect_change_type(self, old_text: str, new_text: str, file_path: str) -> str: + """Detect the type of change requested""" + old_lower = old_text.lower() + new_lower = new_text.lower() + + # Additive indicators + additive_keywords = ['add', 'include', 'incorporate', 'insert', 'create section', 'new section', 'best practices'] + if any(keyword in old_lower or keyword in new_lower for keyword in additive_keywords): + return "ADDITIVE" + + # Corrective indicators + corrective_keywords = ['correct', 'fix', 'grammar', 'spelling', 'typo', 'misspell', 'wrong', 'error'] + if any(keyword in old_lower or keyword in new_lower for keyword in corrective_keywords): + return "CORRECTIVE" + + # If new text is much longer than old text, likely additive + if len(new_text.strip()) > len(old_text.strip()) * 2: + return "ADDITIVE" + + # If similar length, likely corrective + if abs(len(new_text.strip()) - len(old_text.strip())) < 50: + return "CORRECTIVE" + + return "GENERAL" + + def _handle_additive_change_copilot(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Handle additive changes using GitHub Copilot""" + self.logger.log("🔨 GitHub Copilot handling additive change - generating new content...") + + try: + import requests + + url = "https://models.inference.ai.azure.com/chat/completions" + headers = { + "Authorization": f"Bearer {self.api_key}", + "Content-Type": "application/json", + } + + prompt = f"""You are helping add new content to a documentation file. + +File: {file_path} +Request: {old_text} +Content to add: {new_text} + +Your task: Generate ONLY the new content that should be added. Do not rewrite the existing file. + +Rules: +1. Generate ONLY the new section/content to be added +2. Use proper markdown formatting if it's a markdown file +3. Make it standalone - don't reference existing content +4. Do not include any existing file content in your response +5. Return only the new content, nothing else + +Generate the new content now:""" + + data = { + "messages": [ + {"role": "system", "content": "You are a content generator. Generate only new content, never rewrite existing content."}, + {"role": "user", "content": prompt} + ], + "model": "gpt-4o", + "temperature": 0.1, + "max_tokens": 2048 + } + + response = requests.post(url, headers=headers, json=data, timeout=60) + response.raise_for_status() + + result = response.json() + new_content = result['choices'][0]['message']['content'].strip() + + # Clean up markdown blocks if needed + if new_content.startswith("```") and new_content.endswith("```"): + lines = new_content.split('\n') + if len(lines) > 2: + new_content = '\n'.join(lines[1:-1]) + + # Find insertion point and insert + insertion_point = self._find_insertion_point(file_content, old_text, file_path) + lines = file_content.split('\n') + lines.insert(insertion_point, '\n' + new_content + '\n') + updated_content = '\n'.join(lines) + + # Count actual changes + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+')]) + + self.logger.log(f"✅ GitHub Copilot added new content ({changed_lines} lines added)") + return updated_content + + except Exception as e: + self.logger.log(f"❌ Error in GitHub Copilot additive change: {str(e)}") + return None + + def _handle_corrective_change_copilot(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Handle corrective changes using GitHub Copilot""" + self.logger.log("🔍 GitHub Copilot handling corrective change - finding specific issues...") + + try: + import requests + + url = "https://models.inference.ai.azure.com/chat/completions" + headers = { + "Authorization": f"Bearer {self.api_key}", + "Content-Type": "application/json", + } + + prompt = f"""You are helping fix a specific issue in a documentation file. + +Issue: {old_text} +Fix: {new_text} +File: {file_path} + +Your task: Find the EXACT text that needs to be corrected and provide the EXACT replacement. + +Return your response in this format: +OLD: [exact text to find] +NEW: [exact replacement text] + +Be very specific - find the minimal text that needs changing. For example: +- If fixing "Microsft" → return OLD: Microsft, NEW: Microsoft +- If fixing grammar → return OLD: [the incorrect phrase], NEW: [corrected phrase] + +File content to search: +{file_content} + +Find the exact text to correct:""" + + data = { + "messages": [ + {"role": "system", "content": "You are a precise error detector. Find exact text that needs correction."}, + {"role": "user", "content": prompt} + ], + "model": "gpt-4o", + "temperature": 0.1, + "max_tokens": 1024 + } + + response = requests.post(url, headers=headers, json=data, timeout=60) + response.raise_for_status() + + result = response.json() + response_text = result['choices'][0]['message']['content'].strip() + + # Parse OLD and NEW + old_match = None + new_match = None + + for line in response_text.split('\n'): + if line.startswith('OLD:'): + old_match = line[4:].strip() + elif line.startswith('NEW:'): + new_match = line[4:].strip() + + if old_match and new_match and old_match in file_content: + updated_content = file_content.replace(old_match, new_match) + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + self.logger.log(f"✅ GitHub Copilot corrective change successful ({changed_lines} lines affected)") + return updated_content + else: + self.logger.log(f"⚠️ GitHub Copilot could not find exact text to correct") + return None + + except Exception as e: + self.logger.log(f"❌ Error in GitHub Copilot corrective change: {str(e)}") + return None + + def _handle_general_change_copilot(self, file_content: str, old_text: str, new_text: str, file_path: str) -> Optional[str]: + """Handle general changes using GitHub Copilot with enhanced targeting""" + self.logger.log("🎯 GitHub Copilot handling general change with enhanced targeting...") + + try: + import requests + + url = "https://models.inference.ai.azure.com/chat/completions" + headers = { + "Authorization": f"Bearer {self.api_key}", + "Content-Type": "application/json", + } + + prompt = f"""You are helping make a specific text change in a documentation file. + +File: {file_path} +Change needed: {old_text} +New content: {new_text} + +CRITICAL: Make the SMALLEST possible change. Do not rewrite or reorganize content. + +Your task: +1. Find the specific section that needs changing +2. Make ONLY that change +3. Preserve everything else exactly as-is +4. Return the complete updated file + +Current file content: +{file_content}""" + + data = { + "messages": [ + {"role": "system", "content": "You are a precise file editor. Make minimal targeted changes only."}, + {"role": "user", "content": prompt} + ], + "model": "gpt-4o", + "temperature": 0.1, + "max_tokens": 8000 + } + + response = requests.post(url, headers=headers, json=data, timeout=60) + response.raise_for_status() + + result = response.json() + updated_content = result['choices'][0]['message']['content'].strip() + + # Clean up markdown code blocks + if updated_content.startswith("```"): + lines = updated_content.split('\n') + if len(lines) > 2: + if lines[0].startswith("```") and lines[-1].strip() == "```": + updated_content = '\n'.join(lines[1:-1]) + + if new_text.strip() in updated_content: + original_lines = file_content.split('\n') + updated_lines = updated_content.split('\n') + + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + + if changed_lines > 30: + self.logger.log(f"⚠️ GitHub Copilot change affected {changed_lines} lines - may be too broad") + else: + self.logger.log(f"✅ GitHub Copilot general change successful ({changed_lines} lines affected)") + + return updated_content + else: + self.logger.log("⚠️ GitHub Copilot: New text not found in result") + return None + + except Exception as e: + self.logger.log(f"❌ Error in GitHub Copilot general change: {str(e)}") + return None + + def _find_insertion_point(self, file_content: str, context: str, file_path: str) -> int: + """Find the best place to insert new content""" + lines = file_content.split('\n') + + # For markdown files, try to find a good section to add after + if file_path.endswith('.md'): + # Look for existing sections + for i, line in enumerate(lines): + if line.startswith('#') and i < len(lines) - 1: + # Insert after this section + continue + + # If no good sections found, add at the end + return len(lines) + + # For other files, add at the end + return len(lines) + + +class LocalGitManager: + """Manages local git operations for making changes before creating PRs""" + + def __init__(self, logger: Logger, github_token: str): + self.logger = logger + self.github_token = github_token + self.last_diff_content = "" # Store the last generated diff content + + def get_repo_path(self, owner: str, repo: str, local_path: Optional[str] = None) -> Path: + """Get or create local repository path + + Args: + owner: Repository owner + repo: Repository name + local_path: Base path from LOCAL_REPO_PATH setting + + Returns: + Full path to the repository (base/owner/repo) + """ + # If LOCAL_REPO_PATH is configured, use it as the base directory + if local_path and local_path.strip(): + base_path = Path(local_path.strip()) + + # Warn if OneDrive path detected + if 'OneDrive' in str(base_path): + self.logger.log("⚠️ WARNING: Local Repo Path is in a OneDrive folder") + self.logger.log(" OneDrive sync can cause file locking issues with git operations") + self.logger.log(" Consider using a non-OneDrive location (e.g., C:\\git\\repos)") + + # Create base directory if it doesn't exist + if not base_path.exists(): + self.logger.log(f"Creating local repo directory: {base_path}") + try: + base_path.mkdir(parents=True, exist_ok=True) + except Exception as e: + self.logger.log(f"⚠️ Could not create directory {base_path}: {e}") + self.logger.log(" Falling back to default location") + # Fall through to default + else: + # Successfully created or exists, use it + repo_path = base_path / owner / repo + return repo_path + else: + # Base path exists, use it + repo_path = base_path / owner / repo + return repo_path + + # Default: Use Downloads folder (typically not in OneDrive) + downloads = Path.home() / "Downloads" + repo_path = downloads / "github_repos" / owner / repo + return repo_path + + def clone_or_pull_repo(self, owner: str, repo: str, local_path: Optional[str] = None) -> Optional[Path]: + """Clone repository if it doesn't exist, or pull latest changes if it does""" + try: + import git + import gc + + repo_path = self.get_repo_path(owner, repo, local_path) + repo_url = f"https://{self.github_token}@github.com/{owner}/{repo}.git" + + if repo_path.exists() and (repo_path / ".git").exists(): + # Repository exists, try to update it + self.logger.log(f"Repository exists at {repo_path}, updating...") + git_repo = None + try: + git_repo = git.Repo(repo_path) + + # Try alternative update methods that are more reliable + try: + # Method 1: Fetch and reset (more reliable than pull) + self.logger.log("Fetching latest changes...") + git_repo.git.fetch('origin') + + # Make sure we're on main/master + try: + git_repo.git.checkout('main') + git_repo.git.reset('--hard', 'origin/main') + except: + git_repo.git.checkout('master') + git_repo.git.reset('--hard', 'origin/master') + + self.logger.log("✅ Repository updated successfully") + return repo_path + except Exception as fetch_error: + self.logger.log(f"⚠️ Fetch/reset failed: {fetch_error}") + # Try simple pull as fallback + origin = git_repo.remotes.origin + origin.pull() + self.logger.log("✅ Pulled latest changes") + return repo_path + + except Exception as e: + self.logger.log(f"⚠️ Error updating repo: {e}") + self.logger.log("Repository will be reused as-is for this operation") + + # Don't try to delete - just reuse the existing repo + # This avoids file locking issues + return repo_path + finally: + # Always clean up + if git_repo: + try: + git_repo.close() + git_repo.__del__() + except: + pass + git_repo = None + gc.collect() + + # Clone repository + self.logger.log(f"Cloning repository to {repo_path}...") + repo_path.parent.mkdir(parents=True, exist_ok=True) + git.Repo.clone_from(repo_url, repo_path) + self.logger.log("✅ Repository cloned successfully") + return repo_path + + except ImportError: + self.logger.log("❌ GitPython not installed. Run: pip install GitPython") + return None + except Exception as e: + self.logger.log(f"❌ Error with git operations: {str(e)}") + return None + + def _safe_remove_tree(self, path: Path, max_retries: int = 3) -> bool: + """Safely remove a directory tree with retry logic for Windows file locking""" + import gc + + for attempt in range(max_retries): + try: + if path.exists(): + # On Windows, make files writable before deletion + if sys.platform == 'win32': + for root, _, files in os.walk(str(path)): + for fname in files: + fpath = os.path.join(root, fname) + try: + os.chmod(fpath, 0o777) + except: + pass + + shutil.rmtree(path, ignore_errors=False) + self.logger.log(f"✅ Removed directory: {path}") + return True + except Exception as e: + if attempt < max_retries - 1: + self.logger.log(f"⚠️ Attempt {attempt + 1} failed to remove {path}: {e}") + gc.collect() # Force garbage collection + time.sleep(1) # Wait longer between retries + else: + self.logger.log(f"❌ Failed to remove {path} after {max_retries} attempts: {e}") + self.logger.log(f"💡 TIP: Close any file explorers or editors that might have this folder open") + return False + return False + + def apply_diff_and_commit(self, repo_path: Path, branch_name: str, + file_path: str, diff_patch: str, commit_message: str) -> bool: + """Apply diff patch using git apply and commit changes + + This is the preferred method as it uses native git to apply patches, + which properly handles line endings, whitespace, and other edge cases. + """ + git_repo = None + try: + import git + import tempfile + import os + + git_repo = git.Repo(repo_path) + + # Create new branch from main + self.logger.log(f"Creating branch {branch_name}...") + try: + git_repo.git.checkout('main') + git_repo.git.pull() + except: + git_repo.git.checkout('master') + git_repo.git.pull() + + git_repo.git.checkout('-b', branch_name) + self.logger.log(f"✅ Branch {branch_name} created") + + # Write diff patch to temp file + # Ensure patch ends with newline for git apply compatibility + patch_content = diff_patch if diff_patch.endswith('\n') else diff_patch + '\n' + + with tempfile.NamedTemporaryFile(mode='w', suffix='.patch', delete=False, encoding='utf-8', newline='\n') as patch_file: + patch_file.write(patch_content) + patch_file_path = patch_file.name + + try: + # Apply patch using git apply + self.logger.log(f"Applying diff patch to {file_path}...") + self.logger.log(f"Patch file: {patch_file_path}") + + try: + git_repo.git.apply('--verbose', '--whitespace=nowarn', patch_file_path) + self.logger.log("✅ Diff patch applied successfully using git apply") + except Exception as apply_error: + self.logger.log(f"⚠️ git apply failed: {str(apply_error)}") + + # Log the patch content for debugging + self.logger.log("📄 Patch content (first 1000 chars):") + self.logger.log(patch_content[:1000]) + + self.logger.log("📝 Attempting to apply patch with --3way merge...") + try: + # Try with 3-way merge which is more forgiving + git_repo.git.apply('--3way', '--whitespace=nowarn', patch_file_path) + self.logger.log("✅ Diff patch applied using 3-way merge") + except Exception as merge_error: + self.logger.log(f"⚠️ 3-way merge also failed: {str(merge_error)}") + + # Try one more time with --ignore-whitespace + self.logger.log("📝 Attempting with --ignore-whitespace...") + try: + git_repo.git.apply('--ignore-whitespace', '--whitespace=nowarn', patch_file_path) + self.logger.log("✅ Diff patch applied with --ignore-whitespace") + except: + self.logger.log("❌ All git apply methods failed") + # Keep the patch file for debugging + self.logger.log(f"💾 Patch file saved for debugging: {patch_file_path}") + raise + + # Stage and commit + git_repo.index.add([file_path]) + git_repo.index.commit(commit_message) + self.logger.log("✅ Changes committed") + + return True + + finally: + # Clean up temp patch file only on success + if git_repo and git_repo.head.is_valid(): + try: + os.unlink(patch_file_path) + except: + pass + + except Exception as e: + self.logger.log(f"❌ Error applying diff and committing: {str(e)}") + self.logger.log("💡 This may indicate the file has changed since it was fetched") + return False + finally: + if git_repo: + try: + git_repo.close() + git_repo.__del__() + except: + pass + import gc + gc.collect() + + def create_branch_and_commit(self, repo_path: Path, branch_name: str, + file_path: str, updated_content: str, + commit_message: str, line_ending: str = '\n') -> bool: + """Create branch, update file, and commit + + Args: + line_ending: Original line ending style to preserve ('\n' or '\r\n') + + NOTE: This method is deprecated in favor of apply_diff_and_commit which uses git apply. + """ + git_repo = None + try: + import git + import gc + + git_repo = git.Repo(repo_path) + + # Create new branch from main + self.logger.log(f"Creating branch {branch_name}...") + try: + git_repo.git.checkout('main') + git_repo.git.pull() + except: + git_repo.git.checkout('master') + git_repo.git.pull() + + git_repo.git.checkout('-b', branch_name) + self.logger.log(f"✅ Branch {branch_name} created") + + # Update the file + full_file_path = repo_path / file_path + if not full_file_path.exists(): + self.logger.log(f"❌ File not found: {file_path}") + return False + + self.logger.log(f"Writing changes to {file_path}...") + + # Preserve original line endings + if line_ending == '\r\n': + # Normalize to CRLF if original had CRLF + content_to_write = updated_content.replace('\r\n', '\n').replace('\n', '\r\n') + self.logger.log(f"✅ Preserving CRLF line endings") + else: + content_to_write = updated_content + + full_file_path.write_text(content_to_write, encoding='utf-8', newline='') + + # Stage and commit + git_repo.index.add([file_path]) + git_repo.index.commit(commit_message) + self.logger.log("✅ Changes committed") + + return True + + except Exception as e: + self.logger.log(f"❌ Error creating branch and committing: {str(e)}") + return False + finally: + if git_repo: + try: + git_repo.close() + git_repo.__del__() + except: + pass + import gc + gc.collect() # Force garbage collection to release file handles + + def push_branch(self, repo_path: Path, branch_name: str) -> bool: + """Push branch to remote""" + git_repo = None + try: + import git + import gc + + self.logger.log(f"Pushing branch {branch_name} to remote...") + git_repo = git.Repo(repo_path) + origin = git_repo.remotes.origin + origin.push(branch_name) + self.logger.log("✅ Branch pushed to remote") + return True + + except Exception as e: + self.logger.log(f"❌ Error pushing branch: {str(e)}") + return False + finally: + if git_repo: + try: + git_repo.close() + git_repo.__del__() + except: + pass + import gc + gc.collect() # Force garbage collection to release file handles + + def make_ai_assisted_change(self, owner: str, repo: str, branch_name: str, + file_path: str, old_text: str, new_text: str, + commit_message: str, ai_provider: AIProvider, + local_path: Optional[str] = None, custom_instructions: str = None) -> Tuple[bool, Optional[str]]: + """ + Complete workflow: clone, make TARGETED changes, commit, and push + This uses direct string replacement to avoid AI rewriting entire files + + Returns: + (success: bool, error_message: Optional[str]) + """ + try: + # Step 1: Clone or pull repository + repo_path = self.clone_or_pull_repo(owner, repo, local_path) + if not repo_path: + return False, "Failed to clone/pull repository" + + # Step 2: Read the current file + full_file_path = repo_path / file_path + if not full_file_path.exists(): + return False, f"File not found: {file_path}" + + self.logger.log(f"Reading file: {file_path}") + # Read in binary mode to detect and preserve line endings + raw_bytes = full_file_path.read_bytes() + current_content = raw_bytes.decode('utf-8') + + # Detect original line ending style + original_line_ending = '\r\n' if b'\r\n' in raw_bytes else '\n' + self.logger.log(f"📝 Detected line endings: {'CRLF' if original_line_ending == '\\r\\n' else 'LF'}") + + # Normalize everything to LF for consistent processing + normalized_content = current_content.replace('\r\n', '\n') + normalized_old = old_text.replace('\r\n', '\n') + normalized_new = new_text.replace('\r\n', '\n') + + # Step 3: Make TARGETED change + updated_content = None + + # Strategy 1: Very conservative direct replacement (only for exact, specific content) + # Only use this for replacements where old_text is substantial and very specific + use_direct_replacement = ( + normalized_old.strip() and + len(normalized_old.strip()) > 20 and # Must be substantial content + normalized_old.strip().count('\n') >= 2 and # Must be multi-line + normalized_old.strip() in normalized_content and + normalized_content.count(normalized_old.strip()) == 1 # Must be unique match + ) + + if use_direct_replacement: + self.logger.log("✅ Making very targeted direct replacement") + updated_content = normalized_content.replace(normalized_old.strip(), normalized_new.strip()) + + # Verify the replacement worked and was targeted + original_lines = normalized_content.split('\n') + updated_lines = updated_content.split('\n') + + import difflib + diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm='')) + changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')]) + + if changed_lines > 20: # If too many changes, something went wrong + self.logger.log(f"⚠️ Direct replacement affected {changed_lines} lines - falling back to AI") + updated_content = None # Fall back to AI + else: + self.logger.log(f"✅ Direct replacement successful ({changed_lines} lines changed)") + + # Strategy 2: Use AI to generate complete updated document + if not updated_content: + self.logger.log("Using AI to modify complete document...") + self.logger.log(f"AI Provider type: {type(ai_provider).__name__}") + self.logger.log(f"Old text preview: {normalized_old[:100]}...") + self.logger.log(f"New text preview: {normalized_new[:100]}...") + + # Pass normalized versions to AI provider + try: + updated_content = ai_provider.make_change(normalized_content, normalized_old, normalized_new, file_path, custom_instructions) + if not updated_content: + self.logger.log("❌ AI provider returned None or empty content") + return False, "AI failed to make the change - provider returned no content" + else: + self.logger.log(f"✅ AI provider returned content ({len(updated_content)} characters)") + except Exception as e: + self.logger.log(f"❌ AI provider threw exception: {str(e)}") + return False, f"AI failed to make the change - error: {str(e)}" + + # Step 4: Apply changes directly using file write method + # Restore original line endings in the updated content before writing + if original_line_ending == '\r\n': + updated_content_with_endings = updated_content.replace('\n', '\r\n') + else: + updated_content_with_endings = updated_content + + if not self.create_branch_and_commit(repo_path, branch_name, file_path, + updated_content_with_endings, commit_message): + return False, "Failed to apply changes using direct file write method" + + self.logger.log("✅ Changes applied using direct file write method") + + # Step 5: Push to remote + if not self.push_branch(repo_path, branch_name): + return False, "Failed to push branch to remote" + + self.logger.log("✅ AI-assisted changes completed successfully") + return True, None + + except Exception as e: + error_msg = f"Error in AI-assisted change workflow: {str(e)}" + self.logger.log(f"❌ {error_msg}") + return False, error_msg + + def get_last_diff_content(self) -> str: + """Get the last generated diff content for display in the UI""" + return self.last_diff_content + + def clear_diff_content(self): + """Clear the stored diff content""" + self.last_diff_content = "" + + def get_git_diff_from_repo(self, repo_path: str, branch_name: str) -> str: + """Get the actual git diff from the repository for the specified branch""" + try: + import subprocess + import os + + self.logger.log(f"🔍 Getting git diff from: {repo_path}") + self.logger.log(f"🔍 Branch: {branch_name}") + + # Change to repo directory + original_dir = os.getcwd() + + if not os.path.exists(repo_path): + self.logger.log(f"❌ Repository path does not exist: {repo_path}") + return "" + + os.chdir(repo_path) + self.logger.log(f"🔍 Changed to directory: {os.getcwd()}") + + try: + diff_content = "" + + # Check current git status first + try: + result = subprocess.run(['git', 'status', '--porcelain'], + capture_output=True, text=True, check=True, encoding='utf-8', errors='replace') + if result.stdout.strip(): + self.logger.log(f"🔍 Git status shows changes: {result.stdout.strip()[:100]}...") + else: + self.logger.log("🔍 Git status shows no uncommitted changes") + except Exception as e: + self.logger.log(f"⚠️ Could not check git status: {e}") + + # Check current branch + try: + result = subprocess.run(['git', 'branch', '--show-current'], + capture_output=True, text=True, check=True, encoding='utf-8', errors='replace') + current_branch = result.stdout.strip() + self.logger.log(f"🔍 Current branch: {current_branch}") + except Exception as e: + self.logger.log(f"⚠️ Could not get current branch: {e}") + + # First, try to get diff from the current commit against main/master + try: + self.logger.log("🔍 Trying: git diff main HEAD") + result = subprocess.run(['git', 'diff', 'main', 'HEAD'], + capture_output=True, text=True, check=True, encoding='utf-8', errors='replace') + diff_content = result.stdout + if diff_content: + self.logger.log(f"✅ Retrieved git diff against main ({len(diff_content)} characters)") + else: + self.logger.log("🔍 No diff found against main") + except subprocess.CalledProcessError as e: + self.logger.log(f"🔍 git diff main HEAD failed: {e}") + try: + self.logger.log("🔍 Trying: git diff master HEAD") + result = subprocess.run(['git', 'diff', 'master', 'HEAD'], + capture_output=True, text=True, check=True, encoding='utf-8', errors='replace') + diff_content = result.stdout + if diff_content: + self.logger.log(f"✅ Retrieved git diff against master ({len(diff_content)} characters)") + else: + self.logger.log("🔍 No diff found against master") + except subprocess.CalledProcessError as e: + self.logger.log(f"🔍 git diff master HEAD failed: {e}") + + # If still no diff, try against previous commit (only if it exists) + if not diff_content: + try: + self.logger.log("🔍 Trying: git rev-parse --verify HEAD~1") + subprocess.run(['git', 'rev-parse', '--verify', 'HEAD~1'], + capture_output=True, check=True, encoding='utf-8', errors='replace') + # If we get here, HEAD~1 exists + self.logger.log("🔍 Trying: git diff HEAD~1 HEAD") + result = subprocess.run(['git', 'diff', 'HEAD~1', 'HEAD'], + capture_output=True, text=True, check=True, encoding='utf-8', errors='replace') + diff_content = result.stdout + if diff_content: + self.logger.log(f"✅ Retrieved git diff against HEAD~1 ({len(diff_content)} characters)") + else: + self.logger.log("🔍 No diff found against HEAD~1") + except subprocess.CalledProcessError as e: + self.logger.log(f"🔍 HEAD~1 doesn't exist or diff failed: {e}") + + # If still no diff, try to get the diff of all changes in the current commit + if not diff_content: + try: + self.logger.log("🔍 Trying: git show --format= HEAD") + result = subprocess.run(['git', 'show', '--format=', 'HEAD'], + capture_output=True, text=True, check=True, encoding='utf-8', errors='replace') + diff_content = result.stdout + if diff_content: + self.logger.log(f"✅ Retrieved git show for HEAD commit ({len(diff_content)} characters)") + else: + self.logger.log("🔍 No content from git show HEAD") + except subprocess.CalledProcessError as e: + self.logger.log(f"🔍 git show HEAD failed: {e}") + + # If we have diff content, save it to a .diff file + if diff_content: + self._save_diff_to_file(diff_content, repo_path, branch_name) + else: + self.logger.log("❌ No diff content found using any method") + + return diff_content + + finally: + os.chdir(original_dir) + self.logger.log(f"🔍 Changed back to: {os.getcwd()}") + + except Exception as e: + self.logger.log(f"❌ Error getting git diff from repository: {str(e)}") + import traceback + self.logger.log(f"❌ Traceback: {traceback.format_exc()}") + return "" + + def _save_diff_to_file(self, diff_content: str, repo_path: str, branch_name: str) -> None: + """Save the diff content to a .diff file in the repository""" + try: + import os + from datetime import datetime + + # Create a filename with timestamp and branch name + timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") + safe_branch_name = branch_name.replace('/', '_').replace(':', '_') + diff_filename = f"changes_{safe_branch_name}_{timestamp}.diff" + diff_filepath = os.path.join(repo_path, diff_filename) + + # Write the diff content to file + with open(diff_filepath, 'w', encoding='utf-8') as f: + f.write(diff_content) + + self.logger.log(f"💾 Saved diff to: {diff_filename}") + + except Exception as e: + self.logger.log(f"❌ Error saving diff to file: {str(e)}") + + +def create_ai_provider(provider_name: str, api_key: str, logger: Logger) -> Optional[AIProvider]: + """Factory function to create AI provider instances""" + if provider_name.lower() == 'claude': + return ClaudeProvider(api_key, logger) + elif provider_name.lower() in ['chatgpt', 'openai', 'gpt']: + return ChatGPTProvider(api_key, logger) + elif provider_name.lower() in ['github-copilot', 'copilot', 'github_copilot']: + return GitHubCopilotProvider(api_key, logger) + else: + logger.log(f"⚠️ Unknown AI provider: {provider_name}") + return None + + +def get_detailed_python_environment_info() -> dict: + """Get detailed information about the current Python environment + + Returns: + dict: Environment information including venv status, Python version, etc. + """ + import sys + import os + + # Detect virtual environment + in_venv = (hasattr(sys, 'real_prefix') or + (hasattr(sys, 'base_prefix') and sys.base_prefix != sys.prefix) or + os.environ.get('VIRTUAL_ENV') is not None) + + env_info = { + 'in_venv': in_venv, + 'python_version': f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}", + 'python_executable': sys.executable, + } + + if in_venv: + venv_path = os.environ.get('VIRTUAL_ENV', sys.prefix) + env_info['venv_name'] = os.path.basename(venv_path) + env_info['venv_path'] = venv_path + else: + env_info['venv_name'] = None + env_info['venv_path'] = None + + return env_info + + +def install_ai_packages_enhanced(packages: List[str], parent_window=None) -> bool: + """Enhanced AI provider package installation with better error handling + + Args: + packages: List of package names to install + parent_window: Parent tkinter window for dialog (optional) + + Returns: + bool: True if installation successful or user declined, False if failed + """ + import subprocess + import sys + import os + + if not packages: + return True + + # Detect virtual environment + in_venv = (hasattr(sys, 'real_prefix') or + (hasattr(sys, 'base_prefix') and sys.base_prefix != sys.prefix) or + os.environ.get('VIRTUAL_ENV') is not None) + + venv_info = "" + install_location = "" + + if in_venv: + venv_path = os.environ.get('VIRTUAL_ENV', sys.prefix) + venv_name = os.path.basename(venv_path) + install_location = f"virtual environment '{venv_name}'" + venv_info = f"\n🌐 Virtual environment detected: {venv_name}" + else: + install_location = "system-wide (may require administrator rights)" + venv_info = f"\n⚠️ No virtual environment detected - installing system-wide" + + # Create confirmation message + package_list = ', '.join(packages) + message = (f"The following packages are required for AI functionality:\n\n" + f"{package_list}\n\n" + f"Installation location: {install_location}" + f"{venv_info}\n\n" + f"Would you like to install them now?\n\n" + f"This will run: pip install {' '.join(packages)}") + + # Show confirmation dialog + try: + import tkinter as tk + from tkinter import messagebox + + # If we have a parent window, use it; otherwise create a temporary root + if parent_window: + result = messagebox.askyesno("Install AI Packages", message, parent=parent_window) + else: + # Create temporary root window for the dialog + temp_root = tk.Tk() + temp_root.withdraw() # Hide the temporary window + result = messagebox.askyesno("Install AI Packages", message) + temp_root.destroy() + + if not result: + print("User declined to install AI packages") + return True # User declined, but this isn't a failure + + except Exception as e: + print(f"Could not show dialog, proceeding with installation: {e}") + # If dialog fails, ask in console + response = input(f"Install AI packages ({package_list})? [y/N]: ").lower() + if response not in ['y', 'yes']: + return True + + # Install packages + try: + if in_venv: + print(f"Installing packages to virtual environment: {package_list}") + else: + print(f"Installing packages system-wide: {package_list}") + + for package in packages: + print(f"Installing {package}...") + + # Build pip command + pip_cmd = [sys.executable, '-m', 'pip', 'install', package] + + # First attempt: Direct installation + result = subprocess.run(pip_cmd, capture_output=True, text=True, timeout=300) + + # If direct install fails and we're not in venv, try with --user flag + if result.returncode != 0 and not in_venv: + print(f" Direct installation failed, trying with --user flag...") + pip_cmd_user = [sys.executable, '-m', 'pip', 'install', '--user', package] + result = subprocess.run(pip_cmd_user, capture_output=True, text=True, timeout=300) + + if result.returncode == 0: + print(f"✅ Successfully installed {package} (user-local)") + continue + + if result.returncode != 0: + print(f"❌ Failed to install {package}:") + print(f"Error: {result.stderr}") + + # Show more helpful error message + if "permission" in result.stderr.lower() or "access" in result.stderr.lower(): + print(" This appears to be a permissions issue.") + if not in_venv: + print(" Consider:") + print(" 1. Running as administrator") + print(" 2. Using a virtual environment") + print(" 3. Installing with --user flag") + + return False + else: + install_type = "to virtual environment" if in_venv else "system-wide" + print(f"✅ Successfully installed {package} ({install_type})") + + success_msg = "✅ AI packages installed successfully!" + if in_venv: + success_msg += f" (installed to virtual environment)" + else: + success_msg += f" (installed system-wide)" + + print(success_msg) + print("Please restart the application to use the new AI features.") + return True + + except subprocess.TimeoutExpired: + print("❌ Installation timed out") + return False + except Exception as e: + print(f"❌ Error installing packages: {e}") + return False + + +def validate_ai_provider_setup(config: dict, parent_window=None) -> bool: + """Validate AI provider setup and offer to install missing modules + + Args: + config: Configuration dictionary + parent_window: Parent tkinter window for dialogs + + Returns: + bool: True if setup is valid or user handled the issue + """ + ai_provider = config.get('AI_PROVIDER', '').lower() + + if not ai_provider or ai_provider == 'none': + return True # No AI provider selected, nothing to validate + + # Create a temporary AI manager to check modules + temp_manager = AIManager() + + # Check if modules are available + available, missing = temp_manager.check_ai_module_availability(ai_provider) + + if available: + return True # All modules available + + print(f"⚠️ AI Provider '{ai_provider}' selected but missing required packages: {', '.join(missing)}") + + # Offer to install missing packages using enhanced installer + success = install_ai_packages_enhanced(missing, parent_window) + + if success: + # Re-check availability after installation + available, still_missing = temp_manager.check_ai_module_availability(ai_provider) + if available: + print(f"✅ AI Provider '{ai_provider}' is now ready to use") + return True + else: + print(f"⚠️ Some packages may still be missing: {', '.join(still_missing)}") + print("Please restart the application after installation completes") + return False + + return False + + +# AI Providers availability flag - now always True since they're included +AI_PROVIDERS_AVAILABLE = True + + +class AIManager: + """Manages AI providers and module installations""" + + def __init__(self, logger=None): + self.logger = logger + self.last_diff_content = "" # Store the last generated diff content + + def log(self, message: str) -> None: + """Log a message with Unicode support""" + if self.logger: + self.logger.log(message) + else: + try: + print(message) + except UnicodeEncodeError: + # Fallback: replace Unicode emojis with ASCII equivalents + safe_message = message.replace('✅', '[SUCCESS]').replace('❌', '[ERROR]').replace('⚠️', '[WARNING]').replace('📋', '[INFO]').replace('📄', '[FILE]').replace('📍', '[LOCATION]').replace('📝', '[EDIT]') + print(safe_message) + + def check_ai_module_availability(self, provider_name: str) -> Tuple[bool, List[str]]: + """Check if AI provider modules are available and return missing packages + + Args: + provider_name: 'chatgpt', 'claude', 'anthropic', or 'github-copilot' + + Returns: + tuple: (all_available, missing_packages) + """ + missing_packages = [] + + # Common packages needed for AI providers + required_common = ['GitPython'] + + # Provider-specific packages + if provider_name.lower() == 'chatgpt': + required_packages = required_common + ['openai'] + elif provider_name.lower() in ['claude', 'anthropic']: + required_packages = required_common + ['anthropic'] + elif provider_name.lower() in ['github-copilot', 'copilot', 'github_copilot']: + required_packages = required_common + ['requests'] + else: + return True, [] # Unknown provider, assume no check needed + + for package in required_packages: + try: + if package == 'GitPython': + import git + elif package == 'openai': + import openai + elif package == 'anthropic': + import anthropic + except ImportError: + missing_packages.append(package) + + all_available = len(missing_packages) == 0 + return all_available, missing_packages + + def get_python_environment_info(self) -> dict: + """Get information about the current Python environment""" + return get_detailed_python_environment_info() + + def install_ai_packages(self, packages: List[str], parent_window=None) -> bool: + """Install AI packages using pip""" + try: + env_info = self.get_python_environment_info() + install_location = f"virtual environment '{env_info['venv_name']}'" if env_info['in_venv'] else "system-wide" + + # Show confirmation dialog + if parent_window: + install_choice = messagebox.askyesno( + "Install AI Packages", + f"🐍 Python {env_info['python_version']}\n" + f"📦 Location: {install_location}\n\n" + f"The following packages will be installed:\n" + f"• {', '.join(packages)}\n\n" + f"This will run: pip install {' '.join(packages)}\n\n" + f"Continue with installation?", + parent=parent_window + ) + + if not install_choice: + return False + + self.log(f"Installing packages: {', '.join(packages)}") + self.log(f"Installation location: {install_location}") + + # Run pip install + cmd = [sys.executable, '-m', 'pip', 'install'] + packages + self.log(f"Running: {' '.join(cmd)}") + + result = subprocess.run(cmd, capture_output=True, text=True, timeout=300) + + if result.returncode == 0: + self.log("✅ Installation completed successfully!") + if parent_window: + messagebox.showinfo( + "Installation Complete", + f"✅ Successfully installed: {', '.join(packages)}\n\n" + f"Location: {install_location}", + parent=parent_window + ) + return True + else: + error_msg = f"❌ Installation failed!\n\nError: {result.stderr}" + self.log(error_msg) + if parent_window: + messagebox.showerror("Installation Failed", error_msg, parent=parent_window) + return False + + except subprocess.TimeoutExpired: + error_msg = "❌ Installation timed out (>5 minutes)" + self.log(error_msg) + if parent_window: + messagebox.showerror("Installation Timeout", error_msg, parent=parent_window) + return False + except Exception as e: + error_msg = f"❌ Installation error: {str(e)}" + self.log(error_msg) + if parent_window: + messagebox.showerror("Installation Error", error_msg, parent=parent_window) + return False + + def check_and_install_ai_modules(self, provider_name: str, parent_window=None) -> bool: + """Check AI modules and offer to install if missing""" + if not provider_name or provider_name.lower() in ['none', '']: + return True + + if provider_name.lower() not in ['chatgpt', 'claude', 'anthropic', 'github-copilot', 'copilot', 'github_copilot']: + return True + + # Check module availability + available, missing = self.check_ai_module_availability(provider_name) + + if available: + self.log(f"✅ All required modules for {provider_name} are available") + return True + + # Modules are missing, offer to install + self.log(f"⚠️ Missing modules for {provider_name}: {', '.join(missing)}") + + return self.install_ai_packages(missing, parent_window) + + def show_ai_modules_info(self, provider_name: str, parent_window=None) -> None: + """Show detailed AI modules information""" + try: + # Get environment information + env_info = self.get_python_environment_info() + env_status = f"🐍 Python {env_info['python_version']}" + + if env_info['in_venv']: + env_status += f" (venv: {env_info['venv_name']})" + else: + env_status += " (system-wide)" + + if not provider_name or provider_name.lower() in ['none', '']: + messagebox.showinfo("AI Modules Check", + f"{env_status}\n\n" + f"No AI provider selected.\n\n" + f"Available providers:\n" + f"• ChatGPT (requires 'openai' package)\n" + f"• Claude/Anthropic (requires 'anthropic' package)\n" + f"• GitHub Copilot (requires 'requests' package)\n" + f"• All require 'GitPython' package", + parent=parent_window) + return + + if provider_name.lower() not in ['chatgpt', 'claude', 'anthropic', 'github-copilot', 'copilot', 'github_copilot']: + messagebox.showinfo("AI Modules Check", + f"AI provider '{provider_name}' is not recognized.\n\n" + f"Supported providers: ChatGPT, Claude/Anthropic, GitHub Copilot", + parent=parent_window) + return + + # Check module availability + available, missing = self.check_ai_module_availability(provider_name) + + if available: + messagebox.showinfo("AI Modules Status", + f"{env_status}\n\n" + f"✅ All required modules for '{provider_name}' are installed!\n\n" + f"AI-assisted features are ready to use.", + parent=parent_window) + return + + # Modules are missing, show detailed info and offer to install + missing_list = '\n'.join(f"• {pkg}" for pkg in missing) + install_location = f"virtual environment '{env_info['venv_name']}'" if env_info['in_venv'] else "system-wide" + + install_choice = messagebox.askyesno("Missing AI Modules", + f"{env_status}\n\n" + f"AI provider '{provider_name}' requires the following packages:\n\n" + f"{missing_list}\n\n" + f"Installation location: {install_location}\n\n" + f"Would you like to install them now?\n\n" + f"This will run: pip install {' '.join(missing)}", + parent=parent_window) + + if install_choice: + success = self.install_ai_packages(missing, parent_window) + + if success: + messagebox.showinfo("Installation Complete", + f"✅ AI modules installed successfully!\n\n" + f"Provider: {provider_name}\n" + f"Location: {install_location}\n\n" + f"AI-assisted features are now ready to use.", + parent=parent_window) + else: + messagebox.showerror("Installation Failed", + f"❌ Failed to install AI modules.\n\n" + f"Please try installing manually:\n" + f"pip install {' '.join(missing)}", + parent=parent_window) + else: + messagebox.showinfo("Installation Skipped", + f"AI modules were not installed.\n\n" + f"You can install them later with:\n" + f"pip install {' '.join(missing)}", + parent=parent_window) + + except Exception as e: + if parent_window: + messagebox.showerror("Error", + f"Error checking AI modules: {str(e)}", + parent=parent_window) + if self.logger: + self.logger.log(f"Error in AI modules check: {str(e)}") + + def create_ai_provider(self, provider_name: str, api_key: str): + """Create an AI provider instance""" + if not AI_PROVIDERS_AVAILABLE: + return None + + try: + ai_logger = Logger(self.log) + return create_ai_provider(provider_name, api_key, ai_logger) + except Exception as e: + self.log(f"Error creating AI provider: {e}") + return None + + def create_local_git_manager(self, github_token: str): + """Create a LocalGitManager instance""" + if not AI_PROVIDERS_AVAILABLE: + return None + + try: + ai_logger = Logger(self.log) + return LocalGitManager(ai_logger, github_token) + except Exception as e: + self.log(f"Error creating LocalGitManager: {e}") + return None + + def get_last_diff_content(self) -> str: + """Get the last generated diff content for display in the UI""" + return self.last_diff_content + + def clear_diff_content(self): + """Clear the stored diff content""" + self.last_diff_content = "" \ No newline at end of file diff --git a/application/app_components/azure_devops_api.py b/application/app_components/azure_devops_api.py new file mode 100644 index 0000000..7a66ceb --- /dev/null +++ b/application/app_components/azure_devops_api.py @@ -0,0 +1,213 @@ +""" +Azure DevOps API Manager +Handles Azure DevOps REST API operations for work items +""" + +import base64 +import json +import re +import requests +from typing import List, Dict, Any, Tuple +from urllib.parse import urlparse, parse_qs + +# User agent for Azure DevOps API requests +USER_AGENT = "azure-devops-github-processor/2.0" + + +class AzureDevOpsAPI: + """Azure DevOps REST API client""" + + def __init__(self, organization: str, pat_token: str, logger=None): + self.organization = organization + self.pat_token = pat_token + self.logger = logger + self.base_url = f"https://dev.azure.com/{organization}" + self.api_version = "7.0" + + def log(self, message: str) -> None: + """Log a message""" + if self.logger: + self.logger.log(message) + else: + print(message) + + def _headers(self): + """Get headers for Azure DevOps API requests""" + return { + "Authorization": f"Basic {base64.b64encode(f':{self.pat_token}'.encode()).decode()}", + "Content-Type": "application/json-patch+json", + "User-Agent": USER_AGENT + } + + def parse_query_url(self, url: str) -> Tuple[str, str, str]: + """Parse Azure DevOps query URL to extract org, project, and query ID + + Supports both URL formats: + 1. https://dev.azure.com/organization/project/_queries/query/12345/ + 2. https://organization.visualstudio.com/project/_queries/query/12345/ + """ + parsed_url = urlparse(url) + + # Check for dev.azure.com format + if 'dev.azure.com' in parsed_url.netloc: + path_parts = parsed_url.path.strip('/').split('/') + + if len(path_parts) < 5: + raise ValueError("Invalid query URL format for dev.azure.com") + + organization = path_parts[0] + project = path_parts[1] + + # Check for visualstudio.com format + elif 'visualstudio.com' in parsed_url.netloc: + # Extract organization from subdomain (e.g., msft-skilling.visualstudio.com) + hostname_parts = parsed_url.netloc.split('.') + if len(hostname_parts) < 3 or hostname_parts[1] != 'visualstudio': + raise ValueError("Invalid visualstudio.com URL format") + + organization = hostname_parts[0] + + path_parts = parsed_url.path.strip('/').split('/') + + if len(path_parts) < 4: + raise ValueError("Invalid query URL format for visualstudio.com") + + project = path_parts[0] + + else: + raise ValueError("URL must be from dev.azure.com or visualstudio.com") + + # Find query ID in the URL (same logic for both formats) + query_id = None + if '_queries/query/' in url: + # Extract query ID from path + for i, part in enumerate(path_parts): + if part == 'query' and i > 0 and path_parts[i-1] == '_queries': + if i + 1 < len(path_parts): + query_id = path_parts[i + 1] + break + elif 'queryId=' in url: + match = re.search(r'queryId=([^&]+)', url) + if match: + query_id = match.group(1) + + if not query_id: + raise ValueError("Could not extract query ID from URL") + + return organization, project, query_id + + def execute_query(self, org: str, project: str, query_id: str, token: str) -> List[Dict[str, Any]]: + """Execute Azure DevOps query and return work items""" + # Build API URL for query execution + api_url = f"https://dev.azure.com/{org}/{project}/_apis/wit/wiql/{query_id}?api-version=6.0" + + # Prepare headers + auth_string = base64.b64encode(f":{token}".encode()).decode() + headers = { + 'Authorization': f'Basic {auth_string}', + 'Content-Type': 'application/json', + 'User-Agent': USER_AGENT + } + + # Execute query + self.log(f"Executing query at: {api_url}") + response = requests.get(api_url, headers=headers, timeout=30) + + if response.status_code != 200: + raise RuntimeError(f"Query execution failed: {response.status_code} - {response.text}") + + query_result = response.json() + work_item_refs = query_result.get('workItems', []) + + if not work_item_refs: + self.log("No work items found in query result") + return [] + + # Get detailed work item data + work_item_ids = [str(item['id']) for item in work_item_refs] + ids_param = ','.join(work_item_ids) + + details_url = f"https://dev.azure.com/{org}/{project}/_apis/wit/workitems?ids={ids_param}&api-version=6.0" + + self.log(f"Fetching details for {len(work_item_ids)} work items") + details_response = requests.get(details_url, headers=headers, timeout=30) + + if details_response.status_code != 200: + raise RuntimeError(f"Work item details fetch failed: {details_response.status_code}") + + return details_response.json().get('value', []) + + def _get_work_items_details(self, organization: str, work_item_ids: List[str], pat_token: str) -> List[Dict[str, Any]]: + """Get detailed information for work items""" + try: + # Build batch request URL + ids_param = ','.join(work_item_ids) + details_url = f"https://dev.azure.com/{organization}/_apis/wit/workitems" + + headers = { + "Authorization": f"Basic {self._encode_pat(pat_token)}", + "Content-Type": "application/json", + "Accept": "application/json", + "User-Agent": USER_AGENT + } + + params = { + "ids": ids_param, + "api-version": self.api_version, + "$expand": "fields" + } + + self.log(f"Fetching details for work items: {ids_param}") + response = requests.get(details_url, headers=headers, params=params, timeout=30) + response.raise_for_status() + + result = response.json() + work_items = result.get('value', []) + + self.log(f"Retrieved details for {len(work_items)} work item(s)") + return work_items + + except requests.RequestException as e: + raise Exception(f"Network error fetching work item details: {str(e)}") + except Exception as e: + raise Exception(f"Error fetching work item details: {str(e)}") + + def add_github_link_to_work_item(self, work_item_id: str, github_url: str, link_title: str = "GitHub Issue"): + """Add a GitHub issue/PR link to an Azure DevOps work item""" + self.log(f"Adding GitHub link to work item #{work_item_id}: {github_url}") + + url = f"{self.base_url}/_apis/wit/workitems/{work_item_id}?api-version=7.0" + + patch_document = [ + { + "op": "add", + "path": "/relations/-", + "value": { + "rel": "Hyperlink", + "url": github_url, + "attributes": { + "comment": link_title + } + } + } + ] + + try: + response = requests.patch(url, headers=self._headers(), json=patch_document, timeout=30) + if response.status_code == 200: + self.log(f"✅ Successfully linked GitHub resource to work item #{work_item_id}") + return True + else: + self.log(f"❌ Failed to link GitHub resource: {response.status_code} - {response.text}") + return False + except Exception as e: + self.log(f"❌ Exception linking GitHub resource: {str(e)}") + return False + + def _encode_pat(self, pat_token: str) -> str: + """Encode PAT token for Basic authentication""" + import base64 + # For Azure DevOps, username can be empty, just use :token + credentials = f":{pat_token}" + encoded = base64.b64encode(credentials.encode()).decode() + return encoded \ No newline at end of file diff --git a/application/app_components/cache_manager.py b/application/app_components/cache_manager.py new file mode 100644 index 0000000..dbd5a6d --- /dev/null +++ b/application/app_components/cache_manager.py @@ -0,0 +1,185 @@ +""" +Cache Manager for Work Items and UUF Items +Stores fetched items in temporary cache to avoid reloading on every app start +""" + +import json +import os +import tempfile +import time +from pathlib import Path +from typing import List, Dict, Any, Optional +from hashlib import md5 + + +class CacheManager: + """Manages caching of work items and UUF items""" + + def __init__(self, cache_duration_hours: int = 24): + """ + Initialize cache manager + + Args: + cache_duration_hours: How long cache is valid (default 24 hours) + """ + self.cache_duration_seconds = cache_duration_hours * 3600 + self.cache_dir = Path(tempfile.gettempdir()) / "devops_to_github_cache" + self.cache_dir.mkdir(exist_ok=True) + + def _get_cache_key(self, source_type: str, identifier: str) -> str: + """Generate cache key from source type and identifier""" + # Use MD5 hash to create safe filename + key_str = f"{source_type}_{identifier}" + return md5(key_str.encode()).hexdigest() + + def _get_cache_path(self, cache_key: str) -> Path: + """Get full path to cache file""" + return self.cache_dir / f"{cache_key}.json" + + def is_cache_valid(self, source_type: str, identifier: str) -> bool: + """Check if cache exists and is still valid""" + cache_key = self._get_cache_key(source_type, identifier) + cache_path = self._get_cache_path(cache_key) + + if not cache_path.exists(): + return False + + # Check if cache has expired + file_age = time.time() - cache_path.stat().st_mtime + return file_age < self.cache_duration_seconds + + def load_from_cache(self, source_type: str, identifier: str) -> Optional[List[Dict[str, Any]]]: + """ + Load work items from cache + + Args: + source_type: 'azure_devops' or 'uuf' + identifier: query URL hash or config hash + + Returns: + List of work items if cache is valid, None otherwise + """ + if not self.is_cache_valid(source_type, identifier): + return None + + cache_key = self._get_cache_key(source_type, identifier) + cache_path = self._get_cache_path(cache_key) + + try: + with open(cache_path, 'r', encoding='utf-8') as f: + cache_data = json.load(f) + + # Validate cache structure + if 'timestamp' not in cache_data or 'items' not in cache_data: + return None + + return cache_data['items'] + + except Exception as e: + print(f"Error loading cache: {e}") + return None + + def save_to_cache(self, source_type: str, identifier: str, items: List[Dict[str, Any]]) -> bool: + """ + Save work items to cache + + Args: + source_type: 'azure_devops' or 'uuf' + identifier: query URL hash or config hash + items: List of work items to cache + + Returns: + True if successful, False otherwise + """ + cache_key = self._get_cache_key(source_type, identifier) + cache_path = self._get_cache_path(cache_key) + + try: + cache_data = { + 'timestamp': time.time(), + 'source_type': source_type, + 'identifier': identifier, + 'items': items + } + + with open(cache_path, 'w', encoding='utf-8') as f: + json.dump(cache_data, f, indent=2, ensure_ascii=False) + + return True + + except Exception as e: + print(f"Error saving cache: {e}") + return False + + def invalidate_cache(self, source_type: str = None, identifier: str = None): + """ + Invalidate (delete) cache + + Args: + source_type: If specified, only invalidate this source type + identifier: If specified, only invalidate this specific cache + """ + if source_type and identifier: + # Invalidate specific cache + cache_key = self._get_cache_key(source_type, identifier) + cache_path = self._get_cache_path(cache_key) + if cache_path.exists(): + cache_path.unlink() + elif source_type: + # Invalidate all caches for this source type + for cache_file in self.cache_dir.glob("*.json"): + try: + with open(cache_file, 'r', encoding='utf-8') as f: + cache_data = json.load(f) + if cache_data.get('source_type') == source_type: + cache_file.unlink() + except: + pass + else: + # Invalidate all caches + for cache_file in self.cache_dir.glob("*.json"): + cache_file.unlink() + + def get_cache_info(self) -> Dict[str, Any]: + """Get information about cached items""" + cache_files = list(self.cache_dir.glob("*.json")) + + info = { + 'cache_dir': str(self.cache_dir), + 'total_files': len(cache_files), + 'total_size_bytes': sum(f.stat().st_size for f in cache_files), + 'caches': [] + } + + for cache_file in cache_files: + try: + with open(cache_file, 'r', encoding='utf-8') as f: + cache_data = json.load(f) + + file_age = time.time() - cache_file.stat().st_mtime + is_valid = file_age < self.cache_duration_seconds + + info['caches'].append({ + 'source_type': cache_data.get('source_type', 'unknown'), + 'item_count': len(cache_data.get('items', [])), + 'age_hours': round(file_age / 3600, 1), + 'is_valid': is_valid, + 'size_kb': round(cache_file.stat().st_size / 1024, 1) + }) + except: + pass + + return info + + def cleanup_expired(self): + """Remove expired cache files""" + current_time = time.time() + removed_count = 0 + + for cache_file in self.cache_dir.glob("*.json"): + file_age = current_time - cache_file.stat().st_mtime + if file_age >= self.cache_duration_seconds: + cache_file.unlink() + removed_count += 1 + + return removed_count diff --git a/application/app_components/config_manager.py b/application/app_components/config_manager.py new file mode 100644 index 0000000..845e53b --- /dev/null +++ b/application/app_components/config_manager.py @@ -0,0 +1,304 @@ +""" +Configuration Manager +Handles loading/saving configuration from .env files and launch.json +""" + +import os +import json +from typing import Dict, Any, Optional + + +class ConfigManager: + """Manages application configuration from multiple sources""" + + def __init__(self): + self.config = self.load_configuration() + + def _get_default_config(self) -> Dict[str, Any]: + """Get default configuration values""" + return { + 'AZURE_DEVOPS_QUERY': None, + 'AZURE_DEVOPS_PAT': None, + 'GITHUB_PAT': None, + 'GITHUB_REPO': None, + 'FORKED_REPO': None, # User's fork repository + 'AI_PROVIDER': None, + 'CLAUDE_API_KEY': None, + 'OPENAI_API_KEY': None, + 'GITHUB_TOKEN': None, # For GitHub Copilot AI Provider + 'LOCAL_REPO_PATH': None, + 'DRY_RUN': 'false', + 'DATAVERSE_ENVIRONMENT_URL': None, + 'DATAVERSE_TABLE_NAME': None, + 'AZURE_AD_CLIENT_ID': None, + 'AZURE_AD_CLIENT_SECRET': None, + 'AZURE_AD_TENANT_ID': None, + 'CUSTOM_INSTRUCTIONS': None # Custom AI instructions + } + + def load_configuration(self) -> Dict[str, Any]: + """Load configuration from launch.json first, then .env as fallback""" + config = self._get_default_config() + launch_json_keys = set() + + # First, try to load from launch.json + launch_json_path = os.path.join('.vscode', 'launch.json') + if os.path.exists(launch_json_path): + try: + with open(launch_json_path, 'r', encoding='utf-8') as f: + launch_data = json.load(f) + + # Look for configurations with env variables + for configuration in launch_data.get('configurations', []): + env_vars = configuration.get('env', {}) + for key in config.keys(): + if key in env_vars and env_vars[key] and not env_vars[key].startswith('<'): + config[key] = env_vars[key] + launch_json_keys.add(key) + + if launch_json_keys: + print(f"Loaded configuration from launch.json: {launch_json_path}") + + except (json.JSONDecodeError, FileNotFoundError, KeyError) as e: + print(f"Could not load launch.json: {e}") + + # Check if .env file exists, create default if not + if not os.path.exists('.env'): + print("No .env file found. Creating default .env file...") + self._create_default_env_file(config) + + # Load values from .env file (but don't override launch.json values) + if os.path.exists('.env'): + try: + env_loaded = False + with open('.env', 'r', encoding='utf-8') as f: + for line in f: + line = line.strip() + if line and not line.startswith('#') and '=' in line: + key, value = line.split('=', 1) + key = key.strip() + value = value.strip().strip('"').strip("'") + + # Load from .env if key exists in config and wasn't loaded from launch.json + if key in config and key not in launch_json_keys: + config[key] = value if value else '' + env_loaded = True + + if env_loaded: + print("Loaded configuration from .env file") + elif not launch_json_keys: + print("Configuration files found but no valid values loaded") + + except FileNotFoundError: + print("No .env file found") + except Exception as e: + print(f"Could not load .env file: {e}") + + # Ensure all config values are strings, not None + for key in config: + if config[key] is None: + config[key] = '' + + # Special handling for AI_PROVIDER - default to 'none' if empty + if not config.get('AI_PROVIDER'): + config['AI_PROVIDER'] = 'none' + + # Debug output + loaded_from = [] + for key, value in config.items(): + if value: + loaded_from.append(f"{key}: {'loaded' if value else 'not found'}") + + if loaded_from: + print(f"Configuration status: {', '.join(loaded_from)}") + else: + print("No configuration values loaded - all fields will be blank") + + self.config = config + return config + + def _create_default_env_file(self, config: Dict[str, Any]) -> None: + """Create a default .env file with empty values""" + try: + env_template = """# Azure DevOps to GitHub Tool Configuration +# Generated automatically - fill in your values +# IMPORTANT: Do NOT commit this file to source control. Add it to .gitignore. + +# Azure DevOps Configuration +AZURE_DEVOPS_QUERY= +AZURE_DEVOPS_PAT= + +# GitHub Configuration +GITHUB_PAT= +GITHUB_REPO= +FORKED_REPO= + +# Application Settings +DRY_RUN=false + +# AI Provider Configuration (for local PR creation with AI assistance) +AI_PROVIDER= +CLAUDE_API_KEY= +OPENAI_API_KEY= +GITHUB_TOKEN= +LOCAL_REPO_PATH= + +# PowerApp/Dataverse Configuration (for UUF items - optional) +DATAVERSE_ENVIRONMENT_URL= +DATAVERSE_TABLE_NAME= +AZURE_AD_CLIENT_ID= +AZURE_AD_CLIENT_SECRET= +AZURE_AD_TENANT_ID= + +# Custom AI Instructions (optional) +CUSTOM_INSTRUCTIONS= +""" + + with open('.env', 'w', encoding='utf-8') as f: + f.write(env_template) + + print("Created default .env file with blank values") + + except Exception as e: + print(f"Error creating default .env file: {e}") + + def save_configuration(self, config_values: Dict[str, Any]) -> bool: + """Save configuration to .env file""" + try: + print(f"DEBUG: Saving config values: {config_values}") + print(f"DEBUG: AI_PROVIDER value being saved: '{config_values.get('AI_PROVIDER', 'NOT_FOUND')}'") + + # Update internal config + for key, value in config_values.items(): + if key in self.config: + old_value = self.config[key] + new_value = value or '' + self.config[key] = new_value + if key == 'AI_PROVIDER': + print(f"DEBUG: Updated AI_PROVIDER from '{old_value}' to '{new_value}'") + + # Build .env file content + env_content = [] + env_content.append("# Azure DevOps to GitHub Tool Configuration") + env_content.append("# Generated by Settings Dialog") + env_content.append("# IMPORTANT: Do NOT commit this file to source control. Add it to .gitignore.") + env_content.append("") + + env_content.append("# Azure DevOps Configuration") + env_content.append(f"AZURE_DEVOPS_QUERY={self.config.get('AZURE_DEVOPS_QUERY', '')}") + env_content.append(f"AZURE_DEVOPS_PAT={self.config.get('AZURE_DEVOPS_PAT', '')}") + env_content.append("") + + env_content.append("# GitHub Configuration") + env_content.append(f"GITHUB_PAT={self.config.get('GITHUB_PAT', '')}") + env_content.append(f"GITHUB_REPO={self.config.get('GITHUB_REPO', '')}") + env_content.append(f"FORKED_REPO={self.config.get('FORKED_REPO', '')}") + env_content.append("") + + env_content.append("# Application Settings") + dry_run_value = str(self.config.get('DRY_RUN', 'false')).lower() + env_content.append(f"DRY_RUN={dry_run_value}") + env_content.append("") + + env_content.append("# AI Provider Configuration (for local PR creation with AI assistance)") + ai_provider_value = self.config.get('AI_PROVIDER', '') + print(f"DEBUG: Writing AI_PROVIDER to file: '{ai_provider_value}'") + env_content.append(f"AI_PROVIDER={ai_provider_value}") + env_content.append(f"CLAUDE_API_KEY={self.config.get('CLAUDE_API_KEY', '')}") + env_content.append(f"OPENAI_API_KEY={self.config.get('OPENAI_API_KEY', '')}") + env_content.append(f"GITHUB_TOKEN={self.config.get('GITHUB_TOKEN', '')}") + env_content.append(f"LOCAL_REPO_PATH={self.config.get('LOCAL_REPO_PATH', '')}") + env_content.append("") + + env_content.append("# PowerApp/Dataverse Configuration (for UUF items - optional)") + env_content.append(f"DATAVERSE_ENVIRONMENT_URL={self.config.get('DATAVERSE_ENVIRONMENT_URL', '')}") + env_content.append(f"DATAVERSE_TABLE_NAME={self.config.get('DATAVERSE_TABLE_NAME', '')}") + env_content.append(f"AZURE_AD_CLIENT_ID={self.config.get('AZURE_AD_CLIENT_ID', '')}") + env_content.append(f"AZURE_AD_CLIENT_SECRET={self.config.get('AZURE_AD_CLIENT_SECRET', '')}") + env_content.append(f"AZURE_AD_TENANT_ID={self.config.get('AZURE_AD_TENANT_ID', '')}") + env_content.append("") + + env_content.append("# Custom AI Instructions (optional)") + env_content.append(f"CUSTOM_INSTRUCTIONS={self.config.get('CUSTOM_INSTRUCTIONS', '')}") + env_content.append("") + + # Write to file + with open('.env', 'w', encoding='utf-8') as f: + f.write('\n'.join(env_content)) + + print("Configuration saved to .env file") + return True + + except Exception as e: + print(f"Error saving configuration: {e}") + return False + + def get_config(self) -> Dict[str, Any]: + """Get current configuration with automatic GITHUB_TOKEN defaulting""" + config = self.config.copy() + + # Auto-default GITHUB_TOKEN to GITHUB_PAT if GITHUB_TOKEN is empty or None + github_token = config.get('GITHUB_TOKEN', '').strip() if config.get('GITHUB_TOKEN') else '' + github_pat = config.get('GITHUB_PAT', '').strip() if config.get('GITHUB_PAT') else '' + + if not github_token and github_pat: + config['GITHUB_TOKEN'] = github_pat + + return config + + def get_value(self, key: str, default: Any = None) -> Any: + """Get a specific configuration value""" + return self.config.get(key, default) + + def get(self, key: str, default: Any = None) -> Any: + """Get a specific configuration value (dictionary-like interface)""" + return self.config.get(key, default) + + def set_value(self, key: str, value: Any) -> None: + """Set a specific configuration value""" + if key in self.config: + self.config[key] = value + + def get_pr_counter_file(self) -> str: + """Get the path to the PR counter file""" + script_dir = os.path.dirname(os.path.abspath(__file__)) + return os.path.join(script_dir, '..', '.pr_counter.json') + + def load_pr_counter(self) -> Dict[str, int]: + """Load the PR counter from file""" + counter_file = self.get_pr_counter_file() + if os.path.exists(counter_file): + try: + with open(counter_file, 'r', encoding='utf-8') as f: + return json.load(f) + except (json.JSONDecodeError, FileNotFoundError): + pass + return {} + + def save_pr_counter(self, counter: Dict[str, int]) -> None: + """Save the PR counter to file""" + counter_file = self.get_pr_counter_file() + try: + # Ensure directory exists + os.makedirs(os.path.dirname(counter_file), exist_ok=True) + with open(counter_file, 'w', encoding='utf-8') as f: + json.dump(counter, f, indent=2) + except Exception as e: + print(f"Warning: Could not save PR counter: {e}") + + def get_next_pr_number(self, provider_key: str) -> int: + """ + Get the next PR number for a given provider. + + Args: + provider_key: Either the AI provider name ('chatgpt', 'claude') or 'gh_copilot' + + Returns: + The next PR number for this provider + """ + counter = self.load_pr_counter() + current_number = counter.get(provider_key, 0) + next_number = current_number + 1 + counter[provider_key] = next_number + self.save_pr_counter(counter) + return next_number \ No newline at end of file diff --git a/application/app_components/dataverse_api.py b/application/app_components/dataverse_api.py new file mode 100644 index 0000000..862450a --- /dev/null +++ b/application/app_components/dataverse_api.py @@ -0,0 +1,255 @@ +""" +Dataverse API Manager +Handles PowerApp/Dataverse operations for UUF items +""" + +import json +import re +import requests +import urllib.parse +from typing import List, Dict, Any, Optional +from urllib.parse import urlparse + +# Constants +USER_AGENT = "azure-devops-github-processor/2.0" + + +class DataverseAPI: + """Dataverse/PowerApp API client for UUF items""" + + def __init__(self, environment_url: str, table_name: str, logger=None, config: dict = None): + self.environment_url = environment_url.rstrip('/') + self.table_name = table_name + self.logger = logger + self.config = config or {} + self.access_token = None + self.api_version = "v9.2" + + def log(self, message: str) -> None: + """Log a message""" + if self.logger: + self.logger.log(message) + else: + print(message) + + def authenticate(self, client_id: str, client_secret: str, tenant_id: str) -> bool: + """Authenticate with Azure AD and get access token""" + try: + # Azure AD token endpoint + token_url = f"https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token" + + # Prepare request data + data = { + 'grant_type': 'client_credentials', + 'client_id': client_id, + 'client_secret': client_secret, + 'scope': f"{self.environment_url}/.default" + } + + headers = { + 'Content-Type': 'application/x-www-form-urlencoded' + } + + self.log("Authenticating with Azure AD...") + response = requests.post(token_url, data=data, headers=headers, timeout=30) + response.raise_for_status() + + token_data = response.json() + self.access_token = token_data['access_token'] + + self.log("✅ Successfully authenticated with Azure AD") + return True + + except requests.RequestException as e: + self.log(f"❌ Network error during authentication: {str(e)}") + return False + except KeyError as e: + self.log(f"❌ Invalid token response: {str(e)}") + return False + except Exception as e: + self.log(f"❌ Authentication error: {str(e)}") + return False + + def _headers(self): + """Get headers for Dataverse API requests""" + return { + "Authorization": f"Bearer {self.access_token}", + "OData-MaxVersion": "4.0", + "OData-Version": "4.0", + "Accept": "application/json", + "Content-Type": "application/json", + "User-Agent": USER_AGENT + } + + def fetch_uuf_items(self, filter_query: Optional[str] = None) -> List[Dict[str, Any]]: + """Fetch UUF items from Dataverse""" + try: + if not self.access_token: + raise RuntimeError("Not authenticated. Call authenticate() first.") + + self.log(f"Fetching UUF items from table: {self.table_name}") + + # Build API URL + api_url = f"{self.environment_url}/api/data/{self.api_version}/{self.table_name}" + + # Add filter if provided + if filter_query: + api_url += f"?$filter={urllib.parse.quote(filter_query)}" + + response = requests.get(api_url, headers=self._headers(), timeout=60) + + if response.status_code != 200: + raise RuntimeError(f"Failed to fetch UUF items: {response.status_code} - {response.text}") + + data = response.json() + items = data.get('value', []) + + self.log(f"✅ Fetched {len(items)} UUF items from Dataverse") + return items + + except Exception as e: + self.log(f"❌ Error fetching UUF items: {str(e)}") + raise + + def process_uuf_item(self, uuf_item: dict) -> dict | None: + """Process a single UUF item from Dataverse/PowerApp + + UUF items may have different field names than Azure DevOps work items. + Adjust the field mapping based on your actual Dataverse table schema. + """ + try: + # Extract UUF item ID (adjust field names as needed) + uuf_id = uuf_item.get('cr4af_uufid') or uuf_item.get('cr4af_name') or uuf_item.get('cr_uufitemid') or 'unknown' + + # Extract title + title = uuf_item.get('cr4af_title') or uuf_item.get('cr4af_subject') or uuf_item.get('cr_title') or 'No Title' + + # Extract description/details + description = uuf_item.get('cr4af_description') or uuf_item.get('cr4af_details') or uuf_item.get('cr_description') or '' + + if not description: + self.log(f"UUF item {uuf_id} has no description, skipping") + return None + + # Extract document URL + doc_url = uuf_item.get('cr4af_documenturl') or uuf_item.get('cr4af_docurl') or uuf_item.get('cr_documenturl') or '' + + if not doc_url: + self.log(f"UUF item {uuf_id} has no document URL, skipping") + return None + + # Extract text to change and new text + text_to_change = uuf_item.get('cr4af_texttochange') or uuf_item.get('cr4af_currenttext') or uuf_item.get('cr_currenttext') or '' + new_text = uuf_item.get('cr4af_proposednewtext') or uuf_item.get('cr4af_newtext') or uuf_item.get('cr_newtext') or '' + + if not text_to_change or not new_text: + self.log(f"UUF item {uuf_id} missing text fields, skipping") + return None + + # Extract GitHub info from document URL + github_info = self._extract_github_info(doc_url) + + # If the document does not include an original_content_git_url, skip this item + if not github_info.get('original_content_git_url'): + self.log(f"UUF item {uuf_id} skipped: original_content_git_url not found in document {doc_url}") + return None + + processed_item = { + 'id': uuf_id, + 'title': title, + 'nature_of_request': 'UUF Item - Modify existing docs', + 'mydoc_url': doc_url, + 'text_to_change': text_to_change, + 'new_text': new_text, + 'github_info': github_info, + 'status': 'Ready', + 'original_new_text': new_text, + 'source': 'UUF' # Mark as UUF item + } + + self.log(f"Successfully processed UUF item {uuf_id}") + return processed_item + + except Exception as e: + self.log(f"Error processing UUF item {uuf_item.get('cr4af_uufid', 'unknown')}: {str(e)}") + return None + + def _extract_github_info(self, doc_url: str) -> dict: + """Extract GitHub repository info and ms.author from document URL + + If GITHUB_REPO is configured in .env, it will be used instead of the repo + extracted from the document metadata. This allows you to create PRs in your + fork while preserving the file path and ms.author from the original document. + """ + try: + # Fetch the document + headers = {'User-Agent': USER_AGENT} + response = requests.get(doc_url, headers=headers, timeout=30) + response.raise_for_status() + + html = response.text + + # Extract ms.author + ms_author = self._extract_meta_tag(html, 'ms.author') + + # Extract original_content_git_url + original_content_git_url = self._extract_meta_tag(html, 'original_content_git_url') + + if not original_content_git_url: + # Try alternative extraction method + match = re.search(r"original_content_git_url[\"\']?\s*[:=]\s*[\"\']([^\"']+)[\"']", html, re.IGNORECASE) + if match: + original_content_git_url = match.group(1).strip() + + if not original_content_git_url: + raise ValueError("original_content_git_url not found in document") + + # Check if GITHUB_REPO is configured in .env + # If it is, use that instead of the repo from the document + configured_repo = self.config.get('GITHUB_REPO') + + if configured_repo and '/' in configured_repo: + # Use the configured repository (e.g., "b-tsammons/fabric-docs-pr") + parts = configured_repo.split('/', 1) + owner = parts[0].strip() + repo = parts[1].strip() + self.log(f"Using configured GITHUB_REPO: {owner}/{repo} (overriding document metadata)") + else: + # Parse GitHub owner/repo from original_content_git_url (fallback to document metadata) + owner, repo = self._parse_github_url(original_content_git_url) + self.log(f"Using repository from document metadata: {owner}/{repo}") + + return { + 'ms_author': ms_author, + 'original_content_git_url': original_content_git_url, + 'owner': owner, + 'repo': repo + } + + except Exception as e: + self.log(f"Error extracting GitHub info from {doc_url}: {str(e)}") + return { + 'ms_author': None, + 'original_content_git_url': None, + 'owner': None, + 'repo': None, + 'error': str(e) + } + + def _extract_meta_tag(self, html: str, name: str) -> str | None: + """Extract content from meta tag""" + pattern = rf']*?\s)?(?:name|property)\s*=\s*["\'](?P{re.escape(name)})["\']\s+[^>]*?\bcontent\s*=\s*["\'](?P[^"\']+)["\'][^>]*?>' + match = re.search(pattern, html, re.IGNORECASE) + if match: + return match.group('content').strip() + return None + + def _parse_github_url(self, url: str) -> tuple[str, str]: + """Parse GitHub URL to extract owner and repo""" + parsed = urlparse(url) + if "github.com" not in parsed.netloc.lower(): + raise ValueError(f"Not a GitHub URL: {url}") + parts = [p for p in parsed.path.split("/") if p] + if len(parts) < 2: + raise ValueError(f"Unable to parse owner/repo from: {url}") + return parts[0], parts[1] \ No newline at end of file diff --git a/application/app_components/github_api.py b/application/app_components/github_api.py new file mode 100644 index 0000000..4024c3b --- /dev/null +++ b/application/app_components/github_api.py @@ -0,0 +1,992 @@ +""" +GitHub API Manager +Handles GitHub GraphQL operations, PR/Issue creation, and Copilot interactions +""" + +import base64 +import difflib +import json +import requests +from typing import Optional, Tuple, Dict, Any, List +from urllib.parse import urlparse + +# Constants +GITHUB_GRAPHQL_ENDPOINT = "https://api.github.com/graphql" +USER_AGENT = "azure-devops-github-processor/2.0" + + +class GitHubGQL: + """GitHub GraphQL API client for creating issues, PRs, and managing assignments""" + + def __init__(self, token: str, logger=None, dry_run: bool = False): + self.token = token + self.logger = logger + self.dry_run = dry_run + + def log(self, message: str) -> None: + """Log a message""" + if self.logger: + self.logger.log(message) + else: + print(message) + + def _headers(self): + """Get headers for GitHub API requests""" + return { + "Authorization": f"Bearer {self.token}", + "User-Agent": USER_AGENT, + "Content-Type": "application/json" + } + + def run(self, query: str, variables: dict | None = None) -> dict: + """Execute a GraphQL query""" + payload = {"query": query, "variables": variables or {}} + + if self.dry_run: + self.log("[DRY-RUN] Would POST GraphQL payload:") + pretty = json.dumps(payload, indent=2) + self.log(pretty) + return {"dryRun": True, "data": None} + + try: + resp = requests.post(GITHUB_GRAPHQL_ENDPOINT, headers=self._headers(), json=payload, timeout=60) + if resp.status_code != 200: + raise RuntimeError(f"GraphQL HTTP {resp.status_code}: {resp.text}") + + data = resp.json() + if "errors" in data and data["errors"]: + raise RuntimeError(f"GraphQL errors: {json.dumps(data['errors'], indent=2)}") + + return data + except requests.RequestException as e: + raise RuntimeError(f"Request failed: {str(e)}") + + def _make_rest_request(self, method: str, url: str, data: Dict[str, Any] = None) -> Dict[str, Any]: + """Make a REST API request to GitHub""" + headers = { + "Authorization": f"Bearer {self.token}", + "Accept": "application/vnd.github+json", + "User-Agent": USER_AGENT + } + + if self.dry_run: + self.log(f"[DRY-RUN] Would make {method} request to: {url}") + return {"number": 123, "html_url": "https://github.com/example/repo/pull/123"} + + response = requests.request(method, url, headers=headers, json=data, timeout=30) + response.raise_for_status() + + return response.json() + + def get_repo_id(self, owner: str, name: str) -> str: + """Get GitHub repository ID""" + self.log(f"Fetching repositoryId for {owner}/{name}...") + query = """ + query($owner:String!, $name:String!) { + repository(owner:$owner, name:$name) { + id + url + } + } + """ + data = self.run(query, {"owner": owner, "name": name}) + + if data.get("dryRun"): + return "DRY_RUN_REPO_ID" + + repo = data["data"]["repository"] + if not repo: + raise RuntimeError(f"Repository {owner}/{name} not found or token lacks access.") + + self.log(f"Repository ID: {repo['id']} ({repo['url']})") + return repo["id"] + + def get_copilot_actor_id(self, owner: str, name: str) -> tuple[str | None, str | None]: + """Find Copilot actor ID for assignment""" + self.log("Querying suggestedActors for CAN_BE_ASSIGNED...") + query = """ + query($owner:String!, $name:String!) { + repository(owner:$owner, name:$name) { + suggestedActors(capabilities:[CAN_BE_ASSIGNED], first:100) { + nodes { + login + __typename + ... on Bot { id } + ... on User { id } + } + } + } + } + """ + data = self.run(query, {"owner": owner, "name": name}) + + if data.get("dryRun"): + return ("DRY_RUN_ACTOR_ID", "copilot-swe-agent") + + nodes = data["data"]["repository"]["suggestedActors"]["nodes"] + if not nodes: + self.log("No suggestedActors returned.") + return (None, None) + + # Log all available actors for debugging + self.log(f"Available assignable actors ({len(nodes)}):") + for node in nodes: + self.log(f" - {node.get('login', 'N/A')} ({node.get('__typename', 'N/A')}) ID: {node.get('id', 'N/A')}") + + # Prefer known Copilot logins + preferred = ("copilot-swe-agent", "copilot", "github-copilot", "github-advanced-security") + chosen = None + + # First, try exact matches + for candidate in nodes: + login = candidate.get("login", "").lower() + if login in preferred: + chosen = candidate + break + + # If no exact match, try partial matches + if not chosen: + for candidate in nodes: + login = candidate.get("login", "").lower() + if "copilot" in login: + chosen = candidate + break + + if not chosen: + self.log("Copilot not found in suggestedActors list.") + self.log("Available actors: " + ", ".join([n.get("login", "N/A") for n in nodes])) + return (None, None) + + login = chosen["login"] + actor_id = chosen.get("id") + + if not actor_id: + self.log(f"Warning: No actor ID found for {login}") + return (None, None) + + self.log(f"Found assignable Copilot actor: {login} (id={actor_id})") + return (actor_id, login) + + def create_issue(self, repository_id: str, title: str, body: str) -> tuple[str, str, int]: + """Create a GitHub issue""" + self.log("Creating issue with createIssue mutation...") + mutation = """ + mutation($repositoryId:ID!, $title:String!, $body:String!) { + createIssue(input:{repositoryId:$repositoryId, title:$title, body:$body}) { + issue { + id + url + number + title + } + } + } + """ + data = self.run(mutation, {"repositoryId": repository_id, "title": title, "body": body}) + + if data.get("dryRun"): + return ("DRY_RUN_ISSUE_ID", "https://github.com/owner/repo/issues/123", 123) + + issue = data["data"]["createIssue"]["issue"] + self.log(f"Issue created: {issue['url']} (#{issue['number']})") + return (issue["id"], issue["url"], issue["number"]) + + def create_branch_from_main(self, owner: str, repo: str, branch_name: str) -> bool: + """Create a new branch from the main branch""" + self.log(f"Creating branch '{branch_name}' in {owner}/{repo}") + + try: + # Get the SHA of the main branch + main_ref_url = f"https://api.github.com/repos/{owner}/{repo}/git/ref/heads/main" + main_ref_response = self._make_rest_request("GET", main_ref_url) + main_sha = main_ref_response["object"]["sha"] + + self.log(f"Main branch SHA: {main_sha}") + + # Create new branch + new_ref_url = f"https://api.github.com/repos/{owner}/{repo}/git/refs" + new_ref_data = { + "ref": f"refs/heads/{branch_name}", + "sha": main_sha + } + + if self.dry_run: + self.log(f"🧪 DRY RUN: Would create branch '{branch_name}' from main ({main_sha})") + return True + + self._make_rest_request("POST", new_ref_url, new_ref_data) + self.log(f"✅ Branch '{branch_name}' created successfully") + return True + + except Exception as e: + self.log(f"❌ Failed to create branch: {str(e)}") + return False + + def get_user_forks(self, include_org_repos: bool = True) -> List[str]: + """Get list of user's forked repositories""" + self.log("Fetching user's forked repositories...") + + if self.dry_run: + # Return sample data for dry run + return [ + "username/fabric-docs", + "username/azure-docs", + "username/powerbi-docs" + ] + + try: + forks = [] + page = 1 + per_page = 100 + + while page <= 5: # Limit to 5 pages to avoid long waits + url = f"https://api.github.com/user/repos?type=forks&per_page={per_page}&page={page}" + + response = self._make_rest_request("GET", url) + repos = response if isinstance(response, list) else response.get('data', []) + + if not repos: + break + + for repo in repos: + if repo.get('fork', False): + forks.append(f"{repo['owner']['login']}/{repo['name']}") + + if len(repos) < per_page: + break + + page += 1 + + self.log(f"Found {len(forks)} forked repositories") + return forks + + except Exception as e: + self.log(f"❌ Failed to fetch user forks: {str(e)}") + return [] + + def get_authenticated_user(self) -> Dict[str, Any]: + """Get authenticated user information""" + if self.dry_run: + return {"login": "dry-run-user", "name": "Dry Run User"} + + try: + return self._make_rest_request("GET", "https://api.github.com/user") + except Exception as e: + self.log(f"❌ Failed to get user info: {str(e)}") + return {} + + def fork_repository(self, owner: str, repo: str, target_org: str = None) -> tuple[str, str]: + """Fork a repository to the authenticated user's account or specified organization""" + self.log(f"Forking repository {owner}/{repo}") + + fork_url = f"https://api.github.com/repos/{owner}/{repo}/forks" + fork_data = {} + + if target_org: + fork_data["organization"] = target_org + + if self.dry_run: + # Get authenticated user for dry run + user_url = "https://api.github.com/user" + try: + user_data = self._make_rest_request("GET", user_url) + fork_owner = target_org if target_org else user_data["login"] + self.log(f"🧪 DRY RUN: Would fork {owner}/{repo} to {fork_owner}/{repo}") + return fork_owner, repo + except: + self.log(f"🧪 DRY RUN: Would fork {owner}/{repo}") + return "dry-run-user", repo + + try: + fork_response = self._make_rest_request("POST", fork_url, fork_data) + fork_owner = fork_response["owner"]["login"] + fork_name = fork_response["name"] + + self.log(f"✅ Repository forked to {fork_owner}/{fork_name}") + return fork_owner, fork_name + + except Exception as e: + self.log(f"❌ Failed to fork repository: {str(e)}") + raise + + def check_repository_exists(self, owner: str, repo: str) -> bool: + """Check if a repository exists and is accessible""" + try: + url = f"https://api.github.com/repos/{owner}/{repo}" + response = self._make_rest_request("GET", url) + return bool(response.get('id')) + except: + return False + + def find_matching_repositories(self, target_repo: str, fork_repo: str) -> Dict[str, List[str]]: + """Find matching repositories to suggest alternatives for mismatched repos""" + self.log(f"Finding matching repositories for target: {target_repo}, fork: {fork_repo}") + + if self.dry_run: + return { + "target_alternatives": ["microsoftdocs/fabric-docs-pr"], + "fork_alternatives": ["b-tsammons/azure-docs-pr"] + } + + try: + target_owner, target_name = target_repo.split('/', 1) if '/' in target_repo else ("", target_repo) + fork_owner, fork_name = fork_repo.split('/', 1) if '/' in fork_repo else ("", fork_repo) + + target_alternatives = [] + fork_alternatives = [] + + # Get authenticated user info + user_info = self.get_authenticated_user() + user_login = user_info.get('login', '') + + # Search for repositories with similar names + search_terms = [target_name, fork_name] + for term in search_terms: + if term: + # Clean up the search term (remove common suffixes) + clean_term = term.replace('-docs', '').replace('-pr', '').replace('_', ' ') + + # Search for repositories + search_url = f"https://api.github.com/search/repositories?q={clean_term}&per_page=20" + try: + search_response = self._make_rest_request("GET", search_url) + repositories = search_response.get('items', []) + + for repo_data in repositories: + repo_full_name = repo_data['full_name'] + repo_owner = repo_data['owner']['login'] + + # Check if this is a potential target alternative + if (repo_owner == target_owner and + repo_data['name'] != target_name and + repo_full_name not in target_alternatives): + target_alternatives.append(repo_full_name) + + # Check if this is a potential fork alternative + if (repo_owner == user_login and + repo_data['name'] != fork_name and + repo_data.get('fork', False) and + repo_full_name not in fork_alternatives): + fork_alternatives.append(repo_full_name) + + except Exception as e: + self.log(f"❌ Search failed for term '{term}': {str(e)}") + + return { + "target_alternatives": target_alternatives[:5], # Limit to 5 suggestions + "fork_alternatives": fork_alternatives[:5] + } + + except Exception as e: + self.log(f"❌ Failed to find matching repositories: {str(e)}") + return {"target_alternatives": [], "fork_alternatives": []} + + def make_documentation_change(self, owner: str, repo: str, branch_name: str, file_path: str, + old_text: str, new_text: str, commit_message: str) -> bool: + """Make actual documentation changes to a file in the repository + + This fetches the file, makes the text replacement, and commits it to the branch. + + Returns True if successful, False otherwise. + """ + if self.dry_run: + self.log(f"[DRY-RUN] Would update {file_path} in branch {branch_name}") + self.log(f"[DRY-RUN] Replace: {old_text[:50]}...") + self.log(f"[DRY-RUN] With: {new_text[:50]}...") + return True + + try: + rest_headers = { + "Authorization": f"Bearer {self.token}", + "Accept": "application/vnd.github+json", + "User-Agent": USER_AGENT + } + + # 1. Get the current file content from the branch + self.log(f"Fetching file: {file_path}") + file_url = f"https://api.github.com/repos/{owner}/{repo}/contents/{file_path}?ref={branch_name}" + resp = requests.get(file_url, headers=rest_headers, timeout=30) + + if resp.status_code == 404: + self.log(f"❌ File not found: {file_path}") + self.log(f" The file path might be incorrect or the file doesn't exist") + return False + + resp.raise_for_status() + file_data = resp.json() + + # Decode the file content + current_content = base64.b64decode(file_data["content"]).decode('utf-8') + file_sha = file_data["sha"] + + self.log(f"✅ File retrieved ({len(current_content)} bytes)") + + # Detect line ending style to preserve it + line_ending = '\r\n' if '\r\n' in current_content else '\n' + self.log(f"📝 Detected line endings: {'CRLF' if line_ending == '\\r\\n' else 'LF'}") + + # Normalize everything to LF for consistent processing + normalized_content = current_content.replace('\r\n', '\n') + normalized_old = old_text.replace('\r\n', '\n') + normalized_new = new_text.replace('\r\n', '\n') + + # 2. Make the text replacement + if normalized_old not in normalized_content: + self.log(f"⚠️ Warning: Could not find exact text to replace in {file_path}") + self.log(f" Searching for similar text...") + + # Try to find similar text (case-insensitive, whitespace-flexible) + lines = normalized_content.split('\n') + old_lines = normalized_old.split('\n') + + # Find the best matching sequence + matcher = difflib.SequenceMatcher(None, old_lines, lines) + match = matcher.find_longest_match(0, len(old_lines), 0, len(lines)) + + if match.size > len(old_lines) * 0.7: # If we find 70% match + self.log(f" Found similar text at line {match.b + 1}") + self.log(f" Making best-effort replacement...") + # This is a simplified approach - in production you'd want more sophisticated matching + else: + self.log(f"❌ Could not find text to replace. The document may have changed.") + self.log(f" Creating PR with instructions instead...") + return False + + # Replace the text (using normalized versions) + updated_content = normalized_content.replace(normalized_old, normalized_new) + + if updated_content == normalized_content: + self.log(f"⚠️ No changes made - text might not exist in file") + return False + + self.log(f"✅ Text replacement successful") + + # Restore original line endings + if line_ending == '\r\n': + updated_content = updated_content.replace('\n', '\r\n') + self.log(f"✅ Restored CRLF line endings") + + # 3. Commit the updated file + self.log(f"Committing changes to {file_path}...") + encoded_content = base64.b64encode(updated_content.encode('utf-8')).decode() + + update_payload = { + "message": commit_message, + "content": encoded_content, + "sha": file_sha, + "branch": branch_name + } + + update_url = f"https://api.github.com/repos/{owner}/{repo}/contents/{file_path}" + resp = requests.put(update_url, headers=rest_headers, json=update_payload, timeout=30) + resp.raise_for_status() + + self.log(f"✅ Changes committed to branch {branch_name}") + return True + + except requests.HTTPError as e: + self.log(f"❌ HTTP Error making changes: {e}") + if e.response.status_code == 403: + self.log(" Permission denied - token doesn't have write access") + elif e.response.status_code == 404: + self.log(f" File not found: {file_path}") + return False + except Exception as e: + self.log(f"❌ Error making changes: {str(e)}") + return False + + def create_cross_repo_pull_request(self, source_owner: str, source_repo: str, target_owner: str, target_repo: str, + title: str, body: str, head_ref: str, base_ref: str = "main") -> tuple[str, str, int]: + """Create a pull request from source repo to target repo""" + self.log(f"Creating cross-repository PR from {source_owner}/{source_repo}:{head_ref} to {target_owner}/{target_repo}:{base_ref}") + + # Get target repository ID + target_repo_id = self.get_repo_id(target_owner, target_repo) + + # Format the head reference for cross-repo PR + head_ref_full = f"{source_owner}:{head_ref}" + + mutation = """ + mutation($repositoryId:ID!, $title:String!, $body:String!, $headRefName:String!, $baseRefName:String!) { + createPullRequest(input:{ + repositoryId:$repositoryId, + title:$title, + body:$body, + headRefName:$headRefName, + baseRefName:$baseRefName + }) { + pullRequest { + id + url + number + } + } + } + """ + + variables = { + "repositoryId": target_repo_id, + "title": title, + "body": body, + "headRefName": head_ref_full, + "baseRefName": base_ref + } + + if self.dry_run: + self.log(f"🧪 DRY RUN: Would create cross-repo PR '{title}' from {head_ref_full} to {base_ref}") + return "dry-run-pr-id", f"https://github.com/{target_owner}/{target_repo}/pull/0", 0 + + try: + data = self.run(mutation, variables) + pr_data = data["data"]["createPullRequest"]["pullRequest"] + + pr_id = pr_data["id"] + pr_url = pr_data["url"] + pr_number = pr_data["number"] + + self.log(f"✅ Cross-repo pull request created: {pr_url}") + return pr_id, pr_url, pr_number + + except Exception as e: + self.log(f"❌ Failed to create cross-repo pull request: {str(e)}") + raise + + def create_pull_request(self, repository_id: str, title: str, body: str, head_ref: str, base_ref: str = "main") -> tuple[str, str, int]: + """Create a pull request with AB# linking""" + self.log(f"Creating pull request with createPullRequest mutation from {head_ref} to {base_ref}...") + mutation = """ + mutation($repositoryId:ID!, $title:String!, $body:String!, $headRefName:String!, $baseRefName:String!) { + createPullRequest(input:{ + repositoryId:$repositoryId, + title:$title, + body:$body, + headRefName:$headRefName, + baseRefName:$baseRefName + }) { + pullRequest { + id + url + number + title + } + } + } + """ + variables = { + "repositoryId": repository_id, + "title": title, + "body": body, + "headRefName": head_ref, + "baseRefName": base_ref + } + data = self.run(mutation, variables) + if data.get("dryRun"): + return ("DRY_RUN_PR_ID", "https://github.com/owner/repo/pull/456", 456) + pr = data["data"]["createPullRequest"]["pullRequest"] + self.log(f"Pull request created: {pr['url']} (#{pr['number']})") + return (pr["id"], pr["url"], pr["number"]) + + def assign_to_copilot(self, assignable_id: str, actor_ids: list[str]) -> bool: + """Assign issue to Copilot + + Returns True if successful, False otherwise. + """ + self.log("Assigning with replaceActorsForAssignable mutation...") + mutation = """ + mutation($assignableId:ID!, $actorIds:[ID!]!) { + replaceActorsForAssignable(input:{assignableId:$assignableId, actorIds:$actorIds}) { + assignable { + ... on Issue { + id + title + assignees(first:10) { nodes { login } } + url + } + ... on PullRequest { + id + title + assignees(first:10) { nodes { login } } + url + } + } + } + } + """ + try: + data = self.run(mutation, {"assignableId": assignable_id, "actorIds": actor_ids}) + + if data.get("dryRun"): + self.log("[DRY-RUN] Would have assigned Copilot.") + return True + + assigned = data["data"]["replaceActorsForAssignable"]["assignable"]["assignees"]["nodes"] + assignees = ", ".join([n["login"] for n in assigned]) or "(none)" + self.log(f"Current assignees: {assignees}") + return True + except Exception as e: + error_message = str(e) + self.log(f"Error assigning Copilot: {error_message}") + + # Provide specific guidance for common permission issues + if "FORBIDDEN" in error_message and "ReplaceActorsForAssignable" in error_message: + self.log("") + self.log("📋 Permission Issue: Cannot assign GitHub Copilot") + self.log(" This is a repository permission limitation, not an application error.") + self.log("") + self.log(" Possible solutions:") + self.log(" 1. Repository admin can assign Copilot manually to the PR") + self.log(" 2. Repository admin can grant assignment permissions") + self.log(" 3. The @copilot comment will still notify Copilot to work on the PR") + self.log("") + self.log(" ✅ The PR was created successfully with @copilot instructions") + self.log(" ✅ Copilot can still see and act on the @copilot comment") + elif "NOT_FOUND" in error_message: + self.log("") + self.log("📋 Copilot Actor Not Found") + self.log(" This repository may not have GitHub Copilot enabled or available.") + self.log(" The @copilot comment was still added to notify available Copilot services.") + + return False + + def add_copilot_comment(self, owner: str, repo: str, pr_number: int, + file_path: str, old_text: str, new_text: str, branch_name: str, + work_item_id: str = None, item_source: str = None, doc_url: str = None, + custom_instructions: str = None) -> bool: + """Add a comment mentioning @copilot with explicit instructions to work on THIS PR + + This tells Copilot to make changes in the current PR's branch, not create a new PR. + + Args: + owner: Repository owner + repo: Repository name + pr_number: Pull request number + file_path: Path to the file to modify + old_text: Text to find and replace + new_text: New text to replace with + branch_name: Branch name for this PR + work_item_id: Work item or UUF issue ID + item_source: Source of the item ('UUF' or 'Azure DevOps') + + Returns True if successful, False otherwise. + """ + if self.dry_run: + self.log(f"[DRY-RUN] Would add @copilot comment to PR #{pr_number}") + return True + + try: + rest_headers = { + "Authorization": f"Bearer {self.token}", + "Accept": "application/vnd.github+json", + "User-Agent": USER_AGENT + } + + # Build work item reference + if work_item_id: + if item_source == 'UUF': + work_item_ref = f"**UUF Issue:** {work_item_id}\n" + else: + work_item_ref = f"**Azure DevOps Work Item:** AB#{work_item_id}\n" + else: + work_item_ref = "" + + # Build document reference + if file_path and not file_path.startswith("See work item") and not file_path.startswith("File path not specified"): + doc_ref = f"**Document to modify:** `{file_path}`\n" + file_instruction = f"2. Locate the file: `{file_path}`" + elif doc_url: + doc_ref = f"**Document URL:** {doc_url}\n" + file_instruction = f"2. Locate the file from this document URL: {doc_url}" + else: + doc_ref = "**Note:** File path not specified in work item\n" + file_instruction = "2. Review the PR description and work item details to identify the file(s) that need to be modified" + + # Build custom instructions section + if custom_instructions and custom_instructions.strip(): + custom_instructions_section = f""" +**Custom AI Instructions:** +{custom_instructions.strip()} + +""" + else: + custom_instructions_section = "" + + # Create a comment mentioning @copilot with VERY explicit instructions + comment_body = f"""@copilot + +{work_item_ref}{doc_ref} + +**Instructions:** + +Task: Update the documentation file with the changes requested above. + +Steps to complete: + +Locate the file containing the reference shown below. +Find the reference text within the file +Replace it with the 'Proposed New Text' shown above or use the reference as guidance +Maintain the existing formatting, indentation, and markdown structure +Ensure no other content in the file is modified + +> [!IMPORTANT] +> Only replace the specified text - do not make additional changes. +> Preserve all markdown formatting, links, and code blocks. +> If the current text cannot be found exactly, search for similar text. +> Please ensure the changes align with Microsoft documentation standards. +> Do not remove any text unless the reference or suggested guidance indicates to do so, if the text is obsolete or incorrect. + +1. Make changes to `{branch_name}` branch for this pull request. + +{file_instruction} + +3. Find this reference in the content: +``` +{old_text} +``` + +4. Use this text as guidance for the new content: +``` +{new_text} +``` + +5. Ensure the changes align with the context of the work item. + +6. Do a freshness check to ensure the file content is up-to-date before making changes. + +7. Commit the changes to the `{branch_name}` branch + +> [!NOTE] +> This documentation is maintained by spelluru. +> If guidance is empty, follow the reference to make changes. + +{custom_instructions_section} +Thank you! +""" + + # Post the comment to the PR + comments_url = f"https://api.github.com/repos/{owner}/{repo}/issues/{pr_number}/comments" + comment_data = {"body": comment_body} + + resp = requests.post(comments_url, headers=rest_headers, json=comment_data, timeout=30) + + if resp.status_code == 403: + self.log("❌ Permission denied when adding comment") + return False + + resp.raise_for_status() + self.log(f"✅ Added @copilot comment to PR #{pr_number}") + self.log(" Copilot has been instructed to work on THIS PR's branch") + return True + + except requests.HTTPError as e: + self.log(f"❌ HTTP Error adding comment: {e}") + return False + except Exception as e: + self.log(f"❌ Error adding comment: {str(e)}") + return False + + def add_pr_suggestion(self, owner: str, repo: str, pr_number: int, file_path: str, + old_text: str, new_text: str) -> bool: + """Add a suggested change comment to a PR + + This creates a review comment with a code suggestion that can be applied + with one click, keeping everything in the same PR. + + Returns True if successful, False otherwise. + """ + if self.dry_run: + self.log(f"[DRY-RUN] Would add suggested change to PR #{pr_number}") + return True + + try: + # Use REST API to create a review comment with suggestion + rest_headers = { + "Authorization": f"Bearer {self.token}", + "Accept": "application/vnd.github+json", + "User-Agent": USER_AGENT + } + + # First, get the latest commit SHA from the PR + pr_url = f"https://api.github.com/repos/{owner}/{repo}/pulls/{pr_number}" + resp = requests.get(pr_url, headers=rest_headers, timeout=30) + resp.raise_for_status() + pr_data = resp.json() + commit_sha = pr_data["head"]["sha"] + + self.log(f"Latest commit SHA: {commit_sha}") + + # Get the file content to find line numbers + file_url = f"https://api.github.com/repos/{owner}/{repo}/contents/{file_path}?ref={commit_sha}" + resp = requests.get(file_url, headers=rest_headers, timeout=30) + + if resp.status_code == 404: + self.log(f"⚠️ File not found in PR: {file_path}") + return False + + resp.raise_for_status() + file_data = resp.json() + + content = base64.b64decode(file_data["content"]).decode('utf-8') + lines = content.split('\n') + + # Find the line number where the old text appears + old_text_lines = old_text.split('\n') + start_line = None + + for i in range(len(lines) - len(old_text_lines) + 1): + if '\n'.join(lines[i:i+len(old_text_lines)]) == old_text: + start_line = i + 1 # Line numbers are 1-based + break + + if not start_line: + self.log("⚠️ Could not find text in file to create suggestion") + return False + + end_line = start_line + len(old_text_lines) - 1 + + # Create a review comment with suggested change + suggestion_body = f"""```suggestion +{new_text} +``` + +**Automated Suggestion:** This change was requested in Azure DevOps work item. + +Click "Commit suggestion" above to apply this change directly to the PR.""" + + comment_data = { + "body": suggestion_body, + "commit_id": commit_sha, + "path": file_path, + "line": end_line, + "start_line": start_line if start_line != end_line else None, + "start_side": "RIGHT" + } + + # Remove start_line if it's the same as line (single-line comment) + if start_line == end_line: + del comment_data["start_line"] + + comments_url = f"https://api.github.com/repos/{owner}/{repo}/pulls/{pr_number}/comments" + resp = requests.post(comments_url, headers=rest_headers, json=comment_data, timeout=30) + + if resp.status_code == 403: + self.log("❌ Permission denied when adding suggestion") + return False + + resp.raise_for_status() + self.log(f"✅ Added suggested change comment to PR #{pr_number}") + self.log(" User can click 'Commit suggestion' to apply it") + return True + + except requests.HTTPError as e: + self.log(f"❌ HTTP Error adding suggestion: {e}") + if hasattr(e, 'response') and e.response is not None: + self.log(f" Response: {e.response.text[:200]}") + return False + except Exception as e: + self.log(f"❌ Error adding suggestion: {str(e)}") + return False + + def create_branch_with_placeholder(self, owner: str, repo: str, branch_name: str, instructions: str) -> bool: + """Create a branch with a placeholder commit using REST API + + This creates a branch from main and adds a .copilot-instructions.md file + so that the branch has at least one commit, allowing PR creation. + + Returns True if successful, False otherwise. + """ + if self.dry_run: + self.log(f"[DRY-RUN] Would create branch {branch_name} with placeholder commit") + return True + + try: + # Use REST API for branch/file creation + rest_headers = { + "Authorization": f"Bearer {self.token}", + "Accept": "application/vnd.github+json", + "User-Agent": USER_AGENT + } + + # 1. Get the SHA of the main branch + self.log(f"Getting SHA of main branch...") + ref_url = f"https://api.github.com/repos/{owner}/{repo}/git/refs/heads/main" + resp = requests.get(ref_url, headers=rest_headers, timeout=30) + resp.raise_for_status() + main_sha = resp.json()["object"]["sha"] + self.log(f"Main branch SHA: {main_sha}") + + # 2. Create new branch from main + self.log(f"Creating branch {branch_name}...") + create_ref_url = f"https://api.github.com/repos/{owner}/{repo}/git/refs" + create_ref_payload = { + "ref": f"refs/heads/{branch_name}", + "sha": main_sha + } + resp = requests.post(create_ref_url, headers=rest_headers, json=create_ref_payload, timeout=30) + + # Check for permission errors + if resp.status_code == 403: + self.log("❌ Permission denied: GitHub token doesn't have write access to this repository") + self.log(f" Repository: {owner}/{repo}") + self.log(" Required permission: 'repo' scope with write access") + self.log("") + self.log(" Please verify:") + self.log(" 1. Your token has the 'repo' scope enabled") + self.log(" 2. You have write/push access to this repository") + self.log(" 3. The repository exists and the name is correct") + self.log("") + self.log(" TIP: You can still create Issues (uncheck the PR checkbox)") + return False + + # Branch might already exist, that's okay + if resp.status_code == 422: + error_detail = resp.json() + if "already exists" in str(error_detail).lower(): + self.log(f"Branch {branch_name} already exists, using existing branch") + return True + else: + self.log(f"Error creating branch: {error_detail}") + + resp.raise_for_status() + self.log(f"✅ Branch {branch_name} created") + + # 3. Create a placeholder file with instructions + self.log("Creating placeholder commit with Copilot instructions...") + file_content = f"""# Copilot Instructions + +This is a placeholder file created to allow PR creation. + +## Task +{instructions} + +Please process the instructions above and make the necessary changes to the documentation. + +Once you've made the changes, you can delete this file. +""" + + encoded_content = base64.b64encode(file_content.encode('utf-8')).decode() + + file_payload = { + "message": f"Add Copilot instructions for {branch_name}", + "content": encoded_content, + "branch": branch_name + } + + file_url = f"https://api.github.com/repos/{owner}/{repo}/contents/.copilot-instructions.md" + resp = requests.put(file_url, headers=rest_headers, json=file_payload, timeout=30) + resp.raise_for_status() + + self.log(f"✅ Placeholder commit created in branch {branch_name}") + return True + + except requests.HTTPError as e: + self.log(f"❌ HTTP Error creating branch with placeholder: {e}") + if e.response.status_code == 403: + self.log(" Permission denied - token doesn't have write access") + return False + except Exception as e: + self.log(f"❌ Error creating branch with placeholder: {str(e)}") + return False + + +# Backward compatibility alias +GitHubAPI = GitHubGQL \ No newline at end of file diff --git a/application/app_components/main_gui.py b/application/app_components/main_gui.py new file mode 100644 index 0000000..4459570 --- /dev/null +++ b/application/app_components/main_gui.py @@ -0,0 +1,2441 @@ +""" +Main GUI Interface +The primary user interface for the application +""" + +import tkinter as tk +from tkinter import ttk, messagebox, scrolledtext +import os +import threading +import webbrowser +from typing import List, Dict, Any, Optional + +from .utils import Logger +from .settings_dialog import SettingsDialog +from .work_item_processor import WorkItemProcessor +from .azure_devops_api import AzureDevOpsAPI +from .dataverse_api import DataverseAPI + + +class HyperlinkDialog: + """Dialog with clickable hyperlinks""" + + def __init__(self, parent, title: str, message: str, url: str): + self.result = None + self.url = url + + # Create dialog window + self.dialog = tk.Toplevel(parent) + self.dialog.title(title) + self.dialog.geometry("500x280") + self.dialog.transient(parent) + self.dialog.grab_set() + + # Center on parent + self.dialog.geometry("+%d+%d" % ( + parent.winfo_rootx() + 50, + parent.winfo_rooty() + 50 + )) + + # Message + message_label = tk.Label(self.dialog, text=message, wraplength=450, justify=tk.LEFT) + message_label.pack(pady=20, padx=20) + + # URL link + link_label = tk.Label(self.dialog, text=url, fg="blue", cursor="hand2", wraplength=450) + link_label.pack(pady=(0, 20), padx=20) + link_label.bind("", self._open_url) + + # Button frame + button_frame = tk.Frame(self.dialog) + button_frame.pack(pady=(0, 20)) + + # Copy Link button + copy_button = ttk.Button(button_frame, text="Copy Link", command=self._copy_link) + copy_button.pack(side=tk.LEFT, padx=5) + + # OK button + ok_button = ttk.Button(button_frame, text="OK", command=self._ok_clicked) + ok_button.pack(side=tk.LEFT, padx=5) + + # Focus and bindings + ok_button.focus_set() + self.dialog.bind('', lambda e: self._ok_clicked()) + self.dialog.bind('', lambda e: self._ok_clicked()) + + def _open_url(self, event=None): + """Open URL in browser""" + webbrowser.open(self.url) + + def _copy_link(self): + """Copy URL to clipboard""" + self.dialog.clipboard_clear() + self.dialog.clipboard_append(self.url) + self.dialog.update() # Required to finalize clipboard operation + + # Show a brief confirmation (update button text temporarily) + # Find the copy button and change its text + for widget in self.dialog.winfo_children(): + if isinstance(widget, tk.Frame): + for button in widget.winfo_children(): + if isinstance(button, ttk.Button) and button.cget('text') == 'Copy Link': + original_text = button.cget('text') + button.config(text='Copied!') + self.dialog.after(1500, lambda: button.config(text=original_text)) + break + + def _ok_clicked(self): + """Handle OK button click""" + self.result = True + self.dialog.destroy() + + def show(self): + """Show dialog and wait for result""" + self.dialog.wait_window() + return self.result + + +class DryRunVar: + """Compatibility class for dry run variable""" + + def __init__(self, app): + self.app = app + + def get(self): + return self.app.dry_run_enabled + + def set(self, value): + self.app.dry_run_enabled = bool(value) + + +class MainGUI: + """Main GUI interface for the application""" + + def __init__(self, root, config_manager, ai_manager, app): + self.root = root + self.config_manager = config_manager + self.ai_manager = ai_manager + self.app = app + + # Application state + self.current_work_items = [] + self.current_item_index = 0 + self.current_organization = None + self.edit_mode = False + + # API instances + self.azure_api = None + self.dataverse_api = None + + # Create dry run compatibility wrapper + self.dry_run_var = DryRunVar(app) + + # Create GUI + self.create_gui() + + # Initialize logger after GUI is created + self.logger = Logger(self.log_text) + + # Initialize work item processor + self.work_item_processor = WorkItemProcessor(self.logger, self.config_manager.get_config()) + + # Initialize cache manager + from .cache_manager import CacheManager + self.cache_manager = CacheManager(cache_duration_hours=24) + + # Initialize diff display + self.update_diff_display("") + + # Auto-load cached items on startup + self.root.after(500, self._auto_load_cached_items) + + # Load custom instructions after GUI is ready + self.root.after(100, self._load_custom_instructions) + + def create_gui(self): + """Create the main GUI interface""" + # Configure custom styles + self._configure_styles() + + # Main frame with padding + main_frame = ttk.Frame(self.root, padding="10") + main_frame.grid(row=0, column=0, sticky=(tk.W, tk.E, tk.N, tk.S)) + + # Configure grid weights + self.root.columnconfigure(0, weight=1) + self.root.rowconfigure(0, weight=1) + main_frame.columnconfigure(1, weight=1) + + # Create sections + self._create_title_section(main_frame) + self._create_controls_section(main_frame) + self._create_status_section(main_frame) + self._create_tabs_section(main_frame) + + def _configure_styles(self): + """Configure custom styles for the GUI""" + style = ttk.Style() + + # Grouped sections + style.configure('Config.TLabelframe', relief='solid', borderwidth=1) + style.configure('Config.TLabelframe.Label', font=('Arial', 11, 'bold')) + style.configure('WorkItem.TLabelframe', relief='solid', borderwidth=1) + style.configure('WorkItem.TLabelframe.Label', font=('Arial', 11, 'bold')) + + # Notebook tabs + style.configure('TNotebook.Tab', background='lightblue', foreground='black', padding=[10, 5]) + style.map('TNotebook.Tab', + background=[('selected', 'lightblue'), ('active', '#87CEEB')], + foreground=[('selected', 'black'), ('active', 'black')]) + + # Blue edit button + style.configure('BlueEdit.TButton', + background='#2196F3', foreground='black', font=('Arial', 9, 'bold'), + relief='raised', borderwidth=2, focuscolor='none') + style.map('BlueEdit.TButton', + background=[('active', '#1976D2'), ('pressed', '#0D47A1'), ('!disabled', '#2196F3')], + foreground=[('active', 'black'), ('pressed', 'black'), ('!disabled', 'black')], + relief=[('pressed', 'sunken'), ('!pressed', 'raised')]) + + # Orange save button + style.configure('OrangeSave.TButton', + background='#FF9800', foreground='black', font=('Arial', 9, 'bold'), + relief='raised', borderwidth=2, focuscolor='none') + style.map('OrangeSave.TButton', + background=[('active', '#F57C00'), ('pressed', '#E65100'), ('!disabled', '#FF9800')], + foreground=[('active', 'black'), ('pressed', 'black'), ('!disabled', 'black')], + relief=[('pressed', 'sunken'), ('!pressed', 'raised')]) + + # Green save button for custom instructions + style.configure('GreenSave.TButton', + background='#4CAF50', foreground='black', font=('Arial', 9, 'bold'), + relief='raised', borderwidth=2, focuscolor='none') + style.map('GreenSave.TButton', + background=[('active', '#388E3C'), ('pressed', '#2E7D32'), ('!disabled', '#4CAF50')], + foreground=[('active', 'black'), ('pressed', 'black'), ('!disabled', 'black')], + relief=[('pressed', 'sunken'), ('!pressed', 'raised')]) + + def _create_title_section(self, parent): + """Create title section with settings button""" + title_frame = ttk.Frame(parent) + title_frame.grid(row=0, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=(0, 20)) + title_frame.columnconfigure(0, weight=1) + + # Title + title_label = ttk.Label(title_frame, text="MicrosoftDocFlow v3", + font=('Arial', 16, 'bold')) + title_label.grid(row=0, column=0, sticky=tk.W) + + # AI Modules button + self.ai_modules_button = ttk.Button(title_frame, text="🤖 AI Modules", + command=self.check_ai_modules_manual) + self.ai_modules_button.grid(row=0, column=1, sticky=tk.E, padx=(10, 5)) + + # Settings button + self.settings_button = ttk.Button(title_frame, text="⚙️ Settings", + command=self.open_settings) + self.settings_button.grid(row=0, column=2, sticky=tk.E, padx=(5, 0)) + + def _create_controls_section(self, parent): + """Create work item controls section""" + # Work Item Details group frame + workitem_frame = ttk.LabelFrame(parent, text="📋 Work Item Details", + style='WorkItem.TLabelframe', padding="15") + workitem_frame.grid(row=1, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=(0, 15), padx=5) + workitem_frame.columnconfigure(2, weight=1) + + # Controls row + controls_row = ttk.Frame(workitem_frame) + controls_row.grid(row=0, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=(0, 10)) + controls_row.columnconfigure(6, weight=1) + + # Fetch buttons + self.fetch_button = ttk.Button(controls_row, text="📥 Fetch Work Items", + command=self.start_fetch_work_items) + self.fetch_button.grid(row=0, column=0, padx=(0, 10)) + + self.fetch_uuf_button = ttk.Button(controls_row, text="📋 Fetch UUF Items", + command=self.start_fetch_uuf_items) + self.fetch_uuf_button.grid(row=0, column=1, padx=(0, 15)) + + # Navigation buttons + self.prev_button = ttk.Button(controls_row, text="← Previous", + command=self.previous_item, state='disabled') + self.prev_button.grid(row=0, column=2, padx=(0, 5)) + + self.next_button = ttk.Button(controls_row, text="Next →", + command=self.next_item, state='disabled') + self.next_button.grid(row=0, column=3, padx=(0, 15)) + + # Action type dropdown + self.action_type_var = tk.StringVar(value="Create Issue") + self.action_type_dropdown = ttk.Combobox(controls_row, textvariable=self.action_type_var, + values=["Create Issue", "Create PR"], + state="readonly", width=15) + self.action_type_dropdown.grid(row=0, column=4, padx=(0, 5)) + self.action_type_dropdown.bind("<>", lambda e: self.update_action_button_text()) + + # GO button + self.go_button = ttk.Button(controls_row, text="🚀 GO", + command=self.create_github_resource, state='disabled') + self.go_button.grid(row=0, column=5, padx=(0, 20)) + + # Item counter + self.item_counter_label = ttk.Label(controls_row, text="No items loaded", + font=('Arial', 9, 'italic')) + self.item_counter_label.grid(row=0, column=6, sticky=tk.E) + + def _create_status_section(self, parent): + """Create progress and status section""" + # Progress bar + self.progress = ttk.Progressbar(parent, mode='indeterminate') + self.progress.grid(row=5, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=5) + + # Status label + self.status_label = ttk.Label(parent, text="Ready to fetch work items...") + self.status_label.grid(row=6, column=0, columnspan=3, pady=5) + + def _create_tabs_section(self, parent): + """Create tabbed interface section""" + # Create notebook + self.notebook = ttk.Notebook(parent) + self.notebook.grid(row=7, column=0, columnspan=3, sticky=(tk.W, tk.E, tk.N, tk.S), pady=10) + parent.rowconfigure(7, weight=1) + + # Create tabs + self._create_current_item_tab(self.notebook) + self._create_diff_tab(self.notebook) + self._create_log_tab(self.notebook) + self._create_all_items_tab(self.notebook) + + def _create_current_item_tab(self, notebook): + """Create current work item tab""" + item_frame = ttk.Frame(notebook) + notebook.add(item_frame, text="Current Work Item") + item_frame.columnconfigure(1, weight=1) + + # Work Item ID + ttk.Label(item_frame, text="Work Item ID:", font=('Arial', 10, 'bold')).grid( + row=0, column=0, sticky=tk.W, pady=5, padx=5) + self.work_item_id_label = ttk.Label(item_frame, text="Not loaded") + self.work_item_id_label.grid(row=0, column=1, sticky=tk.W, pady=5, padx=5) + self.work_item_id_label.bind("", self.open_work_item_url) + self.work_item_id_label.bind("", self.on_work_item_hover_enter) + self.work_item_id_label.bind("", self.on_work_item_hover_leave) + + # Nature of Request + ttk.Label(item_frame, text="Nature of Request:", font=('Arial', 10, 'bold')).grid( + row=1, column=0, sticky=tk.W, pady=5, padx=5) + self.nature_text = tk.Text(item_frame, height=1, width=70, state='disabled', wrap=tk.WORD) + self.nature_text.grid(row=1, column=1, sticky=(tk.W, tk.E), pady=5, padx=5) + + # Document URL + ttk.Label(item_frame, text="Live Doc URL:", font=('Arial', 10, 'bold')).grid( + row=2, column=0, sticky=tk.W, pady=5, padx=5) + self.doc_url_text = tk.Text(item_frame, height=1, width=70, state='disabled', wrap=tk.WORD) + self.doc_url_text.grid(row=2, column=1, sticky=(tk.W, tk.E), pady=5, padx=5) + + # Text to Change + ttk.Label(item_frame, text="Text to Change:", font=('Arial', 10, 'bold')).grid( + row=3, column=0, sticky=tk.W, pady=5, padx=5) + self.text_to_change_display = scrolledtext.ScrolledText(item_frame, height=5, width=70, state='disabled') + self.text_to_change_display.grid(row=3, column=1, sticky=(tk.W, tk.E), pady=5, padx=5) + + # Proposed New Text with Edit functionality + new_text_frame = ttk.Frame(item_frame) + new_text_frame.grid(row=4, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=5, padx=5) + new_text_frame.columnconfigure(1, weight=1) + + ttk.Label(new_text_frame, text="Proposed New Text:", font=('Arial', 10, 'bold')).grid( + row=0, column=0, sticky=tk.W, pady=5) + + self.edit_button = ttk.Button(new_text_frame, text="✏️ Edit", + command=self.toggle_edit_mode, state='disabled', + style='BlueEdit.TButton') + self.edit_button.grid(row=0, column=1, sticky=tk.E, pady=5, padx=(5, 0)) + + self.new_text_display = scrolledtext.ScrolledText(new_text_frame, height=5, width=70, state='disabled') + self.new_text_display.grid(row=1, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=5) + + # Custom AI Instructions with Save functionality + custom_instructions_frame = ttk.Frame(item_frame) + custom_instructions_frame.grid(row=5, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=5, padx=5) + custom_instructions_frame.columnconfigure(1, weight=1) + + ttk.Label(custom_instructions_frame, text="Custom AI Instructions:", font=('Arial', 10, 'bold')).grid( + row=0, column=0, sticky=tk.W, pady=5) + + # Button frame to hold both save and clear buttons + button_frame = ttk.Frame(custom_instructions_frame) + button_frame.grid(row=0, column=1, sticky=tk.E, pady=5, padx=(5, 0)) + + self.save_instructions_button = ttk.Button(button_frame, text="💾 Save", + command=self.save_custom_instructions, + style='GreenSave.TButton') + self.save_instructions_button.grid(row=0, column=0, padx=(0, 5)) + + self.clear_instructions_button = ttk.Button(button_frame, text="🗑️ Clear", + command=self.clear_custom_instructions) + self.clear_instructions_button.grid(row=0, column=1) + + self.custom_instructions_display = scrolledtext.ScrolledText(custom_instructions_frame, height=4, width=70) + self.custom_instructions_display.grid(row=1, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=5) + + # Configure row weights + for i in range(6): + item_frame.rowconfigure(i, weight=1) + + def _create_diff_tab(self, notebook): + """Create diff view tab""" + diff_frame = ttk.Frame(notebook) + notebook.add(diff_frame, text="View Diff") + + diff_frame.columnconfigure(0, weight=1) + diff_frame.rowconfigure(1, weight=1) # Give weight to row 1 (diff_text) instead of row 0 (header) + + # Add a header label + header_frame = ttk.Frame(diff_frame) + header_frame.grid(row=0, column=0, sticky=(tk.W, tk.E), padx=5, pady=5) + header_frame.columnconfigure(0, weight=1) + + ttk.Label(header_frame, text="Diff Viewer", font=('Arial', 12, 'bold')).grid( + row=0, column=0, sticky=tk.W, pady=5) + + # Add button to find existing diff files + self.find_diff_button = ttk.Button(header_frame, text="Find .diff Files", + command=self.find_and_load_diff_files) + self.find_diff_button.grid(row=0, column=1, sticky=tk.E, pady=5, padx=(0, 5)) + + self.clear_diff_button = ttk.Button(header_frame, text="Clear Diff", + command=self.clear_diff_display, state='disabled') + self.clear_diff_button.grid(row=0, column=2, sticky=tk.E, pady=5) + + # Create the diff display area with syntax highlighting-like colors + self.diff_text = scrolledtext.ScrolledText(diff_frame, + font=('Courier New', 9), + state='disabled', + bg='#f8f8f8') + self.diff_text.grid(row=1, column=0, sticky=(tk.W, tk.E, tk.N, tk.S), padx=5, pady=5) + + # Configure diff syntax highlighting tags + self.diff_text.tag_config('diff_header', foreground='#0066cc', font=('Courier New', 9, 'bold')) + self.diff_text.tag_config('diff_file', foreground='#666666', font=('Courier New', 9, 'bold')) + self.diff_text.tag_config('diff_add', foreground='#008800', background='#e8ffe8') + self.diff_text.tag_config('diff_remove', foreground='#cc0000', background='#ffe8e8') + self.diff_text.tag_config('diff_context', foreground='#666666') + self.diff_text.tag_config('diff_line_numbers', foreground='#999999') + + def _create_log_tab(self, notebook): + """Create processing log tab""" + log_frame = ttk.Frame(notebook) + notebook.add(log_frame, text="Processing Log") + + log_frame.columnconfigure(0, weight=1) + log_frame.rowconfigure(0, weight=1) + + self.log_text = scrolledtext.ScrolledText(log_frame, height=25, width=100) + self.log_text.grid(row=0, column=0, sticky=(tk.W, tk.E, tk.N, tk.S), padx=5, pady=5) + + def _create_all_items_tab(self, notebook): + """Create all work items tab""" + items_frame = ttk.Frame(notebook) + notebook.add(items_frame, text="All Work Items") + + items_frame.columnconfigure(0, weight=1) + items_frame.rowconfigure(0, weight=1) # Treeview gets the weight + # Row 1 (button frame) will not have weight, so it stays fixed size + + # Treeview for all items + columns = ('ID', 'Title', 'Nature', 'GitHub Repo', 'ms.author', 'Status') + self.items_tree = ttk.Treeview(items_frame, columns=columns, show='headings', height=20) + + # Define headings + self.items_tree.heading('ID', text='Work Item ID', anchor=tk.W) + self.items_tree.heading('Title', text='Title', anchor=tk.W) + self.items_tree.heading('Nature', text='Nature of Request', anchor=tk.W) + self.items_tree.heading('GitHub Repo', text='GitHub Repository', anchor=tk.W) + self.items_tree.heading('ms.author', text='ms.author', anchor=tk.W) + self.items_tree.heading('Status', text='Processing Status', anchor=tk.W) + + # Configure columns + self.items_tree.column('ID', width=100, anchor=tk.W) + self.items_tree.column('Title', width=220, anchor=tk.W) + self.items_tree.column('Nature', width=160, anchor=tk.W) + self.items_tree.column('GitHub Repo', width=160, anchor=tk.W) + self.items_tree.column('ms.author', width=100, anchor=tk.W) + self.items_tree.column('Status', width=100, anchor=tk.W) + + self.items_tree.grid(row=0, column=0, sticky=(tk.W, tk.E, tk.N, tk.S), padx=5, pady=5) + + # Add selection functionality + self.items_tree.bind('', self._on_item_double_click) + self.items_tree.bind('<>', self._on_item_select) + + # Add button frame for selection actions + button_frame = ttk.Frame(items_frame) + button_frame.grid(row=1, column=0, columnspan=2, sticky=(tk.W, tk.E), padx=5, pady=5) + + self.select_item_button = ttk.Button(button_frame, text="� Set as Current Item", + command=self._select_current_item, state='disabled') + self.select_item_button.pack(side=tk.LEFT, padx=5) + + ttk.Label(button_frame, text="Double-click an item or use the button above to set it as the current work item", + font=('Arial', 9), foreground='#666666').pack(side=tk.LEFT, padx=10) + + # Scrollbar + items_scrollbar = ttk.Scrollbar(items_frame, orient=tk.VERTICAL, command=self.items_tree.yview) + items_scrollbar.grid(row=0, column=1, sticky=(tk.N, tk.S)) + self.items_tree.configure(yscrollcommand=items_scrollbar.set) + + # Track selected item for enabling/disabling button + self.selected_tree_item = None + + # Event handlers and methods + def update_status(self, message: str): + """Update status label""" + self.status_label.config(text=message) + self.root.update_idletasks() + + def _check_ai_modules_manual(self): + """Manually check AI modules""" + config = self.config_manager.get_config() + ai_provider = config.get('AI_PROVIDER', '').strip().lower() + self.ai_manager.show_ai_modules_info(ai_provider, self.root) + + def _open_settings(self): + """Open settings dialog""" + try: + config = self.config_manager.get_config() + dialog = SettingsDialog(self.root, config, self.config_manager, self.cache_manager) + result = dialog.show() + + if result: + # Reload configuration + self.config_manager.load_configuration() + config = self.config_manager.get_config() + self.app.update_config(config) + + # Update dry run state + dry_run_config = config.get('DRY_RUN', 'false') + self.app.dry_run_enabled = str(dry_run_config).lower() in ('true', '1', 'yes', 'on') + + self.update_status("✅ Settings saved and loaded successfully!") + + except Exception as e: + messagebox.showerror("Error", f"Failed to open settings dialog:\n{str(e)}") + + def _start_fetch_work_items(self): + """Start fetching work items""" + config = self.config_manager.get_config() + query_url = config.get('AZURE_DEVOPS_QUERY', '').strip() + azure_token = config.get('AZURE_DEVOPS_PAT', '').strip() + + if not query_url: + messagebox.showerror("Error", "Please enter an Azure DevOps Query URL in Settings") + return + + if not azure_token: + messagebox.showerror("Error", "Please enter your Azure DevOps token in Settings") + return + + # Clear previous data + self._clear_data() + + # Start processing thread + thread = threading.Thread(target=self._fetch_work_items, args=(query_url, azure_token)) + thread.daemon = True + thread.start() + + def _start_fetch_uuf_items(self): + """Start fetching UUF items""" + config = self.config_manager.get_config() + + # Check configuration + required_fields = [ + 'DATAVERSE_ENVIRONMENT_URL', + 'DATAVERSE_TABLE_NAME', + 'AZURE_AD_CLIENT_ID', + 'AZURE_AD_CLIENT_SECRET', + 'AZURE_AD_TENANT_ID' + ] + + if not all(config.get(field) for field in required_fields): + messagebox.showerror( + "Configuration Missing", + "PowerApp/Dataverse configuration is not complete.\n\n" + "Please ensure all required fields are set in Settings." + ) + return + + # Clear previous data + self._clear_data() + + # Start processing thread + thread = threading.Thread(target=self._fetch_uuf_items) + thread.daemon = True + thread.start() + + def _clear_data(self): + """Clear previous data""" + self.current_work_items = [] + self.current_item_index = 0 + self._clear_current_item_display() + self._clear_all_items_tree() + + def _auto_load_cached_items(self): + """Automatically load cached items on app startup""" + try: + config = self.config_manager.get_config() + + # Try to load Azure DevOps cache first + query_url = config.get('AZURE_DEVOPS_QUERY', '').strip() + azure_token = config.get('AZURE_DEVOPS_PAT', '').strip() + + if query_url and azure_token: + cache_id = query_url + cached_items = self.cache_manager.load_from_cache('azure_devops', cache_id) + + if cached_items: + self.logger.log("=== Auto-loading cached work items ===") + self.logger.log(f"✅ Loaded {len(cached_items)} items from cache") + self.current_work_items = cached_items + + # Setup Azure API for operations + temp_api = AzureDevOpsAPI("", azure_token, self.logger) + org, _, _ = temp_api.parse_query_url(query_url) + self.current_organization = org + self.azure_api = AzureDevOpsAPI(org, azure_token, self.logger) + + self._update_after_fetch() + self.update_status(f"Loaded {len(cached_items)} items from cache") + return + + # Try to load UUF cache if Azure DevOps cache not available + uuf_env_url = config.get('DATAVERSE_ENVIRONMENT_URL', '').strip() + uuf_table = config.get('DATAVERSE_TABLE_NAME', '').strip() + + if uuf_env_url and uuf_table: + cache_id = f"{uuf_env_url}_{uuf_table}" + cached_items = self.cache_manager.load_from_cache('uuf', cache_id) + + if cached_items: + self.logger.log("=== Auto-loading cached UUF items ===") + self.logger.log(f"✅ Loaded {len(cached_items)} items from cache") + self.current_work_items = cached_items + + # Setup Dataverse API for operations + self.dataverse_api = DataverseAPI(config, self.logger) + + self._update_after_fetch() + self.update_status(f"Loaded {len(cached_items)} UUF items from cache") + return + + # No cache available + self.logger.log("No cached items found") + + except Exception as e: + self.logger.log(f"⚠️ Error auto-loading cache: {str(e)}") + + def _fetch_work_items(self, query_url: str, azure_token: str): + """Fetch work items from Azure DevOps (always from server)""" + try: + self.fetch_button.config(state='disabled') + self.progress.start() + + cache_id = query_url + self.update_status("Fetching work items from Azure DevOps...") + self.logger.log("=== Fetching work items from Azure DevOps ===") + + # Initialize Azure DevOps API + temp_api = AzureDevOpsAPI("", azure_token, self.logger) + + # Parse query URL + org, project, query_id = temp_api.parse_query_url(query_url) + self.current_organization = org + self.logger.log(f"Parsed query - Org: {org}, Project: {project}, Query ID: {query_id}") + + # Create proper API instance + self.azure_api = AzureDevOpsAPI(org, azure_token, self.logger) + + # Execute query and process items + work_items = self.azure_api.execute_query(org, project, query_id, azure_token) + self.logger.log(f"Found {len(work_items)} work items") + + # Process items + self.current_work_items = [] + for item in work_items: + processed_item = self.work_item_processor.process_work_item(item) + if processed_item: + self.current_work_items.append(processed_item) + + self.logger.log(f"Successfully processed {len(self.current_work_items)} work items") + + # Save to cache + if self.cache_manager.save_to_cache('azure_devops', cache_id, self.current_work_items): + self.logger.log("✅ Work items cached for faster loading next time") + + # Update GUI + self._update_after_fetch() + + except Exception as e: + error_msg = f"Error fetching work items: {str(e)}" + self.logger.log(error_msg) + self.update_status("Fetch failed!") + messagebox.showerror("Fetch Error", error_msg) + finally: + self.progress.stop() + self.fetch_button.config(state='normal') + + def _fetch_uuf_items(self): + """Fetch UUF items from Dataverse (always from server)""" + try: + self.fetch_uuf_button.config(state='disabled') + self.progress.start() + + config = self.config_manager.get_config() + + # Create cache ID from config + cache_id = f"{config.get('DATAVERSE_ENVIRONMENT_URL')}_{config.get('DATAVERSE_TABLE_NAME')}" + + self.update_status("Fetching UUF items from PowerApp/Dataverse...") + self.logger.log("=== Fetching UUF items from Dataverse ===") + + # Initialize Dataverse API + self.dataverse_api = DataverseAPI( + config['DATAVERSE_ENVIRONMENT_URL'], + config['DATAVERSE_TABLE_NAME'], + self.logger + ) + + # Authenticate and fetch + auth_success = self.dataverse_api.authenticate( + config['AZURE_AD_CLIENT_ID'], + config['AZURE_AD_CLIENT_SECRET'], + config['AZURE_AD_TENANT_ID'] + ) + + if not auth_success: + raise RuntimeError("Failed to authenticate with Azure AD") + + uuf_items = self.dataverse_api.fetch_uuf_items() + self.logger.log(f"Found {len(uuf_items)} UUF items") + + # Process items + self.current_work_items = [] + for item in uuf_items: + processed_item = self.work_item_processor.process_uuf_item(item) + if processed_item: + self.current_work_items.append(processed_item) + + self.logger.log(f"Successfully processed {len(self.current_work_items)} UUF items") + + # Save to cache + if self.cache_manager.save_to_cache('uuf', cache_id, self.current_work_items): + self.logger.log("✅ UUF items cached for faster loading next time") + + # Update GUI + self._update_after_fetch() + + except Exception as e: + error_msg = f"Error fetching UUF items: {str(e)}" + self.logger.log(error_msg) + self.update_status("Fetch failed!") + messagebox.showerror("Fetch Error", error_msg) + finally: + self.progress.stop() + self.fetch_uuf_button.config(state='normal') + + def _update_after_fetch(self): + """Update GUI after successful fetch""" + self._update_all_items_tree() + if self.current_work_items: + self.current_item_index = 0 + self._display_current_item() + self._update_navigation_buttons() + self.update_status(f"Loaded {len(self.current_work_items)} items") + else: + self.update_status("No valid items found") + + def _clear_current_item_display(self): + """Clear current item display""" + self.work_item_id_label.config(text="Not loaded", foreground="black", cursor="") + + # Clear text widgets + for widget in [self.nature_text, self.doc_url_text, self.text_to_change_display, self.new_text_display]: + widget.config(state='normal') + widget.delete(1.0, tk.END) + widget.config(state='disabled') + + # Reset edit mode + self.edit_mode = False + self.edit_button.config(text="✏️ Edit", state='disabled', style='BlueEdit.TButton') + + def _clear_all_items_tree(self): + """Clear all items tree""" + for item in self.items_tree.get_children(): + self.items_tree.delete(item) + + def _display_current_item(self): + """Display current work item""" + if not self.current_work_items or self.current_item_index >= len(self.current_work_items): + return + + item = self.current_work_items[self.current_item_index] + + # Update work item ID with hyperlink styling + self.work_item_id_label.config( + text=f"#{item['id']} - {item['title']}", + foreground="blue", + cursor="hand2" + ) + + # Update text fields + self._update_text_widget(self.nature_text, item['nature_of_request']) + self._update_text_widget(self.doc_url_text, item['mydoc_url']) + self._update_text_widget(self.text_to_change_display, item['text_to_change']) + self._update_text_widget(self.new_text_display, item['new_text']) + + # Reset edit mode + self.edit_mode = False + self.edit_button.config(text="✏️ Edit", state='normal', style='BlueEdit.TButton') + + # Update dropdown based on source + if item.get('source') == 'UUF': + self.action_type_dropdown.set("Create PR") + self.action_type_dropdown.config(state='disabled') + else: + self.action_type_dropdown.config(state='readonly') + + # Update counter + self.item_counter_label.config(text=f"Item {self.current_item_index + 1} of {len(self.current_work_items)}") + + # Update highlighting in All Work Items treeview + self._update_treeview_selection() + + def _update_text_widget(self, widget, text): + """Update a text widget with new content""" + widget.config(state='normal') + widget.delete(1.0, tk.END) + widget.insert(1.0, text) + widget.config(state='disabled') + + def _update_all_items_tree(self): + """Update all items treeview""" + self._clear_all_items_tree() + current_item_id = None + + # Get current item ID if available + if hasattr(self, 'current_work_items') and self.current_work_items and hasattr(self, 'current_item_index'): + if 0 <= self.current_item_index < len(self.current_work_items): + current_item_id = self.current_work_items[self.current_item_index]['id'] + + for item in self.current_work_items: + nature_preview = item['nature_of_request'][:50] + "..." if len(item['nature_of_request']) > 50 else item['nature_of_request'] + + github_info = item.get('github_info', {}) + github_repo = "" + ms_author = "" + + if github_info.get('owner') and github_info.get('repo'): + github_repo = f"{github_info['owner']}/{github_info['repo']}" + elif github_info.get('error'): + github_repo = "Error extracting" + else: + github_repo = "Not determined" + + ms_author = github_info.get('ms_author') or "Not found" + + item_id = self.items_tree.insert('', 'end', values=( + item['id'], + item['title'][:40] + "..." if len(item['title']) > 40 else item['title'], + nature_preview, + github_repo, + ms_author, + item['status'] + )) + + # Highlight the current item + if current_item_id and item['id'] == current_item_id: + self.items_tree.selection_set(item_id) + self.items_tree.focus(item_id) + # Configure a tag for highlighting the current item + self.items_tree.set(item_id, 'Status', f"★ {item['status']}") # Add star to status + + def _update_treeview_selection(self): + """Update the selection highlighting in the All Work Items treeview to match current item""" + if not hasattr(self, 'items_tree') or not self.current_work_items: + return + + try: + # Get current item ID + if not (0 <= self.current_item_index < len(self.current_work_items)): + return + + current_item_id = self.current_work_items[self.current_item_index]['id'] + + # Clear current selection + self.items_tree.selection_remove(self.items_tree.selection()) + + # Find and select the current item in the treeview + for item_id in self.items_tree.get_children(): + item_values = self.items_tree.item(item_id, 'values') + if item_values and item_values[0] == current_item_id: + self.items_tree.selection_set(item_id) + self.items_tree.focus(item_id) + self.items_tree.see(item_id) # Scroll to make sure it's visible + break + + except Exception as e: + # Silently handle errors to avoid disrupting the UI + pass + + def _update_navigation_buttons(self): + """Update navigation button states""" + has_items = len(self.current_work_items) > 0 + + self.prev_button.config(state='normal' if has_items and self.current_item_index > 0 else 'disabled') + self.next_button.config(state='normal' if has_items and self.current_item_index < len(self.current_work_items) - 1 else 'disabled') + + # Enable GO button if current item has valid GitHub info + if has_items: + current_item = self.current_work_items[self.current_item_index] + github_info = current_item['github_info'] + has_valid_github = github_info.get('owner') and github_info.get('repo') + self.go_button.config(state='normal' if has_valid_github else 'disabled') + else: + self.go_button.config(state='disabled') + + def _previous_item(self): + """Navigate to previous item""" + if self.current_item_index > 0: + self.current_item_index -= 1 + self._display_current_item() + self._update_navigation_buttons() + + def _next_item(self): + """Navigate to next item""" + if self.current_item_index < len(self.current_work_items) - 1: + self.current_item_index += 1 + self._display_current_item() + self._update_navigation_buttons() + + def _toggle_edit_mode(self): + """Toggle edit mode for proposed new text""" + if not self.current_work_items or self.current_item_index >= len(self.current_work_items): + return + + if not self.edit_mode: + # Enter edit mode + self.edit_mode = True + self.new_text_display.config(state='normal') + self.edit_button.config(text="💾 Save", style='OrangeSave.TButton') + self.logger.log(f"Editing mode enabled for work item #{self.current_work_items[self.current_item_index]['id']}") + else: + # Save changes + current_item = self.current_work_items[self.current_item_index] + new_text = self.new_text_display.get(1.0, tk.END).strip() + current_item['new_text'] = new_text + + self.edit_mode = False + self.new_text_display.config(state='disabled') + self.edit_button.config(text="✏️ Edit", style='BlueEdit.TButton') + + self.logger.log(f"Proposed new text updated for work item #{current_item['id']}") + messagebox.showinfo("Saved", "Proposed new text has been updated!") + + def _load_custom_instructions(self): + """Load custom instructions from config on startup""" + try: + config = self.config_manager.get_config() + custom_instructions = config.get('CUSTOM_INSTRUCTIONS', '') + + # Set the text in the custom instructions display + if hasattr(self, 'custom_instructions_display'): + self.custom_instructions_display.delete('1.0', tk.END) + if custom_instructions: + self.custom_instructions_display.insert('1.0', custom_instructions) + except Exception as e: + self.logger.log(f"Error loading custom instructions: {str(e)}") + + def save_custom_instructions(self): + """Save custom instructions to .env file""" + try: + # Get the current instructions from the text widget + current_instructions = self.custom_instructions_display.get('1.0', tk.END).strip() + + # Save to config + config_values = {'CUSTOM_INSTRUCTIONS': current_instructions} + success = self.config_manager.save_configuration(config_values) + + if success: + self.logger.log("Custom AI instructions saved to .env file") + messagebox.showinfo("Saved", "Custom AI instructions have been saved to .env file!") + else: + self.logger.log("Failed to save custom AI instructions") + messagebox.showerror("Error", "Failed to save custom AI instructions to .env file.") + + except Exception as e: + self.logger.log(f"Error saving custom instructions: {str(e)}") + messagebox.showerror("Error", f"Error saving custom instructions: {str(e)}") + + def clear_custom_instructions(self): + """Clear custom instructions from both UI and .env file""" + try: + # Clear the text widget + if hasattr(self, 'custom_instructions_display'): + self.custom_instructions_display.delete('1.0', tk.END) + + # Save empty value to config + config_values = {'CUSTOM_INSTRUCTIONS': ''} + success = self.config_manager.save_configuration(config_values) + + if success: + self.logger.log("Custom AI instructions cleared from .env file") + messagebox.showinfo("Cleared", "Custom AI instructions have been cleared!") + else: + self.logger.log("Failed to clear custom AI instructions") + messagebox.showerror("Error", "Failed to clear custom AI instructions from .env file.") + + except Exception as e: + self.logger.log(f"Error clearing custom instructions: {str(e)}") + messagebox.showerror("Error", f"Error clearing custom instructions: {str(e)}") + + def _extract_file_path_from_github_url(self, url: str) -> str: + """Extract file path from GitHub URL + + Example: https://github.com/owner/repo/blob/main/path/to/file.md -> path/to/file.md + """ + if not url or 'github.com' not in url or '/blob/' not in url: + return '' + + try: + # Split by /blob/ to separate the repo part from the file part + parts = url.split('/blob/', 1) + if len(parts) != 2: + return '' + + # Split the second part by / to get branch and file path + path_parts = parts[1].split('/', 1) + if len(path_parts) == 2: + # Return everything after the branch name + return path_parts[1] + except Exception as e: + self.logger.log(f"Warning: Failed to extract file path from URL {url}: {e}") + + return '' + + def _on_work_item_hover_enter(self, event=None): + """Handle mouse enter on work item ID""" + if self.current_work_items and self.current_item_index < len(self.current_work_items): + self.work_item_id_label.configure(font=('Arial', 10, 'underline')) + + def _on_work_item_hover_leave(self, event=None): + """Handle mouse leave on work item ID""" + if self.current_work_items and self.current_item_index < len(self.current_work_items): + self.work_item_id_label.configure(font=('Arial', 10)) + + def _open_work_item_url(self, event=None): + """Open work item URL in browser""" + if not self.current_work_items or self.current_item_index >= len(self.current_work_items): + return + + item = self.current_work_items[self.current_item_index] + work_item_id = item['id'] + + if self.current_organization: + work_item_url = f"https://dev.azure.com/{self.current_organization}/_workitems/edit/{work_item_id}" + webbrowser.open(work_item_url) + self.logger.log(f"Opened work item #{work_item_id} in browser: {work_item_url}") + else: + messagebox.showwarning("Warning", "Organization not available. Cannot open work item URL.") + + def _create_github_resource(self): + """Create GitHub resource (PR) with cross-repository support and repository verification""" + try: + if not self.current_work_items or self.current_item_index >= len(self.current_work_items): + messagebox.showerror("Error", "No work item selected") + return + + # Get current work item first + current_item = self.current_work_items[self.current_item_index] + + # Get configuration + config = self.config_manager.get_config() + github_token = config.get('GITHUB_PAT', '').strip() + target_repo = config.get('GITHUB_REPO', '').strip() # Where PR will be created + forked_repo = config.get('FORKED_REPO', '').strip() # User's fork where changes will be made + local_repo_path = config.get('LOCAL_REPO_PATH', '').strip() + + if not github_token and not self.dry_run_var.get(): + messagebox.showerror("Error", "Please configure your GitHub token in Settings or enable dry run mode") + return + + if not target_repo: + messagebox.showerror("Configuration Error", "GitHub target repository not configured.") + return + + # Use forked repo for changes, fall back to target repo if not specified + source_repo = forked_repo if forked_repo else target_repo + + # Check if AI provider is configured to determine workflow + ai_provider = config.get('AI_PROVIDER', 'none').strip().lower() + use_ai_workflow = ai_provider and ai_provider not in ['none', ''] + + # If using AI workflow, automatically ensure local repository exists + if use_ai_workflow and local_repo_path: + work_item_repo = self._get_work_item_repository(current_item) + if work_item_repo: + self.logger.log(f"🔄 AI workflow detected - ensuring repository {work_item_repo} is available locally...") + try: + self._ensure_local_repo(work_item_repo, local_repo_path, github_token) + except Exception as e: + self.logger.log(f"⚠️ Could not ensure local repository: {str(e)}") + # Continue anyway - the AI workflow may still work + + # Determine if creating issue or PR + is_uuf = current_item.get('source') == 'UUF' + create_pr = self.action_type_var.get() == "Create PR" + + # Start appropriate workflow in separate thread + if is_uuf or (create_pr and not use_ai_workflow): + # Use cross-repo workflow for UUF items or PRs without AI + thread = threading.Thread(target=self._process_cross_repo_pr, args=(source_repo, target_repo)) + elif create_pr and use_ai_workflow: + # Use AI-assisted workflow for PRs with AI provider configured + thread = threading.Thread(target=self._process_github_pr_with_verification, args=(target_repo, source_repo)) + else: + # Create GitHub issue + thread = threading.Thread(target=self._process_github_issue) + + thread.daemon = True + thread.start() + + except Exception as e: + self.logger.log(f"❌ Error in _create_github_resource: {str(e)}") + messagebox.showerror("Error", f"Failed to create GitHub resource: {str(e)}") + + def _process_cross_repo_pr(self, source_repo: str, target_repo: str): + """Process cross-repository PR creation with auto-cloning""" + try: + self.go_button.config(state='disabled') + self.progress.start() + + # Get current work item and config + current_item = self.current_work_items[self.current_item_index] + config = self.config_manager.get_config() + github_token = config.get('GITHUB_PAT', '') + local_repo_path = config.get('LOCAL_REPO_PATH', '') + + # If no source repo specified, try to auto-detect from forked repo config + if not source_repo or source_repo == target_repo: + source_repo = config.get('FORKED_REPO', '') + if not source_repo: + # Try to extract from document URL or use target repo + github_info = current_item.get('github_info', {}) + doc_url = github_info.get('mydoc_url', '') + if doc_url and 'github.com' in doc_url: + # Try to detect repo from URL + source_repo = self._detect_repo_from_url(doc_url, github_token) + + if not source_repo: + source_repo = target_repo + + # Parse repository information + try: + if '/' not in target_repo: + raise ValueError("Invalid target repository format") + target_owner, target_repo_name = target_repo.split('/', 1) + + if '/' not in source_repo: + raise ValueError("Invalid source repository format") + source_owner, source_repo_name = source_repo.split('/', 1) + except ValueError as e: + self.logger.log(f"❌ Repository format error: {e}") + messagebox.showerror("Configuration Error", + f"Invalid repository format. Use 'owner/repo' format.\n" + f"Target: {target_repo}\nSource: {source_repo}") + return + + # Check if local repository exists, clone if needed + if local_repo_path and source_owner != target_owner: + local_source_path = self._ensure_local_repo(source_repo, local_repo_path, github_token) + if local_source_path: + self.logger.log(f"Using local repository: {local_source_path}") + + # Initialize GitHub API + github_api = self.app.create_github_api(github_token) + github_info = current_item['github_info'] + + # Create a unique branch name + from .utils import PRNumberManager + pr_number = PRNumberManager.get_next_pr_number("cross_repo") + branch_name = f"docs-update-{pr_number}" + + self.logger.log("=== Starting Cross-Repository PR Creation ===") + self.logger.log(f"Source Repository: {source_owner}/{source_repo_name}") + self.logger.log(f"Target Repository: {target_owner}/{target_repo_name}") + self.logger.log(f"Branch Name: {branch_name}") + + # Step 1: Create branch in source repository with placeholder commit + self.logger.log("Creating branch with placeholder commit in source repository...") + + # Build instructions for the placeholder + instructions = f""" +Work Item #{current_item.get('id', 'unknown')}: {current_item.get('title', 'Update documentation')} + +**Description:** +{current_item.get('description', 'No description available')} + +**Changes needed:** +{current_item.get('new_text', 'See work item details')} +""" + + if not github_api.create_branch_with_placeholder(source_owner, source_repo_name, branch_name, instructions): + self.logger.log("❌ Failed to create branch with placeholder in source repository") + messagebox.showerror("Error", "Failed to create branch with placeholder in source repository.") + return + + # Step 2: Make documentation changes if AI provider is configured + ai_provider = config.get('AI_PROVIDER', 'none').strip().lower() + if ai_provider and ai_provider not in ['none', '']: + self.logger.log(f"AI provider ({ai_provider}) configured - attempting AI-assisted changes...") + + # Try to make documentation changes if we have a file path + if github_info.get('file_path'): + self.logger.log("Making AI-assisted documentation changes...") + + file_path = github_info['file_path'] + old_text = current_item.get('text_to_change', '') + new_text = current_item.get('new_text', '') + commit_message = f"Update documentation - Work Item #{current_item.get('id', 'unknown')}" + + if github_api.make_documentation_change( + source_owner, source_repo_name, branch_name, + file_path, old_text, new_text, commit_message + ): + self.logger.log("✅ Documentation changes committed successfully") + else: + self.logger.log("⚠️ Failed to make documentation changes, continuing with PR creation...") + else: + # No file path specified, but AI provider is configured + # The AI-assisted workflow should handle this in the full PR creation process + self.logger.log("ℹ️ AI provider configured but no specific file path - will use AI in PR workflow") + else: + self.logger.log("ℹ️ Using placeholder commit for PR creation (no AI provider configured)") + + # Step 3: Create Pull Request + from .utils import ContentBuilders + pr_title = ContentBuilders.build_pr_title(current_item) + pr_body = ContentBuilders.build_pr_body(current_item, github_info) + + if source_owner != target_owner or source_repo_name != target_repo_name: + # Cross-repository PR + self.logger.log("Creating cross-repository pull request...") + pr_id, pr_url, pr_num = github_api.create_cross_repo_pull_request( + source_owner, source_repo_name, target_owner, target_repo_name, + pr_title, pr_body, branch_name + ) + else: + # Same repository PR + self.logger.log("Creating pull request in same repository...") + target_repo_id = github_api.get_repo_id(target_owner, target_repo_name) + pr_id, pr_url, pr_num = github_api.create_pull_request( + target_repo_id, pr_title, pr_body, branch_name + ) + + # Step 4: Handle GitHub Copilot workflow based on AI provider setting + ai_provider = config.get('AI_PROVIDER', 'none').strip().lower() + + if ai_provider and ai_provider not in ['none', '']: + # AI provider is configured - skip Copilot assignment and comments + self.logger.log(f"✅ Using AI provider ({ai_provider}) - Skipping GitHub Copilot @mention workflow") + else: + # No AI provider - use GitHub Copilot workflow + self.logger.log("Using GitHub Copilot workflow (no AI provider configured)") + + # Assign to GitHub Copilot if available + copilot_actor_id, copilot_login = github_api.get_copilot_actor_id(target_owner, target_repo_name) + if copilot_actor_id: + self.logger.log(f"Assigning PR to GitHub Copilot ({copilot_login})...") + success = github_api.assign_to_copilot(pr_id, [copilot_actor_id]) + if not success: + self.logger.log("ℹ️ Copilot assignment failed due to permissions - this is normal for many repositories") + self.logger.log(" The @copilot comment below will still notify Copilot to work on the PR") + else: + self.logger.log("ℹ️ GitHub Copilot not available for assignment in this repository") + + # Add Copilot comment with instructions + self.logger.log("Adding Copilot instruction comment...") + file_path = github_info.get('file_path', '') + + # Extract file path from GitHub URLs if not already set + if not file_path: + # Try extracting from mydoc_url if it's a GitHub URL + mydoc_url = github_info.get('mydoc_url', '') + if mydoc_url: + extracted_path = self._extract_file_path_from_github_url(mydoc_url) + if extracted_path: + file_path = extracted_path + self.logger.log(f"Extracted file path from GitHub URL: {file_path}") + else: + file_path = f"File path not specified in work item (URL: {mydoc_url})" + else: + file_path = "See work item description for file details" + + # Get custom instructions from config + custom_instructions = config.get('CUSTOM_INSTRUCTIONS', '') + + github_api.add_copilot_comment( + target_owner, target_repo_name, pr_num, + file_path, + current_item.get('text_to_change', ''), + current_item.get('new_text', ''), + branch_name, + str(current_item.get('id', 'unknown')), + current_item.get('source', 'Work Item'), + github_info.get('mydoc_url', ''), + custom_instructions + ) + + self.logger.log(f"✅ @copilot comment added with work instructions") + if copilot_actor_id: + self.logger.log(f"📋 Note: Check the PR to see if Copilot assignment worked or needs manual assignment") + + self.logger.log(f"✅ Cross-repository PR created successfully: {pr_url}") + + # Show success dialog with hyperlink + self.root.after(0, lambda: HyperlinkDialog( + self.root, + "PR Created Successfully!", + f"Pull request created successfully!\n\n" + f"Source: {source_owner}/{source_repo_name}:{branch_name}\n" + f"Target: {target_owner}/{target_repo_name}\n" + f"PR Number: #{pr_num}", + pr_url + ).show()) + + except Exception as e: + error_msg = f"Failed to create cross-repository PR: {str(e)}" + self.logger.log(f"❌ {error_msg}") + self.root.after(0, lambda: messagebox.showerror("Error", error_msg)) + finally: + self.root.after(0, lambda: self.progress.stop()) + self.root.after(0, lambda: self.go_button.config(state='normal')) + + def _ensure_local_repo(self, repo_name: str, local_path: str, github_token: str) -> Optional[str]: + """Ensure local repository exists, clone if needed""" + try: + from .utils import LocalRepositoryScanner + + repo_folder = repo_name.split('/')[-1] # Get just the repo name + local_repo_path = os.path.join(local_path, repo_folder) + + if os.path.exists(local_repo_path): + # Check if it's actually a Git repo + if os.path.exists(os.path.join(local_repo_path, '.git')): + self.logger.log(f"Local repository already exists: {local_repo_path}") + return local_repo_path + else: + self.logger.log(f"Directory exists but not a Git repo: {local_repo_path}") + + # Need to clone + self.logger.log(f"Cloning repository {repo_name} to {local_repo_path}") + repo_url = f"https://github.com/{repo_name}.git" + + if LocalRepositoryScanner.clone_repository(repo_url, local_path, repo_name): + return local_repo_path + else: + self.logger.log(f"❌ Failed to clone repository {repo_name}") + return None + + except Exception as e: + self.logger.log(f"❌ Error ensuring local repo: {str(e)}") + return None + + def _detect_repo_from_url(self, doc_url: str, github_token: str) -> str: + """Detect user's fork repository from document URL""" + try: + # Extract the base repo from URL + from urllib.parse import urlparse + parsed = urlparse(doc_url) + + if 'docs.microsoft.com' in parsed.netloc: + # Try to map Microsoft Docs URL to repository + if 'fabric' in doc_url.lower(): + base_repo = 'fabric-docs' + elif 'azure' in doc_url.lower(): + base_repo = 'azure-docs' + elif 'powerbi' in doc_url.lower(): + base_repo = 'powerbi-docs' + else: + return '' + + # Get user's forks to find matching repo + github_api = self.app.create_github_api(github_token) + user_forks = github_api.get_user_forks() + + for fork in user_forks: + if base_repo in fork: + self.logger.log(f"Auto-detected forked repository: {fork}") + return fork + + except Exception as e: + self.logger.log(f"Error detecting repo from URL: {str(e)}") + + return '' + + def _get_work_item_repository(self, work_item: Dict[str, Any]) -> str: + """Extract repository name from work item""" + try: + # First check if github_info has repo information + github_info = work_item.get('github_info', {}) + if github_info.get('owner') and github_info.get('repo'): + return f"{github_info['owner']}/{github_info['repo']}" + + # Try to detect from mydoc_url + doc_url = work_item.get('mydoc_url', '') + if doc_url and 'github.com' in doc_url: + # Parse GitHub URL to extract repo + from urllib.parse import urlparse + parsed = urlparse(doc_url) + path_parts = parsed.path.strip('/').split('/') + if len(path_parts) >= 2: + return f"{path_parts[0]}/{path_parts[1]}" + + # Try to infer from docs URL + if doc_url and 'docs.microsoft.com' in doc_url: + if 'fabric' in doc_url.lower(): + return 'microsoftdocs/fabric-docs' + elif 'azure' in doc_url.lower(): + return 'microsoftdocs/azure-docs' + elif 'powerbi' in doc_url.lower(): + return 'microsoftdocs/powerbi-docs' + + return '' + + except Exception as e: + self.logger.log(f"Error extracting repository from work item: {str(e)}") + return '' + + def _process_github_issue(self): + """Process GitHub issue creation""" + try: + self.go_button.config(state='disabled') + self.progress.start() + + # Get current work item + current_item = self.current_work_items[self.current_item_index] + github_info = current_item['github_info'] + + # Get configuration + config = self.config_manager.get_config() + github_token = config.get('GITHUB_PAT', '').strip() + + # Get dry run setting from config (most up-to-date value) + dry_run_config = config.get('DRY_RUN', 'false') + is_dry_run = str(dry_run_config).lower() in ('true', '1', 'yes', 'on') + + self.logger.log(f"=== Creating GitHub Issue for {current_item.get('source', 'Azure DevOps')} item #{current_item['id']} ===") + if is_dry_run: + self.logger.log("🧪 DRY RUN MODE ENABLED - No actual changes will be made") + self.update_status("Creating GitHub issue...") + + # Create GitHub API instance + from .github_api import GitHubAPI + from .utils import ContentBuilders + + github_api = GitHubAPI(github_token, self.logger, is_dry_run) + + # Get repository ID + owner = github_info['owner'] + repo = github_info['repo'] + + self.logger.log(f"Target repository: {owner}/{repo}") + repo_id = github_api.get_repo_id(owner, repo) + + # Build issue content + issue_title = ContentBuilders.build_issue_title(current_item) + issue_body = ContentBuilders.build_issue_body(current_item, github_info) + + self.logger.log(f"Creating issue: {issue_title}") + + # Create the issue + issue_id, issue_url, issue_number = github_api.create_issue(repo_id, issue_title, issue_body) + + self.logger.log(f"✅ Issue created successfully: {issue_url}") + self.update_status(f"Issue #{issue_number} created successfully!") + + # Get Copilot actor ID and assign to Copilot if available + copilot_id, copilot_login = github_api.get_copilot_actor_id(owner, repo) + + if copilot_id and issue_id: + github_api.assign_to_copilot(issue_id, [copilot_id]) + self.logger.log("✅ Assigned to Copilot") + else: + self.logger.log("⚠️ Skipped assigning to Copilot (not found)") + + # Update work item status + current_item['status'] = f'Issue #{issue_number} created' + current_item['github_url'] = issue_url + self._update_all_items_tree() + + # Link back to Azure DevOps if applicable (non-critical) + if current_item.get('source') == 'Azure DevOps' and self.azure_api: + try: + link_title = f"GitHub Issue #{issue_number}" + success = self.azure_api.add_github_link_to_work_item( + str(current_item['id']), + issue_url, + link_title + ) + if not success: + self.logger.log("⚠️ Could not link issue back to Azure DevOps work item (non-critical)") + self.logger.log(" Possible causes: PAT expired, insufficient permissions, or work item locked") + self.logger.log(" The issue was created successfully - you can manually link it if needed") + except Exception as e: + self.logger.log(f"⚠️ Could not link issue to Azure DevOps (non-critical): {str(e)}") + self.logger.log(" The issue was created successfully - you can manually link it if needed") + + # Show success dialog with clickable link + HyperlinkDialog( + self.root, + "Issue Created", + f"GitHub Issue #{issue_number} has been created successfully!", + issue_url + ).show() + + except Exception as e: + error_msg = f"Error creating GitHub issue: {str(e)}" + self.logger.log(f"❌ {error_msg}") + self.update_status("Issue creation failed!") + messagebox.showerror("Issue Creation Error", error_msg) + finally: + self.progress.stop() + self.go_button.config(state='normal') + + def _process_github_pr_with_verification(self, target_repo: str, source_repo: str): + """Process GitHub PR creation with verified repositories""" + try: + self.go_button.config(state='disabled') + self.progress.start() + + # Get current work item + current_item = self.current_work_items[self.current_item_index] + github_info = current_item['github_info'] + + self.logger.log(f"=== Creating GitHub PR for {current_item.get('source', 'Azure DevOps')} item #{current_item['id']} ===") + self.logger.log(f"Target Repository: {target_repo}") + self.logger.log(f"Source Repository: {source_repo}") + self.update_status("Creating GitHub PR...") + + # Get configuration + config = self.config_manager.get_config() + github_token = config.get('GITHUB_PAT', '').strip() + ai_provider = config.get('AI_PROVIDER', 'none').strip().lower() + + # Get dry run setting from config (most up-to-date value) + dry_run_config = config.get('DRY_RUN', 'false') + is_dry_run = str(dry_run_config).lower() in ('true', '1', 'yes', 'on') + + if is_dry_run: + self.logger.log("🧪 DRY RUN MODE ENABLED - No actual changes will be made") + + # Update config temporarily for this workflow + temp_config = config.copy() + temp_config['GITHUB_REPO'] = target_repo + temp_config['FORKED_REPO'] = source_repo + + # Check if AI provider is configured + if ai_provider and ai_provider not in ['none', '']: + # Use AI-assisted workflow with verified repos + self._process_github_pr_with_ai(current_item, github_info, temp_config) + return + + # Otherwise use Copilot workflow with verified repos + self.logger.log("Using GitHub Copilot workflow with verified repositories") + + # Create GitHub API instance + from .github_api import GitHubAPI + from .utils import ContentBuilders + + github_api = GitHubAPI(github_token, self.logger, is_dry_run) + + # Continue with the standard PR creation but using verified repos + # Parse repository information + if '/' not in target_repo: + raise ValueError("Invalid target repository format") + target_owner, target_repo_name = target_repo.split('/', 1) + + if '/' not in source_repo: + raise ValueError("Invalid source repository format") + source_owner, source_repo_name = source_repo.split('/', 1) + + # Get repository ID for API calls + repository_id = github_api.get_repo_id(target_owner, target_repo_name) + + # Build PR content + builders = ContentBuilders() + pr_title = builders.build_pr_title(current_item) + pr_body = builders.build_pr_body(current_item, github_info) + + # Create branch and PR + from .utils import PRNumberManager + pr_number = PRNumberManager.get_next_pr_number(f"{source_owner}_{source_repo_name}") + branch_name = f"docs-update-{pr_number}" + + # Create branch in source repo + if github_api.create_branch_from_main(source_owner, source_repo_name, branch_name): + self.logger.log(f"✅ Branch '{branch_name}' created in {source_owner}/{source_repo_name}") + + # Create cross-repo PR + pr_url, pr_html_url, pr_num = github_api.create_cross_repo_pull_request( + source_owner, source_repo_name, target_owner, target_repo_name, + branch_name, pr_title, pr_body + ) + + if pr_url: + self.logger.log(f"✅ Pull request created: {pr_html_url}") + + # Add Copilot comment with proper parameters + file_path = github_info.get('file_path', '') + + # Extract file path from GitHub URLs if not already set + if not file_path: + # Try extracting from mydoc_url if it's a GitHub URL + mydoc_url = github_info.get('mydoc_url', '') + if mydoc_url: + extracted_path = self._extract_file_path_from_github_url(mydoc_url) + if extracted_path: + file_path = extracted_path + self.logger.log(f"Extracted file path from GitHub URL: {file_path}") + else: + file_path = f"File path not specified in work item (URL: {mydoc_url})" + else: + file_path = "See work item description for file details" + + # Get custom instructions from config + custom_instructions = config.get('CUSTOM_INSTRUCTIONS', '') + + github_api.add_copilot_comment( + target_owner, target_repo_name, pr_num, + file_path, + current_item.get('text_to_change', ''), + current_item.get('new_text', ''), + branch_name, + str(current_item.get('id', 'unknown')), + current_item.get('source', 'Work Item'), + github_info.get('mydoc_url', ''), + custom_instructions + ) + + # Show success dialog + dialog = HyperlinkDialog( + self.root, + "PR Created Successfully", + f"Pull request #{pr_num} has been created successfully:", + pr_html_url + ) + dialog.show() + + self.update_status(f"PR #{pr_num} created successfully") + else: + messagebox.showerror("Error", "Failed to create pull request") + + except Exception as e: + self.logger.log(f"❌ Error creating GitHub PR: {str(e)}") + messagebox.showerror("Error", f"Failed to create GitHub PR: {str(e)}") + + finally: + self.progress.stop() + self.go_button.config(state='normal') + + def _process_github_pr(self): + """Process GitHub PR creation""" + try: + self.go_button.config(state='disabled') + self.progress.start() + + # Get current work item + current_item = self.current_work_items[self.current_item_index] + github_info = current_item['github_info'] + + self.logger.log(f"=== Creating GitHub PR for {current_item.get('source', 'Azure DevOps')} item #{current_item['id']} ===") + self.update_status("Creating GitHub PR...") + + # Get configuration + config = self.config_manager.get_config() + github_token = config.get('GITHUB_PAT', '').strip() + ai_provider = config.get('AI_PROVIDER', 'none').strip().lower() + + # Get dry run setting from config (most up-to-date value) + dry_run_config = config.get('DRY_RUN', 'false') + is_dry_run = str(dry_run_config).lower() in ('true', '1', 'yes', 'on') + + if is_dry_run: + self.logger.log("🧪 DRY RUN MODE ENABLED - No actual changes will be made") + + # Check if AI provider is configured + if ai_provider and ai_provider not in ['none', '']: + # Use AI-assisted workflow + self._process_github_pr_with_ai(current_item, github_info, config) + return + + # Otherwise use Copilot workflow + self.logger.log("Using GitHub Copilot workflow (no AI provider configured)") + + # Create GitHub API instance + from .github_api import GitHubAPI + from .utils import ContentBuilders + + github_api = GitHubAPI(github_token, self.logger, is_dry_run) + + # Get UPSTREAM repository info (where PR will be created) + upstream_repo = config.get('GITHUB_REPO', '').strip() + if not upstream_repo or '/' not in upstream_repo: + raise ValueError("GITHUB_REPO not configured. Set it in Settings (e.g., microsoft/fabric-docs-pr)") + + upstream_parts = upstream_repo.split('/', 1) + upstream_owner = upstream_parts[0].strip() + upstream_repo_name = upstream_parts[1].strip() + + self.logger.log(f"Upstream repository (for PR): {upstream_owner}/{upstream_repo_name}") + + # Get FORK repository info (where branch will be created) + fork_owner = github_info['owner'] + fork_repo = github_info['repo'] + + self.logger.log(f"Fork repository (for branch): {fork_owner}/{fork_repo}") + + # Get upstream repository ID (for creating PR) + upstream_repo_id = github_api.get_repo_id(upstream_owner, upstream_repo_name) + + # Generate unique branch name + pr_number = self.config_manager.get_next_pr_number('gh_copilot') + source_prefix = 'uuf' if current_item.get('source') == 'UUF' else 'ab' + branch_name = f"{source_prefix}-{current_item['id']}-pr-{pr_number}" + + self.logger.log(f"Creating branch on fork: {branch_name}") + + # Extract file path from GitHub URL + file_path = None + if github_info.get('original_content_git_url'): + # Parse file path from URL + import re + url = github_info['original_content_git_url'] + # Match pattern: .../blob/branch/path/to/file.md + match = re.search(r'/blob/[^/]+/(.+)$', url) + if match: + file_path = match.group(1) + self.logger.log(f"Extracted file path: {file_path}") + + # Build PR content + pr_title = ContentBuilders.build_pr_title(current_item) + pr_body = ContentBuilders.build_pr_body(current_item, github_info) + + # Build instructions for placeholder commit + instructions = f"""Update documentation file: {file_path or 'See PR description'} + +Current text to replace: +{current_item['text_to_change']} + +Proposed new text: +{current_item['new_text']} +""" + + # Create branch on FORK with placeholder commit (so PR can be created) + self.logger.log("Creating branch on fork with placeholder commit...") + branch_created = github_api.create_branch_with_placeholder(fork_owner, fork_repo, branch_name, instructions) + + if not branch_created: + raise RuntimeError("Failed to create branch on fork. Check permissions and try again.") + + # Create the PR on UPSTREAM using fork's branch + # For fork workflow: head ref must be "fork-owner:branch-name" + head_ref = f"{fork_owner}:{branch_name}" + self.logger.log(f"Creating pull request on upstream: {pr_title}") + self.logger.log(f"PR head: {head_ref} -> base: main on {upstream_owner}/{upstream_repo_name}") + + _, pr_url, pr_number_actual = github_api.create_pull_request( + upstream_repo_id, pr_title, pr_body, head_ref, "main" + ) + + self.logger.log(f"✅ Pull request created: {pr_url}") + + # Add Copilot comment with instructions (to the fork's branch) + self.logger.log("Adding instructions for Copilot...") + + # Extract file path from GitHub URLs if not already set + if not file_path: + # Try extracting from mydoc_url if it's a GitHub URL + mydoc_url = current_item.get('mydoc_url', '') + if mydoc_url: + extracted_path = self._extract_file_path_from_github_url(mydoc_url) + if extracted_path: + file_path = extracted_path + self.logger.log(f"Extracted file path from GitHub URL: {file_path}") + else: + file_path = f"File path not specified in work item (URL: {mydoc_url})" + else: + file_path = "See work item description for file details" + + # Get custom instructions from config + config = self.config_manager.get_config() + custom_instructions = config.get('CUSTOM_INSTRUCTIONS', '') + + github_api.add_copilot_comment( + fork_owner, fork_repo, pr_number_actual, + file_path, + current_item['text_to_change'], + current_item['new_text'], + branch_name, + str(current_item['id']), + current_item.get('source'), + current_item.get('mydoc_url'), + custom_instructions + ) + + self.logger.log(f"✅ PR #{pr_number_actual} created successfully with Copilot instructions") + self.update_status(f"PR #{pr_number_actual} created successfully!") + + # Update work item status + current_item['status'] = f'PR #{pr_number_actual} created' + current_item['github_url'] = pr_url + self._update_all_items_tree() + + # Link back to Azure DevOps if applicable (non-critical) + if current_item.get('source') == 'Azure DevOps' and self.azure_api: + try: + link_title = f"GitHub PR #{pr_number_actual}" + success = self.azure_api.add_github_link_to_work_item( + str(current_item['id']), + pr_url, + link_title + ) + if not success: + self.logger.log("⚠️ Could not link PR back to Azure DevOps work item (non-critical)") + self.logger.log(" Possible causes: PAT expired, insufficient permissions, or work item locked") + self.logger.log(" The PR was created successfully - you can manually link it if needed") + except Exception as e: + self.logger.log(f"⚠️ Could not link PR to Azure DevOps (non-critical): {str(e)}") + self.logger.log(" The PR was created successfully - you can manually link it if needed") + + # Show success dialog with clickable link + HyperlinkDialog( + self.root, + "Pull Request Created", + f"GitHub PR #{pr_number_actual} has been created successfully!\n\n" + f"Copilot has been instructed to make the requested changes.", + pr_url + ).show() + + except Exception as e: + error_msg = f"Error creating GitHub PR: {str(e)}" + self.logger.log(f"❌ {error_msg}") + self.update_status("PR creation failed!") + messagebox.showerror("PR Creation Error", error_msg) + finally: + self.progress.stop() + self.go_button.config(state='normal') + + def _process_github_pr_with_ai(self, current_item: Dict[str, Any], github_info: Dict[str, Any], config: Dict[str, Any]): + """Process GitHub PR creation using AI provider (ChatGPT/Claude)""" + try: + self.logger.log("=== Using AI-Assisted PR Creation ===") + + # Get AI configuration + ai_provider = config.get('AI_PROVIDER', '').strip().lower() + if ai_provider == 'claude': + api_key = config.get('CLAUDE_API_KEY', '').strip() + elif ai_provider in ['chatgpt', 'openai', 'gpt']: + api_key = config.get('OPENAI_API_KEY', '').strip() + elif ai_provider in ['github-copilot', 'copilot', 'github_copilot']: + api_key = config.get('GITHUB_TOKEN', '').strip() + else: + api_key = '' + github_token = config.get('GITHUB_PAT', '').strip() + local_repo_path = config.get('LOCAL_REPO_PATH', '').strip() or None + + if not api_key: + raise ValueError(f"No API key configured for {ai_provider}. Please configure in Settings.") + + self.logger.log(f"Using AI Provider: {ai_provider.upper()}") + + # Create AI manager + from .ai_manager import AIManager + ai_manager = AIManager(self.logger) + + # Create AI provider instance + ai_provider_instance = ai_manager.create_ai_provider(ai_provider, api_key) + if not ai_provider_instance: + raise ValueError(f"Failed to create {ai_provider} provider") + + # Create LocalGitManager + git_manager = ai_manager.create_local_git_manager(github_token) + if not git_manager: + raise ValueError("Failed to create git manager") + + # Get UPSTREAM repository info (where PR will be created) + upstream_repo = config.get('GITHUB_REPO', '').strip() + if not upstream_repo or '/' not in upstream_repo: + raise ValueError("GITHUB_REPO not configured. Set it in Settings (e.g., microsoft/fabric-docs-pr)") + + upstream_parts = upstream_repo.split('/', 1) + upstream_owner = upstream_parts[0].strip() + upstream_repo_name = upstream_parts[1].strip() + + self.logger.log(f"Upstream repository (for PR): {upstream_owner}/{upstream_repo_name}") + + # Get FORK repository info (where we work locally) + # Use github_info from document metadata as the fork + fork_owner = github_info['owner'] + fork_repo = github_info['repo'] + + self.logger.log(f"Fork repository (local work): {fork_owner}/{fork_repo}") + self.logger.log(f"Local repository base path: {local_repo_path}") + + # Extract file path from GitHub URL + file_path = None + if github_info.get('original_content_git_url'): + import re + url = github_info['original_content_git_url'] + match = re.search(r'/blob/[^/]+/(.+)$', url) + if match: + file_path = match.group(1) + self.logger.log(f"File to modify: {file_path}") + + if not file_path: + raise ValueError("Could not extract file path from document URL") + + # Generate unique branch name + pr_number = self.config_manager.get_next_pr_number(ai_provider) + source_prefix = 'uuf' if current_item.get('source') == 'UUF' else 'ab' + branch_name = f"{source_prefix}-{current_item['id']}-{ai_provider}-pr-{pr_number}" + + self.logger.log(f"Branch name: {branch_name}") + + # Build commit message + commit_message = f"Update {file_path}\n\nWork Item: {current_item['id']}\nTitle: {current_item['title']}" + + # Get custom instructions from config + custom_instructions = config.get('CUSTOM_INSTRUCTIONS', '').strip() or None + + # Make AI-assisted changes on FORK + self.logger.log("Starting AI-assisted workflow on fork...") + success, error_msg = git_manager.make_ai_assisted_change( + fork_owner, fork_repo, branch_name, + file_path, + current_item['text_to_change'], + current_item['new_text'], + commit_message, + ai_provider_instance, + local_repo_path, + custom_instructions + ) + + if not success: + raise RuntimeError(error_msg or "AI-assisted change failed") + + # Update the diff display with the actual git diff + try: + # Construct the full repository path for git diff + if local_repo_path: + full_repo_path = os.path.join(local_repo_path, fork_owner, fork_repo) + else: + # Fallback to default Downloads location + from pathlib import Path + full_repo_path = str(Path.home() / "Downloads" / "github_repos" / fork_owner / fork_repo) + + diff_content = git_manager.get_git_diff_from_repo(full_repo_path, branch_name) + if diff_content: + self.update_diff_display(diff_content) + self.logger.log("📋 Git diff content updated in View Diff tab") + else: + self.logger.log("⚠️ No git diff content found") + except Exception as e: + self.logger.log(f"⚠️ Could not update diff display: {e}") + + # Create PR on UPSTREAM repository + from .github_api import GitHubAPI + from .utils import ContentBuilders + + self.logger.log(f"Creating PR on upstream: {upstream_owner}/{upstream_repo_name}") + github_api = GitHubAPI(github_token, self.logger, False) + repo_id = github_api.get_repo_id(upstream_owner, upstream_repo_name) + + pr_title = ContentBuilders.build_pr_title(current_item) + pr_body = ContentBuilders.build_pr_body(current_item, github_info) + pr_body += f"\n\n---\n*Changes made by {ai_provider.upper()} via AI-assisted workflow*" + + # For fork workflow: head ref must be "fork-owner:branch-name" + head_ref = f"{fork_owner}:{branch_name}" + self.logger.log(f"Creating pull request: {pr_title}") + self.logger.log(f"PR head: {head_ref} -> base: main on {upstream_owner}/{upstream_repo_name}") + + _, pr_url, pr_number_actual = github_api.create_pull_request( + repo_id, pr_title, pr_body, head_ref, "main" + ) + + self.logger.log(f"✅ PR #{pr_number_actual} created successfully with AI-generated changes") + self.update_status(f"PR #{pr_number_actual} created successfully!") + + # Update work item status + current_item['status'] = f'PR #{pr_number_actual} created ({ai_provider.upper()})' + current_item['github_url'] = pr_url + self._update_all_items_tree() + + # Link back to Azure DevOps if applicable (non-critical) + if current_item.get('source') == 'Azure DevOps' and self.azure_api: + try: + link_title = f"GitHub PR #{pr_number_actual}" + success = self.azure_api.add_github_link_to_work_item( + str(current_item['id']), + pr_url, + link_title + ) + if not success: + self.logger.log("⚠️ Could not link PR back to Azure DevOps work item (non-critical)") + except Exception as e: + self.logger.log(f"⚠️ Could not link PR to Azure DevOps (non-critical): {str(e)}") + + # Show success dialog + HyperlinkDialog( + self.root, + "Pull Request Created", + f"GitHub PR #{pr_number_actual} has been created successfully!\n\n" + f"{ai_provider.upper()} has made the requested changes and pushed them to the branch.", + pr_url + ).show() + + except Exception as e: + error_msg = f"Error creating AI-assisted PR: {str(e)}" + self.logger.log(f"❌ {error_msg}") + self.update_status("AI-assisted PR creation failed!") + messagebox.showerror("PR Creation Error", error_msg) + finally: + self.progress.stop() + self.go_button.config(state='normal') + + def next_item(self): + """Navigate to next work item""" + if self.current_item_index < len(self.current_work_items) - 1: + self.current_item_index += 1 + self._display_current_item() + self._update_navigation_buttons() + + def previous_item(self): + """Navigate to previous work item""" + if self.current_item_index > 0: + self.current_item_index -= 1 + self._display_current_item() + self._update_navigation_buttons() + + def _on_item_select(self, event): + """Handle item selection in the All Work Items treeview""" + selection = self.items_tree.selection() + if selection: + self.selected_tree_item = selection[0] + self.select_item_button.config(state='normal') + else: + self.selected_tree_item = None + self.select_item_button.config(state='disabled') + + def _on_item_double_click(self, event): + """Handle double-click on item in the All Work Items treeview""" + selection = self.items_tree.selection() + if selection: + self.selected_tree_item = selection[0] + self._select_current_item() + + def _select_current_item(self): + """Select the highlighted item from the treeview as the current work item""" + if not self.selected_tree_item: + return + + try: + # Get the work item ID from the selected tree item + item_values = self.items_tree.item(self.selected_tree_item, 'values') + if not item_values: + return + + selected_work_item_id = item_values[0] # ID is in the first column + + # Debug logging + self.logger.log(f"Looking for work item ID: {selected_work_item_id} (type: {type(selected_work_item_id)})") + self.logger.log(f"Available work items: {len(self.current_work_items)}") + if self.current_work_items: + self.logger.log(f"Sample work item ID: {self.current_work_items[0]['id']} (type: {type(self.current_work_items[0]['id'])})") + + # Find the work item in the current_work_items list (which contains all loaded items) + if not self.current_work_items: + messagebox.showwarning("No Work Items", "No work items are loaded.") + return + + selected_work_item = None + for work_item in self.current_work_items: + # Convert both IDs to strings for comparison to handle type mismatches + if str(work_item['id']) == str(selected_work_item_id): + selected_work_item = work_item + break + + if not selected_work_item: + messagebox.showerror("Item Not Found", + f"Work item #{selected_work_item_id} was not found in the loaded work items.") + return + + # Find the index of the selected work item in the current list + selected_index = -1 + for i, work_item in enumerate(self.current_work_items): + if str(work_item['id']) == str(selected_work_item_id): + selected_index = i + break + + if selected_index == -1: + messagebox.showerror("Item Not Found", + f"Work item #{selected_work_item_id} was not found in the loaded work items.") + return + + # Set the current item index to the selected item (keeping the full list intact) + self.current_item_index = selected_index + + # Update the display + self._display_current_item() + self._update_navigation_buttons() + + # Switch to the main work item tab to show the selected item + self.notebook.select(0) # Select the first tab (main work item tab) + + # Log the selection + self.logger.log(f"📌 Selected work item #{selected_work_item_id} as current item") + self.logger.log(f"Title: {selected_work_item['title']}") + + except Exception as e: + self.logger.log(f"❌ Error selecting work item: {e}") + messagebox.showerror("Error", f"Failed to select work item:\n{str(e)}") + + def create_github_resource(self): + """Create GitHub issue or PR for current work item""" + return self._create_github_resource() + + def start_fetch_work_items(self): + """Start fetching work items""" + return self._start_fetch_work_items() + + def start_fetch_uuf_items(self): + """Start fetching UUF items""" + return self._start_fetch_uuf_items() + + def toggle_edit_mode(self): + """Toggle edit mode for the Proposed New Text field""" + return self._toggle_edit_mode() + + def on_work_item_hover_enter(self, event=None): + """Handle mouse enter on work item ID label""" + return self._on_work_item_hover_enter(event) + + def on_work_item_hover_leave(self, event=None): + """Handle mouse leave on work item ID label""" + return self._on_work_item_hover_leave(event) + + def open_work_item_url(self, event=None): + """Open the Azure DevOps work item URL in the browser""" + return self._open_work_item_url(event) + + def check_ai_modules_manual(self): + """Manually check AI modules""" + return self._check_ai_modules_manual() + + def open_settings(self): + """Open settings dialog""" + return self._open_settings() + + def update_action_button_text(self): + """Update action button text based on dropdown selection""" + action_type = self.action_type_var.get() + if action_type == "Create PR": + self.go_button.config(text="🚀 Create PR") + else: + self.go_button.config(text="🚀 Create Issue") + + def check_ai_provider_setup(self): + """Check AI provider setup and offer to install missing modules""" + try: + config = self.config_manager.get_config() + ai_provider = config.get('AI_PROVIDER', '').strip().lower() + + if not ai_provider or ai_provider == 'none' or ai_provider == '': + return # No AI provider selected + + # Check if this provider requires special modules + if ai_provider not in ['chatgpt', 'claude', 'anthropic', 'github-copilot', 'copilot', 'github_copilot']: + return # Unknown provider, skip check + + # Check module availability using AI manager + self.ai_manager.check_and_install_ai_modules(ai_provider, self.root) + + except Exception as e: + self.logger.log(f"Error checking AI provider setup: {str(e)}") + + def display_current_item(self): + """Display current work item (public method for compatibility)""" + return self._display_current_item() + + def update_navigation_buttons(self): + """Update navigation button states (public method for compatibility)""" + return self._update_navigation_buttons() + + def update_all_items_tree(self): + """Update all items tree (public method for compatibility)""" + return self._update_all_items_tree() + + def process_github_issue(self): + """Process GitHub issue creation (public method for compatibility)""" + return self._process_github_issue() + + def process_github_pr(self): + """Process GitHub PR creation (public method for compatibility)""" + return self._process_github_pr() + + def update_diff_display(self, diff_content): + """Update the diff display with AI-generated patch content""" + try: + self.diff_text.config(state='normal') + self.diff_text.delete('1.0', tk.END) + + if not diff_content or diff_content.strip() == "": + self.diff_text.insert(tk.END, "No diff content available yet.\nDiffs will be generated from git changes or you can load existing .diff files using the 'Find .diff Files' button.") + self.diff_text.config(state='disabled') + self.clear_diff_button.config(state='disabled') + return + + # Clean and validate diff content + diff_content = self._clean_diff_content(diff_content) + + # Parse and highlight diff content + lines = diff_content.split('\n') + for line in lines: + if line.startswith('---') or line.startswith('+++'): + self.diff_text.insert(tk.END, line + '\n', 'diff_file') + elif line.startswith('@@'): + self.diff_text.insert(tk.END, line + '\n', 'diff_line_numbers') + elif line.startswith('+') and not line.startswith('+++'): + self.diff_text.insert(tk.END, line + '\n', 'diff_add') + elif line.startswith('-') and not line.startswith('---'): + self.diff_text.insert(tk.END, line + '\n', 'diff_remove') + elif line.startswith('diff ') or line.startswith('index '): + self.diff_text.insert(tk.END, line + '\n', 'diff_header') + else: + self.diff_text.insert(tk.END, line + '\n', 'diff_context') + + self.diff_text.config(state='disabled') + self.clear_diff_button.config(state='normal') + + # Log the diff update + self.logger.log("✅ Diff content displayed in View Diff tab") + + except Exception as e: + self.logger.log(f"❌ Error updating diff display: {e}") + + def clear_diff_display(self): + """Clear the diff display""" + try: + self.diff_text.config(state='normal') + self.diff_text.delete('1.0', tk.END) + self.diff_text.insert(tk.END, "Diff cleared.\nUse 'Find .diff Files' button to load existing diff files from local repositories.") + self.diff_text.config(state='disabled') + self.clear_diff_button.config(state='disabled') + self.logger.log("🧹 Diff display cleared") + except Exception as e: + self.logger.log(f"❌ Error clearing diff display: {e}") + + def find_and_load_diff_files(self): + """Find and load existing .diff files from local repositories""" + try: + import os + import glob + from tkinter import messagebox + from pathlib import Path + + # Get local repo path from settings + local_repo_path = self.config_manager.get('LOCAL_REPO_PATH', '').strip() + if not local_repo_path or not os.path.exists(local_repo_path): + self.logger.log("⚠️ No local repo path configured or path doesn't exist") + messagebox.showwarning("No Local Repo Path", + "Please configure LOCAL_REPO_PATH in Settings to find diff files.") + return + + base_path = Path(local_repo_path) + diff_files = [] + + # First, try to find detected repositories (owner/repo structure) + detected_repos = [] + try: + # Look for owner/repo structure: base_path/owner/repo/.git + for owner_dir in base_path.iterdir(): + if not owner_dir.is_dir(): + continue + for repo_dir in owner_dir.iterdir(): + if not repo_dir.is_dir(): + continue + # Check if it's a git repo + git_dir = repo_dir / ".git" + if git_dir.exists(): + detected_repos.append(repo_dir) + self.logger.log(f"🔍 Scanning for diff files in: {owner_dir.name}/{repo_dir.name}") + except Exception as e: + self.logger.log(f"⚠️ Error scanning for repositories: {e}") + + # Search for .diff files in detected repositories first + if detected_repos: + for repo_path in detected_repos: + for root, dirs, files in os.walk(repo_path): + for file in files: + if file.endswith('.diff'): + full_path = os.path.join(root, file) + relative_path = os.path.relpath(full_path, local_repo_path) + diff_files.append((relative_path, full_path)) + + # If no diff files found in detected repos, fallback to searching entire base path + if not diff_files: + self.logger.log("🔍 No diff files found in detected repositories, searching entire base path...") + for root, dirs, files in os.walk(local_repo_path): + for file in files: + if file.endswith('.diff'): + full_path = os.path.join(root, file) + relative_path = os.path.relpath(full_path, local_repo_path) + diff_files.append((relative_path, full_path)) + + if not diff_files: + self.logger.log("ℹ️ No .diff files found in local repositories") + messagebox.showinfo("No Diff Files Found", + f"No .diff files found in {local_repo_path}\n\nSearched in:\n" + + "\n".join([f" • {repo.parent.name}/{repo.name}" for repo in detected_repos]) if detected_repos else f" • {local_repo_path}") + return + + self.logger.log(f"📁 Found {len(diff_files)} diff file(s)") + + # If only one diff file, load it directly + if len(diff_files) == 1: + file_path = diff_files[0][1] + self._load_diff_file(file_path) + return + + # If multiple files, show selection dialog + self._show_diff_file_selection(diff_files) + + except Exception as e: + self.logger.log(f"❌ Error finding diff files: {e}") + messagebox.showerror("Error", f"Error finding diff files: {e}") + + def _show_diff_file_selection(self, diff_files): + """Show dialog to select which diff file to load""" + try: + import tkinter as tk + from tkinter import ttk, messagebox + + # Create selection dialog + selection_window = tk.Toplevel(self.root) + selection_window.title("Select Diff File") + selection_window.geometry("600x400") + selection_window.transient(self.root) + selection_window.grab_set() + + # Center the window + selection_window.geometry("+%d+%d" % + (self.root.winfo_rootx() + 50, self.root.winfo_rooty() + 50)) + + frame = ttk.Frame(selection_window) + frame.pack(fill=tk.BOTH, expand=True, padx=10, pady=10) + + # Title + ttk.Label(frame, text="Select a .diff file to view:", + font=('Arial', 11, 'bold')).pack(anchor=tk.W, pady=(0, 10)) + + # Listbox with scrollbar + listbox_frame = ttk.Frame(frame) + listbox_frame.pack(fill=tk.BOTH, expand=True) + + scrollbar = ttk.Scrollbar(listbox_frame) + scrollbar.pack(side=tk.RIGHT, fill=tk.Y) + + listbox = tk.Listbox(listbox_frame, yscrollcommand=scrollbar.set, + font=('Courier New', 9)) + listbox.pack(side=tk.LEFT, fill=tk.BOTH, expand=True) + scrollbar.config(command=listbox.yview) + + # Populate listbox + for relative_path, full_path in diff_files: + listbox.insert(tk.END, relative_path) + + # Buttons + button_frame = ttk.Frame(frame) + button_frame.pack(fill=tk.X, pady=(10, 0)) + + def load_selected(): + selection = listbox.curselection() + if selection: + selected_file = diff_files[selection[0]][1] + selection_window.destroy() + self._load_diff_file(selected_file) + else: + messagebox.showwarning("No Selection", "Please select a diff file to load.") + + ttk.Button(button_frame, text="Load Selected", command=load_selected).pack(side=tk.LEFT) + ttk.Button(button_frame, text="Cancel", + command=selection_window.destroy).pack(side=tk.LEFT, padx=(10, 0)) + + # Double-click to load + listbox.bind('', lambda e: load_selected()) + + except Exception as e: + self.logger.log(f"❌ Error showing diff file selection: {e}") + + def _load_diff_file(self, file_path): + """Load and display a specific diff file""" + try: + with open(file_path, 'r', encoding='utf-8') as f: + diff_content = f.read() + + if diff_content.strip(): + self.update_diff_display(diff_content) + self.logger.log(f"✅ Loaded diff file: {os.path.basename(file_path)}") + else: + self.logger.log(f"⚠️ Diff file is empty: {file_path}") + + except Exception as e: + self.logger.log(f"❌ Error loading diff file {file_path}: {e}") + from tkinter import messagebox + messagebox.showerror("Error", f"Error loading diff file:\n{e}") + + def _clean_diff_content(self, diff_content: str) -> str: + """Clean and fix common issues with AI-generated diff content""" + try: + lines = diff_content.split('\n') + cleaned_lines = [] + + for i, line in enumerate(lines): + # Remove duplicate +++ lines that sometimes appear + if line.startswith('+++') and i > 0: + # Check if previous line was also +++ + prev_line = lines[i-1] if i > 0 else "" + if prev_line.startswith('+++'): + continue # Skip duplicate + + # Fix malformed file headers + if line.startswith('title:') and not line.startswith('---'): + # This looks like metadata that shouldn't be removed + continue + + cleaned_lines.append(line) + + cleaned_diff = '\n'.join(cleaned_lines) + + # If the diff looks seriously malformed, add a warning + if '+++' in cleaned_diff and cleaned_diff.count('+++') > 2: + warning = "⚠️ WARNING: This diff may have formatting issues. Please review carefully.\n\n" + return warning + cleaned_diff + + return cleaned_diff + + except Exception as e: + self.logger.log(f"⚠️ Error cleaning diff content: {e}") + return diff_content # Return original if cleaning fails \ No newline at end of file diff --git a/application/app_components/settings_dialog.py b/application/app_components/settings_dialog.py new file mode 100644 index 0000000..ae52532 --- /dev/null +++ b/application/app_components/settings_dialog.py @@ -0,0 +1,850 @@ +""" +Settings Dialog +GUI for configuring application settings +""" + +import tkinter as tk +import threading +from tkinter import ttk, messagebox, scrolledtext +from typing import Dict, Any, Optional +import sys +import os + + +class SettingsDialog: + """Settings configuration dialog""" + + def __init__(self, parent, config: Dict[str, Any], config_manager=None, cache_manager=None): + self.parent = parent + self.config = config.copy() + self.config_manager = config_manager + self.cache_manager = cache_manager + self.result = None + self.entries = {} + + # Create dialog + self.dialog = tk.Toplevel(parent) + self.dialog.title("⚙️ Settings") + self.dialog.geometry("900x1000") + self.dialog.resizable(True, True) + self.dialog.transient(parent) + self.dialog.grab_set() + + self._create_widgets() + self._bind_events() + + def _create_widgets(self): + """Create dialog widgets""" + # Main frame with scrollbar + main_frame = ttk.Frame(self.dialog, padding="20") + main_frame.pack(fill=tk.BOTH, expand=True) + + # Create notebook for tabbed settings + notebook = ttk.Notebook(main_frame) + notebook.pack(fill=tk.BOTH, expand=True, pady=(0, 20)) + + # Create tabs + self._create_general_tab(notebook) + self._create_ai_tab(notebook) + self._create_dataverse_tab(notebook) + + # Buttons frame + buttons_frame = ttk.Frame(main_frame) + buttons_frame.pack(fill=tk.X, pady=(10, 0)) + + # Buttons + ttk.Button(buttons_frame, text="💾 Save Settings", command=self._save_clicked).pack(side=tk.RIGHT, padx=(5, 0)) + ttk.Button(buttons_frame, text="❌ Cancel", command=self._cancel_clicked).pack(side=tk.RIGHT) + ttk.Button(buttons_frame, text="🗑️ Clear Cache", command=self._clear_cache).pack(side=tk.LEFT, padx=(5, 0)) + ttk.Button(buttons_frame, text="Test Connection", command=self._test_connection).pack(side=tk.LEFT) + + # Center dialog after everything is created + self._center_dialog() + + def _create_general_tab(self, notebook): + """Create general settings tab""" + general_frame = ttk.Frame(notebook) + notebook.add(general_frame, text="General") + + # Scrollable frame + canvas = tk.Canvas(general_frame) + scrollbar = ttk.Scrollbar(general_frame, orient="vertical", command=canvas.yview) + scrollable_frame = ttk.Frame(canvas) + + scrollable_frame.bind( + "", + lambda e: canvas.configure(scrollregion=canvas.bbox("all")) + ) + + canvas.create_window((0, 0), window=scrollable_frame, anchor="nw") + canvas.configure(yscrollcommand=scrollbar.set) + + # Configure column weights for proper expansion + scrollable_frame.columnconfigure(1, weight=1) + + current_row = 0 + + # Azure DevOps section + self._create_section_header(scrollable_frame, current_row, "🔷 Azure DevOps Configuration") + current_row += 1 + + self._create_label_entry(scrollable_frame, current_row, "Query URL:", 'AZURE_DEVOPS_QUERY', width=60, multiline=True) + current_row += 1 + + self._create_label_entry(scrollable_frame, current_row, "Personal Access Token:", 'AZURE_DEVOPS_PAT', password=True, width=60) + current_row += 1 + + # GitHub section + self._create_section_header(scrollable_frame, current_row, "🐙 GitHub Configuration") + current_row += 1 + + self._create_label_entry(scrollable_frame, current_row, "Personal Access Token:", 'GITHUB_PAT', password=True, width=60) + current_row += 1 + + self._create_label_entry(scrollable_frame, current_row, "Target Repository (owner/repo):", 'GITHUB_REPO', width=60) + current_row += 1 + + self._create_forked_repo_dropdown(scrollable_frame, current_row) + current_row += 1 + + # General options section + self._create_section_header(scrollable_frame, current_row, "⚙️ General Options") + current_row += 1 + + self._create_dry_run_checkbox(scrollable_frame, current_row) + current_row += 1 + + self._create_label_entry(scrollable_frame, current_row, "Local Repo Path:", 'LOCAL_REPO_PATH', width=60) + current_row += 1 + + # Detected repos dropdown + ttk.Label(scrollable_frame, text="Detected Repos:", font=('Arial', 10, 'bold')).grid( + row=current_row, column=0, sticky=tk.W, pady=5, padx=10) + + # Frame for dropdown and refresh button + detected_frame = ttk.Frame(scrollable_frame) + detected_frame.grid(row=current_row, column=1, sticky=(tk.W, tk.E), pady=5, padx=10) + + self.detected_repos_var = tk.StringVar(value='Scanning...') + self.detected_repos_dropdown = ttk.Combobox(detected_frame, textvariable=self.detected_repos_var, + state='readonly', width=45) + self.detected_repos_dropdown.pack(side=tk.LEFT, fill=tk.X, expand=True) + self.detected_repos_dropdown.bind('<>', self._on_repo_selected) + + refresh_button = ttk.Button(detected_frame, text="🔄 Scan", command=self._scan_repos, width=8) + refresh_button.pack(side=tk.LEFT, padx=(5, 0)) + current_row += 1 + + # Help text for local repo path + repo_help = ttk.Label(scrollable_frame, + text="💡 Repository Setup Guide:\n" + " • Local Repo Path: Where your fork repos are cloned (e.g., C:\\git\\repos)\n" + " • Detected Repos: Shows your local fork (e.g., yourname/repo)\n" + " • Target Repository: Upstream repo for PRs (e.g., microsoft/repo)\n" + " • Fork Workflow: Work on your fork locally, create PRs to upstream", + font=('Arial', 9), foreground='gray', justify=tk.LEFT, wraplength=850) + repo_help.grid(row=current_row, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=(10, 20), padx=10) + current_row += 1 + + # Help text + help_text = ttk.Label(scrollable_frame, text="💡 Getting Started:\n" + "1. Set your Azure DevOps Query URL (copy from browser)\n" + "2. Create Personal Access Tokens for both services\n" + "3. Configure GitHub repositories:\n" + " • Target Repository: Where PRs will be created\n" + " • Forked Repository: Your fork where changes are made\n" + "4. Set Local Repo Path for automatic repository detection\n" + "5. Configure AI provider in the AI tab (optional)\n" + "6. Test your connection before processing items", + font=('Arial', 9), foreground='blue', justify=tk.LEFT, wraplength=850) + help_text.grid(row=current_row, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=(20, 30), padx=10) + + # Scan for repos after creating the UI + self.dialog.after(100, self._scan_repos) + + # Pack canvas and scrollbar + canvas.pack(side="left", fill="both", expand=True) + scrollbar.pack(side="right", fill="y") + + def _create_ai_tab(self, notebook): + """Create AI settings tab""" + ai_frame = ttk.Frame(notebook) + notebook.add(ai_frame, text="AI Providers") + + # Scrollable frame + canvas = tk.Canvas(ai_frame) + scrollbar = ttk.Scrollbar(ai_frame, orient="vertical", command=canvas.yview) + scrollable_frame = ttk.Frame(canvas) + + scrollable_frame.bind( + "", + lambda e: canvas.configure(scrollregion=canvas.bbox("all")) + ) + + canvas.create_window((0, 0), window=scrollable_frame, anchor="nw") + canvas.configure(yscrollcommand=scrollbar.set) + + # AI Provider section + self._create_section_header(scrollable_frame, 0, "🤖 AI Provider Configuration") + + # Provider dropdown + ttk.Label(scrollable_frame, text="AI Provider:", font=('Arial', 10, 'bold')).grid( + row=1, column=0, sticky=tk.W, pady=5, padx=10) + + self.ai_provider_var = tk.StringVar(value=self.config.get('AI_PROVIDER', 'none')) + provider_dropdown = ttk.Combobox(scrollable_frame, textvariable=self.ai_provider_var, + values=['none', 'claude', 'chatgpt', 'github-copilot'], state='readonly', width=47) + provider_dropdown.grid(row=1, column=1, sticky=(tk.W, tk.E), pady=5, padx=10) + self.entries['AI_PROVIDER'] = self.ai_provider_var + + # API Keys + self._create_label_entry(scrollable_frame, 2, "Claude API Key:", 'CLAUDE_API_KEY', password=True) + self._create_label_entry(scrollable_frame, 3, "ChatGPT API Key:", 'OPENAI_API_KEY', password=True) + self._create_label_entry(scrollable_frame, 4, "GitHub Token (for Copilot) [defaults to GitHub PAT]:", 'GITHUB_TOKEN', password=True) + + # Help text + help_text = ttk.Label(scrollable_frame, text="\n💡 Tips:\n" + "• Provider: Choose 'none' to disable AI (uses Copilot workflow)\n" + "• Claude: Get key at console.anthropic.com\n" + "• ChatGPT: Get key at platform.openai.com/api-keys\n" + "• GitHub Copilot: Uses GitHub Models API (requires GitHub token)\n" + "• GitHub Token: Auto-defaults to GitHub PAT if left empty\n" + "• Cost: ~$0.01-0.05 per PR with AI, free with 'none'\n" + "• AI providers clone repos locally to make changes before pushing", + font=('Arial', 9), foreground='blue', justify=tk.LEFT, wraplength=800) + help_text.grid(row=5, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=20, padx=10) + + # Pack canvas and scrollbar + canvas.pack(side="left", fill="both", expand=True) + scrollbar.pack(side="right", fill="y") + + def _create_dataverse_tab(self, notebook): + """Create Dataverse/PowerApp settings tab""" + dataverse_frame = ttk.Frame(notebook) + notebook.add(dataverse_frame, text="UUF/Dataverse") + + # Scrollable frame + canvas = tk.Canvas(dataverse_frame) + scrollbar = ttk.Scrollbar(dataverse_frame, orient="vertical", command=canvas.yview) + scrollable_frame = ttk.Frame(canvas) + + scrollable_frame.bind( + "", + lambda e: canvas.configure(scrollregion=canvas.bbox("all")) + ) + + canvas.create_window((0, 0), window=scrollable_frame, anchor="nw") + canvas.configure(yscrollcommand=scrollbar.set) + + # Dataverse section + self._create_section_header(scrollable_frame, 0, "📊 PowerApp/Dataverse Configuration") + self._create_label_entry(scrollable_frame, 1, "Environment URL:", 'DATAVERSE_ENVIRONMENT_URL', width=60, multiline=True) + self._create_label_entry(scrollable_frame, 2, "Table Name:", 'DATAVERSE_TABLE_NAME') + + # Azure AD section + self._create_section_header(scrollable_frame, 3, "🔐 Azure AD Configuration") + self._create_label_entry(scrollable_frame, 4, "Client ID:", 'AZURE_AD_CLIENT_ID', width=60) + self._create_label_entry(scrollable_frame, 5, "Client Secret:", 'AZURE_AD_CLIENT_SECRET', password=True, width=60) + self._create_label_entry(scrollable_frame, 6, "Tenant ID:", 'AZURE_AD_TENANT_ID', width=60) + + # Help text + help_text = ttk.Label(scrollable_frame, text="\n💡 UUF Integration:\n" + "• This section is only needed if you want to fetch UUF items\n" + "• UUF items are processed differently than Azure DevOps work items\n" + "• Environment URL: Your Dataverse environment\n" + "• Azure AD app must have appropriate permissions\n" + "• Contact your PowerApp administrator for these values\n" + "• Leave blank if not using UUF integration", + font=('Arial', 9), foreground='blue', justify=tk.LEFT, wraplength=800) + help_text.grid(row=7, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=20, padx=10) + + # Pack canvas and scrollbar + canvas.pack(side="left", fill="both", expand=True) + scrollbar.pack(side="right", fill="y") + + def _create_section_header(self, parent, row: int, text: str): + """Create a section header""" + header_frame = ttk.Frame(parent) + header_frame.grid(row=row, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=(20, 10), padx=10) + header_frame.columnconfigure(1, weight=1) + + ttk.Label(header_frame, text=text, font=('Arial', 12, 'bold')).grid(row=0, column=0, sticky=tk.W) + ttk.Separator(header_frame, orient='horizontal').grid(row=0, column=1, sticky=(tk.W, tk.E), padx=(10, 0)) + + def _create_label_entry(self, parent, row: int, label_text: str, config_key: str, + password: bool = False, width: int = 50, multiline: bool = False): + """Create a label and entry pair""" + ttk.Label(parent, text=label_text, font=('Arial', 10, 'bold')).grid( + row=row, column=0, sticky=tk.W, pady=5, padx=10) + + if multiline: + entry = scrolledtext.ScrolledText(parent, height=3, width=width) + entry.grid(row=row, column=1, sticky=(tk.W, tk.E), pady=5, padx=10) + entry.insert('1.0', self.config.get(config_key, '') or '') + elif password: + entry = ttk.Entry(parent, show="*", width=width) + entry.grid(row=row, column=1, sticky=(tk.W, tk.E), pady=5, padx=10) + + # Special handling for GITHUB_TOKEN - show placeholder if using default + if config_key == 'GITHUB_TOKEN': + github_token = self.config.get('GITHUB_TOKEN', '').strip() + github_pat = self.config.get('GITHUB_PAT', '').strip() + if not github_token and github_pat: + # Show placeholder for defaulted value, but don't actually set it + entry.config(foreground='gray') + entry.insert(0, '(using GitHub PAT)') + + # Add event handlers to clear placeholder on focus + def on_focus_in(event): + if entry.get() == '(using GitHub PAT)': + entry.delete(0, tk.END) + entry.config(foreground='black') + + def on_focus_out(event): + if not entry.get(): + entry.config(foreground='gray') + entry.insert(0, '(using GitHub PAT)') + + entry.bind('', on_focus_in) + entry.bind('', on_focus_out) + else: + entry.insert(0, github_token) + else: + entry.insert(0, self.config.get(config_key, '') or '') + else: + entry = ttk.Entry(parent, width=width) + entry.grid(row=row, column=1, sticky=(tk.W, tk.E), pady=5, padx=10) + entry.insert(0, self.config.get(config_key, '') or '') + + self.entries[config_key] = entry + parent.columnconfigure(1, weight=1) + + def _create_forked_repo_dropdown(self, parent, row: int): + """Create forked repository dropdown with local repo detection""" + ttk.Label(parent, text="Forked Repository:", font=('Arial', 10, 'bold')).grid( + row=row, column=0, sticky=tk.W, pady=5, padx=10) + + # Frame for dropdown and refresh button + dropdown_frame = ttk.Frame(parent) + dropdown_frame.grid(row=row, column=1, sticky=(tk.W, tk.E), pady=5, padx=10) + dropdown_frame.columnconfigure(0, weight=1) + + # Initial options + repo_options = [''] # Empty option + + # Add local repositories + local_repo_path = self.config.get('LOCAL_REPO_PATH', '') + if local_repo_path: + try: + from .utils import LocalRepositoryScanner + local_repos = LocalRepositoryScanner.scan_local_repos(local_repo_path) + if local_repos: + repo_options.append('--- Local Repositories ---') + repo_options.extend(local_repos) + except Exception as e: + print(f"Error scanning local repos: {e}") + + # Placeholder for user's forks (will be populated asynchronously) + self.forked_repos = [] + + self.forked_repo_var = tk.StringVar(value=self.config.get('FORKED_REPO', '')) + self.forked_repo_dropdown = ttk.Combobox(dropdown_frame, textvariable=self.forked_repo_var, + values=repo_options, width=50) + self.forked_repo_dropdown.grid(row=0, column=0, sticky=(tk.W, tk.E), padx=(0, 5)) + self.entries['FORKED_REPO'] = self.forked_repo_var + + # Refresh button + refresh_btn = ttk.Button(dropdown_frame, text="🔄", width=3, + command=self._refresh_forked_repos) + refresh_btn.grid(row=0, column=1) + + # Help text for forked repo + help_label = ttk.Label(parent, + text=" ℹ️ Your fork where changes will be made. Leave empty to auto-detect from document URL.", + font=('Arial', 9), foreground='gray') + help_label.grid(row=row+1, column=0, columnspan=3, sticky=tk.W, padx=10) + + # Start async loading of user's forks + self.dialog.after(100, self._load_user_forks_async) + + def _refresh_forked_repos(self): + """Refresh the forked repositories dropdown""" + self._load_user_forks_async() + + # Also refresh local repos + local_repo_path = self.config.get('LOCAL_REPO_PATH', '') + if local_repo_path: + try: + from .utils import LocalRepositoryScanner + local_repos = LocalRepositoryScanner.scan_local_repos(local_repo_path) + + # Update dropdown with current values plus refreshed local repos + current_values = list(self.forked_repo_dropdown['values']) + + # Remove old local repos section + if '--- Local Repositories ---' in current_values: + start_idx = current_values.index('--- Local Repositories ---') + # Find where GitHub repos start or end of list + end_idx = len(current_values) + for i in range(start_idx + 1, len(current_values)): + if current_values[i].startswith('--- ') and 'GitHub' in current_values[i]: + end_idx = i + break + + # Remove local repos section + current_values = current_values[:start_idx] + current_values[end_idx:] + + # Add refreshed local repos + if local_repos: + current_values.insert(1, '--- Local Repositories ---') + for i, repo in enumerate(local_repos): + current_values.insert(2 + i, repo) + + self.forked_repo_dropdown['values'] = current_values + + except Exception as e: + print(f"Error refreshing local repos: {e}") + + def _load_user_forks_async(self): + """Load user's GitHub forks asynchronously""" + def load_forks(): + try: + github_token = self.config.get('GITHUB_PAT', '') + if not github_token: + return + + from .github_api import GitHubGQL + github_api = GitHubGQL(github_token, dry_run=False) + self.forked_repos = github_api.get_user_forks() + + # Update dropdown on main thread + if hasattr(self, 'dialog') and self.dialog.winfo_exists(): + self.dialog.after(0, self._update_forked_dropdown) + + except Exception as e: + print(f"Error loading user forks: {e}") + + threading.Thread(target=load_forks, daemon=True).start() + + def _update_forked_dropdown(self): + """Update the forked repository dropdown with GitHub forks""" + try: + # Check if dialog and dropdown still exist + if not hasattr(self, 'dialog') or not self.dialog.winfo_exists(): + return + if not hasattr(self, 'forked_repo_dropdown') or not self.forked_repo_dropdown.winfo_exists(): + return + + current_values = list(self.forked_repo_dropdown['values']) + + # Remove old GitHub forks section if exists + if '--- Your GitHub Forks ---' in current_values: + start_idx = current_values.index('--- Your GitHub Forks ---') + current_values = current_values[:start_idx] + + # Add GitHub forks section + if self.forked_repos: + current_values.append('--- Your GitHub Forks ---') + current_values.extend(self.forked_repos) + + self.forked_repo_dropdown['values'] = current_values + + except Exception as e: + print(f"Error updating forked dropdown: {e}") + + def _create_dry_run_checkbox(self, parent, row: int): + """Create dry run checkbox""" + self.dry_run_var = tk.BooleanVar() + dry_run_value = self.config.get('DRY_RUN', 'false') + self.dry_run_var.set(str(dry_run_value).lower() in ('true', '1', 'yes', 'on')) + + dry_run_frame = ttk.Frame(parent) + dry_run_frame.grid(row=row, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=10, padx=10) + + dry_run_checkbox = ttk.Checkbutton( + dry_run_frame, + text="🧪 Dry Run Mode (Test without making changes)", + variable=self.dry_run_var + ) + dry_run_checkbox.pack(side=tk.LEFT) + + help_label = ttk.Label(dry_run_frame, + text=" ℹ️ Simulates operations without creating actual GitHub issues/PRs", + font=('Arial', 9), foreground='gray') + help_label.pack(side=tk.LEFT) + + self.entries['DRY_RUN'] = self.dry_run_var + + def _scan_repos(self): + """Scan work items to detect commonly used repositories""" + try: + # This is a placeholder - could be enhanced to actually scan work items + # and suggest repositories based on document URLs found + pass + except Exception as e: + print(f"Could not scan repositories: {e}") + + def _bind_events(self): + """Bind keyboard events""" + self.dialog.bind('', lambda e: self._save_clicked()) + self.dialog.bind('', lambda e: self._cancel_clicked()) + + # Set focus to first entry if available + if self.entries: + first_entry = next(iter(self.entries.values())) + if hasattr(first_entry, 'focus_set'): + first_entry.focus_set() + + def _test_connection(self): + """Test connection to configured services""" + # Get current values + config_values = self._get_config_values() + + results = [] + + # Test Azure DevOps + if config_values.get('AZURE_DEVOPS_QUERY') and config_values.get('AZURE_DEVOPS_PAT'): + try: + # Try to import and test Azure DevOps API + from .azure_devops_api import AzureDevOpsAPI + api = AzureDevOpsAPI(config_values.get('AZURE_DEVOPS_PAT')) + + # Basic connection test (this would need actual implementation) + results.append("Azure DevOps: ✅ Configuration looks valid") + except ImportError: + results.append("Azure DevOps: ⚠️ Configuration set (API module not available)") + except Exception as e: + results.append(f"Azure DevOps: ❌ Error - {str(e)}") + elif config_values.get('AZURE_DEVOPS_QUERY') or config_values.get('AZURE_DEVOPS_PAT'): + results.append("Azure DevOps: ⚠️ Incomplete configuration") + + # Test GitHub + if config_values.get('GITHUB_PAT'): + try: + # Try to import and test GitHub API + from .github_api import GitHubAPI + api = GitHubAPI(config_values.get('GITHUB_PAT')) + + # Basic connection test + results.append("GitHub: ✅ Token configured") + + if config_values.get('GITHUB_REPO'): + results.append(f"GitHub Repository: ✅ {config_values.get('GITHUB_REPO')}") + else: + results.append("GitHub Repository: ⚠️ Not configured") + + except ImportError: + results.append("GitHub: ⚠️ Token set (API module not available)") + except Exception as e: + results.append(f"GitHub: ❌ Error - {str(e)}") + else: + results.append("GitHub: ❌ No token configured") + + # Test AI Provider + ai_provider = config_values.get('AI_PROVIDER', 'none').lower() + if ai_provider and ai_provider != 'none': + try: + from .ai_manager import AIManager + ai_manager = AIManager() + available, missing = ai_manager.check_ai_module_availability(ai_provider) + + if available: + results.append(f"AI Provider ({ai_provider}): ✅ Available") + else: + results.append(f"AI Provider ({ai_provider}): ⚠️ Missing packages: {', '.join(missing)}") + except ImportError: + results.append(f"AI Provider ({ai_provider}): ⚠️ Configuration set (AI manager not available)") + else: + results.append("AI Provider: ℹ️ Disabled (using standard method)") + + # Show results + if results: + messagebox.showinfo("Connection Test Results", + "\n".join(results) + "\n\n💡 Full validation requires running the application.", + parent=self.dialog) + else: + messagebox.showwarning("Connection Test", "No configuration to test.", parent=self.dialog) + + def _center_dialog(self): + """Center the dialog over the parent window""" + self.dialog.update_idletasks() + + # Get parent window position and size + self.parent.update_idletasks() + x = self.parent.winfo_x() + (self.parent.winfo_width() // 2) - (self.dialog.winfo_width() // 2) + y = self.parent.winfo_y() + (self.parent.winfo_height() // 2) - (self.dialog.winfo_height() // 2) + + self.dialog.geometry(f"+{x}+{y}") + + def _get_config_values(self) -> Dict[str, Any]: + """Get configuration values from entries""" + config_values = {} + + for key, widget in self.entries.items(): + if isinstance(widget, tk.BooleanVar): + config_values[key] = 'true' if widget.get() else 'false' + elif isinstance(widget, tk.StringVar): + config_values[key] = widget.get().strip() + elif isinstance(widget, scrolledtext.ScrolledText): + config_values[key] = widget.get('1.0', tk.END).strip() + elif isinstance(widget, ttk.Combobox): + config_values[key] = widget.get().strip() + else: # Entry widget + value = widget.get().strip() + # Special handling for GITHUB_TOKEN placeholder + if key == 'GITHUB_TOKEN' and value == '(using GitHub PAT)': + value = '' # Save empty string when using placeholder + config_values[key] = value + + return config_values + + def _save_clicked(self): + """Handle save button click""" + try: + # Get configuration values + config_values = self._get_config_values() + + # Validate required fields + required_for_basic = ['AZURE_DEVOPS_QUERY', 'AZURE_DEVOPS_PAT', 'GITHUB_PAT'] + missing_basic = [field for field in required_for_basic if not config_values.get(field)] + + if missing_basic: + messagebox.showwarning( + "Missing Configuration", + f"The following required fields are missing:\n\n" + f"• {', '.join(missing_basic)}\n\n" + f"These are required for basic functionality." + ) + return + + # Check AI provider setup before saving + ai_provider = config_values.get('AI_PROVIDER', '').strip().lower() + if ai_provider and ai_provider not in ['none', '']: + if ai_provider in ['chatgpt', 'claude', 'anthropic', 'github-copilot', 'copilot', 'github_copilot']: + try: + # Import here to avoid circular imports + from .ai_manager import AIManager + ai_manager = AIManager() + + available, missing = ai_manager.check_ai_module_availability(ai_provider) + if not available: + # Offer to install missing packages + install_success = ai_manager.install_ai_packages(missing, self.dialog) + if not install_success: + # Installation failed or was cancelled, but still save settings + messagebox.showwarning("AI Modules Not Installed", + f"Settings saved, but AI provider '{ai_provider}' " + f"requires additional packages: {', '.join(missing)}\n\n" + f"You can install them later with:\n" + f"pip install {' '.join(missing)}", + parent=self.dialog) + except ImportError: + # AIManager not available, skip AI validation + pass + + # Save configuration using the provided config manager + if self.config_manager: + success = self.config_manager.save_configuration(config_values) + else: + # Fallback: create new config manager or save directly to file + try: + from .config_manager import ConfigManager + config_manager = ConfigManager() + success = config_manager.save_configuration(config_values) + except ImportError: + # Fallback to basic file saving if ConfigManager not available + success = self._save_to_env_file(config_values) + + if success: + self.result = config_values + + # Ask user if they want to restart the application + restart = messagebox.askyesno( + "Settings Saved", + "Settings have been saved to .env file!\n\n" + "Would you like to restart the application now to apply changes?", + parent=self.dialog + ) + + self.dialog.destroy() + + if restart: + self._restart_application() + else: + messagebox.showerror("Save Error", + "Failed to save settings to .env file.", + parent=self.dialog) + + except Exception as e: + messagebox.showerror("Save Error", + f"Error saving settings:\n{str(e)}", + parent=self.dialog) + + def _save_to_env_file(self, config_values: Dict[str, Any]) -> bool: + """Fallback method to save configuration to .env file""" + try: + import os + + # Create .env content + env_content = "# Azure DevOps to GitHub Tool Configuration\n" + env_content += "# Generated by Settings Dialog\n\n" + + # Add all configuration values + for key, value in config_values.items(): + if value: # Only add non-empty values + env_content += f"{key}={value}\n" + else: + env_content += f"{key}=\n" + + # Write to .env file + env_path = os.path.join(os.getcwd(), '.env') + with open(env_path, 'w', encoding='utf-8') as f: + f.write(env_content) + + return True + + except Exception as e: + print(f"Error saving to .env file: {e}") + return False + + def _on_repo_selected(self, event=None): + """Handle repo selection from dropdown - informational only for fork workflow""" + # The detected repo dropdown shows which FORK the AI will work on locally + # The GITHUB_REPO field is the UPSTREAM repo where PRs are created + # This supports the fork workflow: work on fork, PR to upstream + pass + + def _scan_repos(self): + """Scan for git repositories in the local repo path""" + try: + from pathlib import Path + + # Get the local repo path from the entry field + local_path = self.entries.get('LOCAL_REPO_PATH') + if local_path and hasattr(local_path, 'get'): + path_str = local_path.get().strip() + else: + path_str = self.config.get('LOCAL_REPO_PATH', '').strip() + + # If no path configured, use default + if not path_str: + path_str = str(Path.home() / "Downloads" / "github_repos") + + base_path = Path(path_str) + + # Check if path exists + if not base_path.exists(): + self.detected_repos_var.set('No repos found (directory does not exist)') + self.detected_repos_dropdown['values'] = [] + return + + # Scan for git repositories + repos = [] + try: + # Look for owner/repo structure: base_path/owner/repo/.git + for owner_dir in base_path.iterdir(): + if not owner_dir.is_dir(): + continue + + for repo_dir in owner_dir.iterdir(): + if not repo_dir.is_dir(): + continue + + # Check if it's a git repo + git_dir = repo_dir / ".git" + if git_dir.exists(): + repo_name = f"{owner_dir.name}/{repo_dir.name}" + repos.append(repo_name) + + except PermissionError: + self.detected_repos_var.set('Permission denied accessing directory') + self.detected_repos_dropdown['values'] = [] + return + except Exception as e: + self.detected_repos_var.set(f'Error scanning: {str(e)[:50]}') + self.detected_repos_dropdown['values'] = [] + return + + # Update dropdown + if repos: + repos.sort() + self.detected_repos_dropdown['values'] = repos + + # Auto-select if only one repo found + if len(repos) == 1: + self.detected_repos_var.set(repos[0]) + # Trigger the selection handler to offer auto-populating GITHUB_REPO + self.dialog.after(200, self._on_repo_selected) + else: + self.detected_repos_var.set(f'{len(repos)} repo(s) found - select one') + else: + self.detected_repos_var.set('No git repositories found') + self.detected_repos_dropdown['values'] = [] + + except Exception as e: + self.detected_repos_var.set(f'Error: {str(e)[:50]}') + self.detected_repos_dropdown['values'] = [] + + def _restart_application(self): + """Restart the application""" + try: + # Get the parent root window (main application) + root = self.parent + while root.master: + root = root.master + + # Close the main window + root.quit() + + # Restart the application using the same Python executable and script + python = sys.executable + script = sys.argv[0] + + # If running as a module (python -m), preserve that + if script.endswith('__main__.py'): + # Running as module, restart with module syntax + os.execl(python, python, '-m', 'app') + else: + # Running as script, restart directly + os.execl(python, python, script, *sys.argv[1:]) + + except Exception as e: + messagebox.showerror( + "Restart Failed", + f"Could not restart application automatically:\n{str(e)}\n\n" + "Please restart the application manually.", + parent=self.parent + ) + + def _cancel_clicked(self): + """Handle cancel button click""" + self.result = None + self.dialog.destroy() + + def _clear_cache(self): + """Clear all cached work items""" + result = messagebox.askyesno( + "Clear Cache", + "Are you sure you want to clear all cached items?\n\n" + "Cached work items and UUF items will be removed.\n" + "The next time you open the app, it will auto-load fresh data." + ) + if result: + try: + # Use cache manager passed to dialog + if self.cache_manager: + self.cache_manager.invalidate_cache() + messagebox.showinfo( + "Cache Cleared", + "All cached items have been cleared.\n" + "Fresh data will be loaded on next app start." + ) + else: + messagebox.showerror("Error", "Cache manager not available") + except Exception as e: + messagebox.showerror("Error", f"Failed to clear cache: {str(e)}") + + def show(self) -> Optional[Dict[str, Any]]: + """Show dialog and return result""" + self.dialog.wait_window() + return self.result \ No newline at end of file diff --git a/application/app_components/utils.py b/application/app_components/utils.py new file mode 100644 index 0000000..c00c79f --- /dev/null +++ b/application/app_components/utils.py @@ -0,0 +1,742 @@ +""" +Utility functions and helpers +""" + +import json +import os +import re +import subprocess +import threading +import datetime +from pathlib import Path +from typing import Dict, Any, Optional, Tuple, List +from urllib.parse import urlparse + + +class Logger: + """Simple logger for GUI applications""" + + def __init__(self, text_widget=None): + self.text_widget = text_widget + self._lock = threading.Lock() + + def log(self, message: str) -> None: + """Log a message to the text widget and console""" + timestamp = __import__('datetime').datetime.now().strftime("%H:%M:%S") + formatted_message = f"[{timestamp}] {message}" + + try: + print(formatted_message) + except UnicodeEncodeError: + # Fallback: replace Unicode emojis with ASCII equivalents + safe_message = formatted_message.replace('✅', '[SUCCESS]').replace('❌', '[ERROR]').replace('⚠️', '[WARNING]').replace('📋', '[INFO]').replace('📄', '[FILE]').replace('📍', '[LOCATION]').replace('📝', '[EDIT]') + print(safe_message) + + if self.text_widget: + def update_widget(): + try: + with self._lock: + self.text_widget.config(state='normal') + self.text_widget.insert('end', formatted_message + '\n') + self.text_widget.see('end') + self.text_widget.config(state='disabled') + self.text_widget.update_idletasks() + except: + pass # Widget might be destroyed + + # Schedule update on main thread + if hasattr(self.text_widget, 'after'): + self.text_widget.after(0, update_widget) + else: + update_widget() + + +class PRNumberManager: + """Manages PR numbers for branch naming""" + + PR_COUNTER_FILE = '.pr_counter.json' + + @classmethod + def get_pr_counter_file(cls) -> str: + """Get the path to the PR counter file""" + script_dir = os.path.dirname(os.path.abspath(__file__)) + return os.path.join(script_dir, cls.PR_COUNTER_FILE) + + @classmethod + def load_pr_counter(cls) -> Dict[str, int]: + """Load the PR counter from file""" + counter_file = cls.get_pr_counter_file() + if os.path.exists(counter_file): + try: + with open(counter_file, 'r', encoding='utf-8') as f: + return json.load(f) + except (json.JSONDecodeError, FileNotFoundError): + pass + return {} + + @classmethod + def save_pr_counter(cls, counter: Dict[str, int]) -> None: + """Save the PR counter to file""" + counter_file = cls.get_pr_counter_file() + try: + with open(counter_file, 'w', encoding='utf-8') as f: + json.dump(counter, f, indent=2) + except Exception as e: + print(f"Warning: Could not save PR counter: {e}") + + @classmethod + def get_next_pr_number(cls, provider_key: str) -> int: + """ + Get the next PR number for a given provider + + Args: + provider_key: Either the AI provider name ('chatgpt', 'claude') or 'gh_copilot' + + Returns: + Next available PR number for this provider + """ + try: + counter = cls.load_pr_counter() + current_number = counter.get(provider_key, 0) + next_number = current_number + 1 + counter[provider_key] = next_number + cls.save_pr_counter(counter) + return next_number + + except Exception as e: + print(f"Error managing PR counter: {e}") + # Fallback to a timestamp-based number + import time + return int(time.time()) % 10000 + + +class GitHubInfoExtractor: + """Extracts GitHub repository information from URLs""" + + @staticmethod + def extract_github_info(doc_url: str) -> Dict[str, Any]: + """Extract GitHub repository information from a document URL""" + try: + if not doc_url or 'github.com' not in doc_url: + return {'error': 'Not a GitHub URL'} + + parsed = urlparse(doc_url) + path_parts = parsed.path.strip('/').split('/') + + if len(path_parts) < 2: + return {'error': 'Invalid GitHub URL format'} + + owner = path_parts[0] + repo = path_parts[1] + + # Try to extract file path if it's a blob URL + file_path = None + if len(path_parts) > 3 and path_parts[2] == 'blob': + # Skip branch name and get file path + if len(path_parts) > 4: + file_path = '/'.join(path_parts[4:]) + + result = { + 'owner': owner, + 'repo': repo, + 'original_content_git_url': doc_url + } + + if file_path: + result['file_path'] = file_path + + # Try to find ms.author from the URL or repo name + ms_author = GitHubInfoExtractor._extract_ms_author(owner, repo, doc_url) + if ms_author: + result['ms_author'] = ms_author + + return result + + except Exception as e: + return {'error': f'Error parsing GitHub URL: {str(e)}'} + + @staticmethod + def _extract_ms_author(owner: str, repo: str, url: str) -> Optional[str]: + """Try to extract ms.author from various sources""" + try: + # Method 1: Check if owner looks like a Microsoft username + if owner.startswith('Microsoft') or 'microsoft' in owner.lower(): + # Try to extract from repo name or URL patterns + if '-' in repo: + parts = repo.split('-') + for part in parts: + if len(part) > 2 and part.islower(): + return part + + # Method 2: Look for patterns in the URL + url_lower = url.lower() + + # Common patterns for ms.author + patterns = [ + r'/([a-z][a-z0-9-]+[a-z0-9])/', # username-like patterns + r'author[=:]([a-z][a-z0-9-]+)', # author= or author: patterns + ] + + for pattern in patterns: + match = re.search(pattern, url_lower) + if match: + candidate = match.group(1) + # Validate it looks like a reasonable username + if 3 <= len(candidate) <= 20 and candidate.replace('-', '').isalnum(): + return candidate + + return None + + except Exception: + return None + + +class WorkItemFieldExtractor: + """Extracts and processes work item fields""" + + @staticmethod + def extract_work_item_fields(work_item: Dict[str, Any]) -> Dict[str, Any]: + """Extract and process fields from Azure DevOps work item""" + fields = work_item.get('fields', {}) + + # Extract basic fields + item_id = work_item.get('id', 'Unknown') + title = fields.get('System.Title', 'No Title') + + # Extract custom fields with fallbacks + nature_of_request = ( + fields.get('Custom.Natureofrequest') or + fields.get('Custom.NatureOfRequest') or + fields.get('Microsoft.VSTS.Common.DescriptionHtml', '') + ) + + # Clean HTML if present + if nature_of_request and '<' in nature_of_request: + nature_of_request = WorkItemFieldExtractor._clean_html(nature_of_request) + + mydoc_url = ( + fields.get('Custom.MyDocURL') or + fields.get('Custom.DocumentURL') or + fields.get('Custom.URL', '') + ) + + text_to_change = ( + fields.get('Custom.TextToChange') or + fields.get('Custom.CurrentText', '') + ) + + new_text = ( + fields.get('Custom.NewText') or + fields.get('Custom.ProposedText') or + fields.get('Custom.ReplacementText', '') + ) + + # Extract GitHub info from the document URL + github_info = GitHubInfoExtractor.extract_github_info(mydoc_url) + + return { + 'id': item_id, + 'title': title, + 'nature_of_request': nature_of_request, + 'mydoc_url': mydoc_url, + 'text_to_change': text_to_change, + 'new_text': new_text, + 'github_info': github_info, + 'status': 'Ready', + 'source': 'Azure DevOps' + } + + @staticmethod + def extract_uuf_item_fields(uuf_item: Dict[str, Any]) -> Dict[str, Any]: + """Extract and process fields from UUF item""" + # UUF items have different field structure + item_id = uuf_item.get('cr_uufitemid', 'Unknown') + title = uuf_item.get('cr_title', 'No Title') + + nature_of_request = uuf_item.get('cr_description', '') + mydoc_url = uuf_item.get('cr_documenturl', '') + text_to_change = uuf_item.get('cr_currenttext', '') + new_text = uuf_item.get('cr_newtext', '') + + # Extract GitHub info + github_info = GitHubInfoExtractor.extract_github_info(mydoc_url) + + return { + 'id': item_id, + 'title': title, + 'nature_of_request': nature_of_request, + 'mydoc_url': mydoc_url, + 'text_to_change': text_to_change, + 'new_text': new_text, + 'github_info': github_info, + 'status': 'Ready', + 'source': 'UUF' + } + + @staticmethod + def _clean_html(html_text: str) -> str: + """Remove HTML tags and decode entities""" + import html + + # Remove HTML tags + clean_text = re.sub(r'<[^>]+>', '', html_text) + + # Decode HTML entities + clean_text = html.unescape(clean_text) + + # Clean up whitespace + clean_text = re.sub(r'\s+', ' ', clean_text).strip() + + return clean_text + + +class ContentBuilders: + """Builds content for GitHub issues and PRs""" + + @staticmethod + def build_issue_title(item: Dict[str, Any]) -> str: + """Build GitHub issue title""" + source_prefix = "UUF" if item.get('source') == 'UUF' else "AB" + return f"[{source_prefix}#{item['id']}] {item['title']}" + + @staticmethod + def build_issue_body(item: Dict[str, Any], github_info: Dict[str, Any]) -> str: + """Build GitHub issue body""" + body_parts = [] + + # Header + source_name = "UUF Item" if item.get('source') == 'UUF' else "Azure DevOps Work Item" + body_parts.append(f"## {source_name} Details") + body_parts.append("") + + # Make ID a hyperlink if source URL is available + if item.get('source_url'): + body_parts.append(f"**ID:** [{item['id']}]({item['source_url']})") + else: + body_parts.append(f"**ID:** {item['id']}") + + body_parts.append(f"**Title:** {item['title']}") + body_parts.append("") + + # Nature of request + if item['nature_of_request']: + body_parts.append("**Nature of Request:**") + body_parts.append(item['nature_of_request']) + body_parts.append("") + + # Document information + if item['mydoc_url']: + body_parts.append("**Document URL:**") + body_parts.append(item['mydoc_url']) + body_parts.append("") + + # Change details + body_parts.append("## Change Details") + body_parts.append("") + + if item['text_to_change']: + body_parts.append("**Text to Change:**") + body_parts.append("```") + body_parts.append(item['text_to_change']) + body_parts.append("```") + body_parts.append("") + + if item['new_text']: + body_parts.append("**Proposed New Text:**") + body_parts.append("```") + body_parts.append(item['new_text']) + body_parts.append("```") + body_parts.append("") + + # Repository info + if github_info.get('owner') and github_info.get('repo'): + body_parts.append("## Repository Information") + body_parts.append("") + body_parts.append(f"**Repository:** {github_info['owner']}/{github_info['repo']}") + + if github_info.get('ms_author'): + body_parts.append(f"**Author:** @{github_info['ms_author']}") + + body_parts.append("") + + # Instructions for manual review + body_parts.append("## Instructions") + body_parts.append("") + body_parts.append("This issue requires manual review of the proposed documentation change.") + body_parts.append("") + body_parts.append("**Next Steps:**") + body_parts.append("1. Review the proposed change above") + body_parts.append("2. Navigate to the document URL") + body_parts.append("3. Locate the text that needs to be changed") + body_parts.append("4. Make the appropriate updates") + body_parts.append("5. Close this issue when complete") + body_parts.append("") + body_parts.append("---") + body_parts.append("*Created automatically by Azure DevOps → GitHub Processor*") + + return "\n".join(body_parts) + + @staticmethod + def build_pr_title(item: Dict[str, Any]) -> str: + """Build GitHub PR title""" + source_prefix = "UUF" if item.get('source') == 'UUF' else "AB" + return f"[{source_prefix}#{item['id']}] {item['title']}" + + @staticmethod + def build_pr_body(item: Dict[str, Any], github_info: Dict[str, Any]) -> str: + """Build GitHub PR body""" + body_parts = [] + + # Header + source_name = "UUF Item" if item.get('source') == 'UUF' else "Azure DevOps Work Item" + body_parts.append(f"## {source_name} Documentation Update") + body_parts.append("") + + # Make ID a hyperlink if source URL is available + if item.get('source_url'): + body_parts.append(f"**ID:** [{item['id']}]({item['source_url']})") + else: + body_parts.append(f"**ID:** {item['id']}") + + body_parts.append(f"**Title:** {item['title']}") + body_parts.append("") + + # Nature of request + if item['nature_of_request']: + body_parts.append("**Description:**") + body_parts.append(item['nature_of_request']) + body_parts.append("") + + # Change summary + body_parts.append("## Changes Made") + body_parts.append("") + body_parts.append("This PR updates documentation as requested.") + body_parts.append("") + + if item['text_to_change'] and item['new_text']: + body_parts.append("**Change Summary:**") + body_parts.append("- Updated specific text content as requested") + body_parts.append("") + + body_parts.append("
") + body_parts.append("View Change Details") + body_parts.append("") + body_parts.append("**Original Text:**") + body_parts.append("```") + body_parts.append(item['text_to_change']) + body_parts.append("```") + body_parts.append("") + body_parts.append("**New Text:**") + body_parts.append("```") + body_parts.append(item['new_text']) + body_parts.append("```") + body_parts.append("
") + body_parts.append("") + + # Repository info + if github_info.get('ms_author'): + body_parts.append(f"**Author:** @{github_info['ms_author']}") + body_parts.append("") + + # Review instructions + body_parts.append("## Review Checklist") + body_parts.append("") + body_parts.append("- [ ] Changes match the requested update") + body_parts.append("- [ ] No unintended changes were made") + body_parts.append("- [ ] Grammar and formatting are correct") + body_parts.append("- [ ] Links and references are working") + body_parts.append("") + + body_parts.append("---") + body_parts.append("*Created automatically by Azure DevOps → GitHub Processor*") + + return "\n".join(body_parts) + + +class LocalRepositoryScanner: + """Scans local repository path for Git repositories""" + + @staticmethod + def scan_local_repos(local_repo_path: str) -> List[str]: + """Scan local path for Git repositories""" + if not local_repo_path or not os.path.exists(local_repo_path): + return [] + + repos = [] + try: + for item in os.listdir(local_repo_path): + item_path = os.path.join(local_repo_path, item) + if os.path.isdir(item_path): + git_path = os.path.join(item_path, '.git') + if os.path.exists(git_path): + # Get remote origin URL to determine repo name + repo_info = LocalRepositoryScanner.get_repo_info(item_path) + if repo_info: + repos.append(repo_info) + else: + # Fallback to folder name + repos.append(f"local/{item}") + except PermissionError: + pass # Skip directories we can't access + except Exception as e: + print(f"Error scanning local repos: {e}") + + return sorted(repos) + + @staticmethod + def get_repo_info(repo_path: str) -> Optional[str]: + """Get repository information from local Git repo""" + try: + # Get remote origin URL + result = subprocess.run( + ['git', 'config', '--get', 'remote.origin.url'], + cwd=repo_path, + capture_output=True, + text=True, + timeout=10 + ) + + if result.returncode == 0: + url = result.stdout.strip() + return LocalRepositoryScanner.parse_git_url(url) + except Exception: + pass + + return None + + @staticmethod + def parse_git_url(url: str) -> Optional[str]: + """Parse Git URL to extract owner/repo format""" + try: + # Handle GitHub URLs + if 'github.com' in url: + # Handle both HTTPS and SSH URLs + if url.startswith('git@'): + # SSH: git@github.com:owner/repo.git + parts = url.split(':')[-1].replace('.git', '') + return parts + else: + # HTTPS: https://github.com/owner/repo.git + parsed = urlparse(url) + path = parsed.path.strip('/').replace('.git', '') + return path + except: + pass + + return None + + @staticmethod + def clone_repository(repo_url: str, local_path: str, repo_name: str) -> bool: + """Clone a repository to local path""" + try: + target_path = os.path.join(local_path, repo_name.split('/')[-1]) + + if os.path.exists(target_path): + print(f"Repository already exists at {target_path}") + return True + + os.makedirs(local_path, exist_ok=True) + + result = subprocess.run( + ['git', 'clone', repo_url, target_path], + capture_output=True, + text=True, + timeout=300 # 5 minutes timeout + ) + + if result.returncode == 0: + print(f"Successfully cloned {repo_url} to {target_path}") + return True + else: + print(f"Failed to clone repository: {result.stderr}") + return False + + except Exception as e: + print(f"Error cloning repository: {e}") + return False + + +class ConfigurationHelpers: + """Configuration and validation utilities""" + + @staticmethod + def validate_ai_provider_setup(config: Dict[str, Any], parent_window=None) -> bool: + """Validate AI provider setup and offer to install missing modules + + Args: + config: Configuration dictionary + parent_window: Parent tkinter window for dialogs + + Returns: + bool: True if setup is valid or user handled the issue + """ + ai_provider = config.get('AI_PROVIDER', '').lower() + + if not ai_provider or ai_provider == 'none': + return True # No AI provider selected, nothing to validate + + try: + # Try to import AI manager for validation + from .ai_manager import AIManager + ai_manager = AIManager() + + # Check if modules are available + available, missing = ai_manager.check_ai_module_availability(ai_provider) + + if available: + return True # All modules available + + print(f"⚠️ AI Provider '{ai_provider}' selected but missing required packages: {', '.join(missing)}") + + # Offer to install missing packages + success = ai_manager.install_ai_packages(missing, parent_window) + + if success: + # Re-check availability after installation + available, still_missing = ai_manager.check_ai_module_availability(ai_provider) + if available: + print(f"✅ AI Provider '{ai_provider}' is now ready to use") + return True + else: + print(f"⚠️ Some packages may still be missing: {', '.join(still_missing)}") + print("Please restart the application after installation completes") + return False + + return False + + except ImportError: + # AI manager not available, skip validation + return True + + @staticmethod + def create_default_env_file() -> bool: + """Create a default .env file with all settings blank""" + try: + default_config = """# Azure DevOps to GitHub Tool Configuration +# Generated automatically - fill in your values +# IMPORTANT: Do NOT commit this file to source control. Add it to .gitignore. + +# Azure DevOps Configuration +AZURE_DEVOPS_QUERY= +AZURE_DEVOPS_PAT= + +# GitHub Configuration +GITHUB_PAT= +GITHUB_REPO= + +# Application Settings +DRY_RUN=false + +# AI Provider Configuration (for local PR creation with AI assistance) +AI_PROVIDER= +CLAUDE_API_KEY= +OPENAI_API_KEY= +GITHUB_TOKEN= +LOCAL_REPO_PATH= + +# PowerApp/Dataverse Configuration (for UUF items - optional) +DATAVERSE_ENVIRONMENT_URL= +DATAVERSE_TABLE_NAME= +AZURE_AD_CLIENT_ID= +AZURE_AD_CLIENT_SECRET= +AZURE_AD_TENANT_ID= +""" + with open('.env', 'w', encoding='utf-8') as f: + f.write(default_config) + + print("Created default .env file with blank values") + return True + + except Exception as e: + print(f"Error creating default .env file: {e}") + return False + + +class EnhancedContentBuilders(ContentBuilders): + """Enhanced content builders with Azure DevOps specific methods""" + + @staticmethod + def build_pr_title_for_azure_devops(item: Dict[str, Any]) -> str: + """Build GitHub PR title for Azure DevOps items""" + return f"Docs update: {item['title'][:80]} (AB#{item['id']})" + + @staticmethod + def build_pr_body_for_azure_devops(item: Dict[str, Any], github_info: Dict[str, Any]) -> str: + """Build GitHub PR body for Azure DevOps items with enhanced Copilot instructions""" + now = datetime.datetime.utcnow().strftime("%Y-%m-%d %H:%M UTC") + + lines = [ + f"**Automated documentation update from Azure DevOps (created on {now})**", + "", + f"**Work Item ID:** AB#{item['id']}", + f"**Document URL:** {item['mydoc_url']}", + ] + + # Add file path information if available + if github_info.get('original_content_git_url'): + lines.append(f"**File Path:** {github_info['original_content_git_url']}") + + # Add ms.author metadata if available + if github_info.get('ms_author'): + lines.append(f"**ms.author:** `{github_info['ms_author']}`") + + # Add nature of request for context + lines.extend([ + "", + "## Change Type", + f"{item['nature_of_request']}", + "", + ]) + + lines.extend([ + "## Changes Requested", + "", + "### Current Text to Replace", + "```", + item['text_to_change'], + "```", + "", + "### Proposed New Text", + "```", + item['new_text'], + "```", + "", + "---", + "", + "## Instructions for GitHub Copilot", + "", + "**Task:** Update the documentation file with the changes requested above.", + "", + "**Steps to complete:**", + "1. Locate the file containing the 'Current Text to Replace' shown above", + "2. Find the exact text that needs to be updated", + "3. Replace it with the 'Proposed New Text'", + "4. Ensure no other changes are made to the file", + "5. Commit the changes with a descriptive message", + "", + "**Important Notes:**", + "- Only change the specific text shown above", + "- Do not modify formatting, links, or other content", + "- Verify the replacement text fits naturally in context", + "", + "---", + "*This PR was created automatically from Azure DevOps work item AB#" + str(item['id']) + "*" + ]) + + return "\n".join(lines) + + +# Compatibility functions for direct function access +def get_next_pr_number(provider_key: str) -> int: + """Compatibility function for direct access to PR number generation""" + return PRNumberManager.get_next_pr_number(provider_key) + + +def validate_ai_provider_setup(config: Dict[str, Any], parent_window=None) -> bool: + """Compatibility function for direct access to AI provider validation""" + return ConfigurationHelpers.validate_ai_provider_setup(config, parent_window) + + +def create_default_env_file() -> bool: + """Compatibility function for direct access to .env file creation""" + return ConfigurationHelpers.create_default_env_file() \ No newline at end of file diff --git a/application/app_components/work_item_processor.py b/application/app_components/work_item_processor.py new file mode 100644 index 0000000..2327eb2 --- /dev/null +++ b/application/app_components/work_item_processor.py @@ -0,0 +1,291 @@ +""" +Work Item Processor +Handles processing of Azure DevOps work items and UUF items +""" + +import re +import html +import requests +from typing import Dict, Any, Optional, Tuple +from urllib.parse import urlparse +from .utils import WorkItemFieldExtractor + +# User agent for web requests +USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36' + + +class WorkItemProcessor: + """Processor for extracting and validating work item data with advanced parsing""" + + def __init__(self, logger, config: Dict[str, Any] = None): + self.logger = logger + self.log = logger.log if hasattr(logger, 'log') else logger + self.config = config or {} + + def process_work_item(self, work_item: Dict[str, Any]) -> Optional[Dict[str, Any]]: + """Process a single work item to extract required fields with advanced validation""" + try: + work_item_id = work_item['id'] + title = work_item.get('fields', {}).get('System.Title', 'No Title') + description = work_item.get('fields', {}).get('System.Description', '') + + if not description: + self.log(f"Work item {work_item_id} has no description, skipping") + return None + + # Parse description for required fields + parsed_data = self._parse_description(description) + + if not parsed_data: + self.log(f"Work item {work_item_id} doesn't contain required fields, skipping") + return None + + # Validate nature of request (check for both variations) + nature_lower = parsed_data['nature_of_request'].lower() + if not ("modify existing docs" in nature_lower or "modifying existing docs" in nature_lower): + self.log(f"Work item {work_item_id} nature of request doesn't contain 'modify existing docs', skipping") + return None + + # Extract GitHub info from document URL + github_info = self._extract_github_info(parsed_data['mydoc_url']) + + # If the document does not include an original_content_git_url, skip this work item + if not github_info.get('original_content_git_url'): + self.log(f"Work item {work_item_id} skipped: original_content_git_url not found in document {parsed_data['mydoc_url']}") + return None + + # Construct proper web URL for work item + # The API returns something like: https://dev.azure.com/org/project/_apis/wit/workItems/123 + # We need to convert it to: https://dev.azure.com/org/project/_workitems/edit/123 + work_item_url = '' + api_url = work_item.get('url', '') + if api_url: + # Convert API URL to web URL + # Replace /_apis/wit/workItems/ with /_workitems/edit/ + work_item_url = api_url.replace('/_apis/wit/workItems/', '/_workitems/edit/') + + processed_item = { + 'id': work_item_id, + 'title': title, + 'nature_of_request': parsed_data['nature_of_request'], + 'mydoc_url': parsed_data['mydoc_url'], + 'text_to_change': parsed_data['text_to_change'], + 'new_text': parsed_data['new_text'], + 'github_info': github_info, + 'status': 'Ready', + 'source': 'Azure DevOps', + 'source_url': work_item_url, # URL to Azure DevOps work item + 'original_new_text': parsed_data['new_text'] # Keep original for reference + } + + self.log(f"Successfully processed work item {work_item_id}") + return processed_item + + except Exception as e: + self.log(f"Error processing work item {work_item.get('id', 'unknown')}: {str(e)}") + return None + + def process_uuf_item(self, uuf_item: Dict[str, Any]) -> Optional[Dict[str, Any]]: + """Process a single UUF item from Dataverse/PowerApp with enhanced field mapping""" + try: + # Extract UUF item ID (adjust field name as needed) + uuf_id = uuf_item.get('cr4af_uufid') or uuf_item.get('cr4af_name') or 'unknown' + + # Extract title + title = uuf_item.get('cr4af_title') or uuf_item.get('cr4af_subject') or 'No Title' + + # Extract description/details + description = uuf_item.get('cr4af_description') or uuf_item.get('cr4af_details') or '' + + if not description: + self.log(f"UUF item {uuf_id} has no description, skipping") + return None + + # Extract document URL + doc_url = uuf_item.get('cr4af_documenturl') or uuf_item.get('cr4af_docurl') or '' + + if not doc_url: + self.log(f"UUF item {uuf_id} has no document URL, skipping") + return None + + # Extract text to change and new text + text_to_change = uuf_item.get('cr4af_texttochange') or uuf_item.get('cr4af_currenttext') or '' + new_text = uuf_item.get('cr4af_proposednewtext') or uuf_item.get('cr4af_newtext') or '' + + if not text_to_change or not new_text: + self.log(f"UUF item {uuf_id} missing text fields, skipping") + return None + + # Extract GitHub info from document URL + github_info = self._extract_github_info(doc_url) + + # If the document does not include an original_content_git_url, skip this item + if not github_info.get('original_content_git_url'): + self.log(f"UUF item {uuf_id} skipped: original_content_git_url not found in document {doc_url}") + return None + + # Get UUF item URL if available (e.g., from Dataverse) + uuf_url = uuf_item.get('cr4af_itemurl', '') or uuf_item.get('cr4af_url', '') + + processed_item = { + 'id': uuf_id, + 'title': title, + 'nature_of_request': 'UUF Item - Modify existing docs', + 'mydoc_url': doc_url, + 'text_to_change': text_to_change, + 'new_text': new_text, + 'github_info': github_info, + 'status': 'Ready', + 'source': 'UUF', # Mark as UUF item + 'source_url': uuf_url, # URL to UUF item (if available) + 'original_new_text': new_text + } + + self.log(f"Successfully processed UUF item {uuf_id}") + return processed_item + + except Exception as e: + self.log(f"Error processing UUF item {uuf_item.get('cr4af_uufid', 'unknown')}: {str(e)}") + return None + + def _parse_description(self, description: str) -> Optional[Dict[str, Any]]: + """Parse work item description to extract required fields using enhanced regex patterns""" + # Enhanced regex patterns from regex_V5 + patterns = { + 'nature_of_request': r'nature\s+of\s+request[:\s]*([^\)]*\))', + 'link_to_doc': r'link\s+to\s+doc[:\s]*([^\s&]+)', + 'text_to_change': r'text\s+to\s+change[:\s]*([\s\S]*?)(?=\n*-+\s*Proposed new text|If adding brand new docs:|$)', + 'proposed_new_text': r'proposed\s+new\s+text[:\s]*([\s\S]+?)(?=\s*If\s+adding\s+brand\s+new\s+docs:)' + } + + # Clean HTML tags if present + clean_description = re.sub(r'<[^>]+>', '', description) + + # Convert HTML entities to characters (e.g., " to ", & to &) + clean_description = html.unescape(clean_description) + + extracted = {} + for field, pattern in patterns.items(): + match = re.search(pattern, clean_description, re.IGNORECASE | re.DOTALL) + if match: + value = match.group(1).strip() + + if field == 'nature_of_request': + extracted['nature_of_request'] = value + elif field == 'link_to_doc': + extracted['mydoc_url'] = value.rstrip('-') + elif field == 'text_to_change': + extracted['text_to_change'] = value + elif field == 'proposed_new_text': + extracted['new_text'] = value + + # If enhanced patterns don't work, fall back to basic patterns + if not all(field in extracted for field in ['nature_of_request', 'mydoc_url', 'text_to_change', 'new_text']): + basic_patterns = { + 'nature_of_request': r'nature\s+of\s+request[:\s]*([^\n]+)', + 'link_to_doc': r'link\s+to\s+doc[:\s]*([^\s]+)', + 'text_to_change': r'text\s+to\s+change[:\s]*(.+?)(?=proposed\s+new\s+text|$)', + 'proposed_new_text': r'proposed\s+new\s+text[:\s]*(.+?)(?=\n\n|$)' + } + + extracted = {} + for field, pattern in basic_patterns.items(): + match = re.search(pattern, clean_description, re.IGNORECASE | re.DOTALL) + if match: + value = match.group(1).strip() + + if field == 'nature_of_request': + extracted['nature_of_request'] = value + elif field == 'link_to_doc': + extracted['mydoc_url'] = value + elif field == 'text_to_change': + extracted['text_to_change'] = value + elif field == 'proposed_new_text': + extracted['new_text'] = value + + # Validate all required fields are present + required_fields = ['nature_of_request', 'mydoc_url', 'text_to_change', 'new_text'] + if not all(field in extracted for field in required_fields): + return None + + return extracted + + def _extract_github_info(self, doc_url: str) -> Dict[str, Any]: + """Extract GitHub repository info and ms.author from document URL + + If GITHUB_REPO is configured in .env, it will be used instead of the repo + extracted from the document metadata. This allows you to create PRs in your + fork while preserving the file path and ms.author from the original document. + """ + try: + # Fetch the document + headers = {'User-Agent': USER_AGENT} + response = requests.get(doc_url, headers=headers, timeout=30) + response.raise_for_status() + + html_content = response.text + + # Extract ms.author + ms_author = self._extract_meta_tag(html_content, 'ms.author') + + # Extract original_content_git_url + original_content_git_url = self._extract_meta_tag(html_content, 'original_content_git_url') + + if not original_content_git_url: + # Try alternative extraction method + match = re.search(r"original_content_git_url[\"\']?\s*[:=]\s*[\"\']([^\"']+)[\"']", html_content, re.IGNORECASE) + if match: + original_content_git_url = match.group(1).strip() + + if not original_content_git_url: + raise ValueError("original_content_git_url not found in document") + + # Check if GITHUB_REPO is configured in .env + # If it is, use that instead of the repo from the document + configured_repo = self.config.get('GITHUB_REPO') + + if configured_repo and '/' in configured_repo: + # Use the configured repository (e.g., "b-tsammons/fabric-docs-pr") + parts = configured_repo.split('/', 1) + owner = parts[0].strip() + repo = parts[1].strip() + self.log(f"Using configured GITHUB_REPO: {owner}/{repo} (overriding document metadata)") + else: + # Parse GitHub owner/repo from original_content_git_url (fallback to document metadata) + owner, repo = self._parse_github_url(original_content_git_url) + self.log(f"Using repository from document metadata: {owner}/{repo}") + + return { + 'ms_author': ms_author, + 'original_content_git_url': original_content_git_url, + 'owner': owner, + 'repo': repo + } + + except Exception as e: + self.log(f"Error extracting GitHub info from {doc_url}: {str(e)}") + return { + 'ms_author': None, + 'original_content_git_url': None, + 'owner': None, + 'repo': None, + 'error': str(e) + } + + def _extract_meta_tag(self, html_content: str, name: str) -> Optional[str]: + """Extract content from meta tag""" + pattern = rf']*?\s)?(?:name|property)\s*=\s*["\'](?P{re.escape(name)})["\']\s+[^>]*?\bcontent\s*=\s*["\'](?P[^"\']+)["\'][^>]*?>' + match = re.search(pattern, html_content, re.IGNORECASE) + if match: + return match.group('content').strip() + return None + + def _parse_github_url(self, url: str) -> Tuple[str, str]: + """Parse GitHub URL to extract owner and repo""" + parsed = urlparse(url) + if "github.com" not in parsed.netloc.lower(): + raise ValueError(f"Not a GitHub URL: {url}") + parts = [p for p in parsed.path.split("/") if p] + if len(parts) < 2: + raise ValueError(f"Unable to parse owner/repo from: {url}") + return parts[0], parts[1] \ No newline at end of file diff --git a/application/requirements.txt b/application/requirements.txt new file mode 100644 index 0000000..15c0984 --- /dev/null +++ b/application/requirements.txt @@ -0,0 +1,9 @@ +# Core dependencies +requests>=2.31.0 + +# AI providers (optional - installed automatically when needed) +anthropic>=0.18.0 # Claude AI +openai>=1.12.0 # ChatGPT/GPT-4 + +# Git operations (required for AI functionality) +GitPython>=3.1.40 diff --git a/media/fetch-work-items.png b/media/fetch-work-items.png new file mode 100644 index 0000000..702a6e1 Binary files /dev/null and b/media/fetch-work-items.png differ diff --git a/media/flow-diagram.png b/media/flow-diagram.png new file mode 100644 index 0000000..81e40ba Binary files /dev/null and b/media/flow-diagram.png differ diff --git a/media/github-issue-copilot-pr.png b/media/github-issue-copilot-pr.png new file mode 100644 index 0000000..e3a37b5 Binary files /dev/null and b/media/github-issue-copilot-pr.png differ diff --git a/media/github-pull-request.png b/media/github-pull-request.png new file mode 100644 index 0000000..2420b86 Binary files /dev/null and b/media/github-pull-request.png differ diff --git a/media/issue-created.png b/media/issue-created.png new file mode 100644 index 0000000..7dde033 Binary files /dev/null and b/media/issue-created.png differ