Refactor GitHub automation tool:

- Updated WorkItemFieldExtractor to be more generic and removed Azure DevOps specific references.
- Removed the EnhancedContentBuilders class as it was specific to Azure DevOps.
- Deleted work_item_processor.py as it was no longer needed.
- Introduced workflow.py to manage GitHub workflow items (issues and pull requests) with improved structure and functionality.
- Enhanced logging and error handling across the new workflow management system.
This commit is contained in:
b-tsammmons
2025-11-11 22:46:02 -10:00
parent d6da461d10
commit 0f41a3e750
15 changed files with 2003 additions and 1446 deletions
+3
View File
@@ -205,3 +205,6 @@ cython_debug/
marimo/_static/ marimo/_static/
marimo/_lsp/ marimo/_lsp/
__marimo__/ __marimo__/
# AI Files
.claude/
+54 -138
View File
@@ -1,163 +1,79 @@
# Azure DevOps → GitHub Processor (GUI) # GitHub Automation Tool
Azure DevOps -> GitHub Processor is a small, focused GUI tool that you can use to perform the following tasks: A Python-based GUI application for GitHub automation workflows.
1. Fetches Azure DevOps work items using a query you specify and extracts documentation change information from them. ## Project Structure
1. Creates a GitHub issues using the information and assigns the issue to GitHub Copilot, which analyzes the request, creates a branch, updates the article to fix the issue, and submits a pull request (PR) on behalf of you.
The following diagram shows you the high-level steps performed by this solution: ```text
github_automation/
![High-level diagram of the Azure DevOps -> GitHub Processor solution](./media/flow-diagram.png) ├── application/ # Main application directory
│ ├── app.py # Application entry point
The PR is assigned to the author of the article based on the author metadata in the article. The author reviews off the PR and signs on it if the changes done by Copilot look good. │ ├── requirements.txt # Python dependencies
│ └── app_components/ # Application modules
The `devops_to_github.py` file in the `Devops_to_GitHub_Python/latest` folder has all the code behind this tool. │ ├── ai_manager.py # AI provider integration
│ ├── cache_manager.py # Caching functionality
│ ├── config_manager.py # Configuration management
│ ├── github_api.py # GitHub API client
│ ├── main_gui.py # Main GUI interface
│ ├── settings_dialog.py # Settings dialog
│ └── utils.py # Utility functions
├── media/ # Images and assets
├── README.md # This file
├── SETUP.md # Setup guide
└── LICENSE # License information
```
## Prerequisites ## Prerequisites
Here are the prerequisites to use the tool:
- Python 3.10 or newer (the code uses modern union types like `dict | None`) - Python 3.8 or higher
- Install packages used in the code, such as `requests` and `tkinter`, which is used for the GUI. - Git installed and configured
- GitHub Copilot must be enabled at the organization level or for the specific repository. - GitHub account with repository access
- You must have admin access to the repository to assign issues to Copilot.
### Azure DevOps prerequisites ## Quick Start
- **Azure DevOps Query URL**: A shared query URL containing documentation update work items. You can use this [sample query](https://dev.azure.com/msft-skilling/Content/_queries/query/ab92a364-1376-43d5-a6da-510aa40a65d1/) for testing purposes. 1. **Clone the repository**
```bash
git clone https://github.com/TySP-Dev/github_automation.git
cd github_automation/application
```
Work items must contain the following fields in their description: 2. **Create and activate virtual environment**
```bash
# Create virtual environment
python -m venv venv
- **Nature of Request**: Must include **Modify existing docs**. # Activate (Windows)
- **Link to doc**: Must be a URL to a Learn article. venv\Scripts\activate
- **Text to change**: Current text that needs to update. Can be blank.
- **Proposed new text**: Suggested replacement text. This text can be instructional such as "Remove last two links from the Related content section". In this example, the GitHub Copilot doesn't replace old text with new text. It just removes those two links.
- **Azure DevOps Personal Access Token (PAT)**: Token with work item read, write, and manage permissions. The write permission is used to write back the link to GitHub issue in the DevOps work item. For instructions on creating a token, see [Authenticate to DevOps with personal access tokens](https://learn.microsoft.com/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows#create-a-pat).
### GitHub prerequisites # Activate (macOS/Linux)
- **GitHub Personal Access Token**: A **classic** token with **repo**, **admin:org**, and **copilot** permissions. For instructions on creating a token, see [Create a personal access token (classic)](https://docs.github.com/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic) source venv/bin/activate
```
## Configuration and tokens management 3. **Install dependencies**
The app first looks for tokens in the **launch.json** file and then in the **.env** file. We recommend that you use the **.env** approach. ```bash
pip install -r requirements.txt
```
Create a `.env` file in the repository root and add: 4. **Run the application**
```bash
python app.py
```
``` ## Configuration
AZURE_DEVOPS_QUERY=your_azure-devops-query-url
AZURE_DEVOPS_PAT=your_azure_devops_pat_here
GITHUB_PAT=your_github_pat_here
```
**Security reminder**: Never commit secrets. Add `.env` to `.gitignore` if you plan to store tokens locally. Configuration is managed through a `.env` file or settings dialog in the application.
### launch.json See [SETUP.md](SETUP.md) for detailed setup instructions.
Create or edit `.vscode/launch.json` and add the values into the `env` object for the configuration you run. Example snippet:
```json
{
"configurations": [
{
"name": "Run devops_to_github",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/POC/latest/devops_to_github.py",
"console": "integratedTerminal",
"env": {
"AZURE_DEVOPS_QUERY": "<your-azure-devops-query-url>",
"AZURE_DEVOPS_PAT": "<your-azure-pat>",
"GITHUB_PAT": "<your-github-pat>"
}
}
]
}
```
- If you use a VS Code launch configuration, do NOT put secrets in `.vscode/launch.json`.
- Instead use an env file and reference it via `"envFile": "${workspaceFolder}/.env"` or keep only placeholders in a committed `launch.json.example`.
- For CI and shared environments, use secret stores (GitHub Actions secrets, Azure Key Vault, etc.)—do not commit tokens into the repo.
**Warning:** Never commit PATs, tokens, or other secrets. If a secret is accidentally committed, rotate it immediately.
## How to run the app
Use one of the following methods:
- From a terminal (PowerShell on Windows), run the following command: `python .\POC\latest\devops_to_github.py`
- Or run the `Run`/`Debug` configuration in VS Code if you configured `launch.json` (or) `.env`.
## Using the app (overview)
1. Enter or verify the Azure DevOps Query URL and the PATs (or rely on loaded config from `launch.json` / `.env`).
1. Toggle **Dry Run**" to simulate operations (recommended for first runs).
1. Select **Fetch Work Items** to execute the Azure query and parse work items (this invokes `start_fetch_work_items` -> `fetch_work_items`).
![UI after fetching work items from Azure DevOps.](./media/fetch-work-items.png)
1. Navigate items with **← Previous** / **Next →**. Each item shows the parsed fields (`nature_of_request`, `mydoc_url`, `text_to_change`, `new_text`), detected GitHub repo (owner/repo), and `ms.author` when available.
1. Edit the **proposed new text** inline by selecting the ✏️ Edit control, make changes, then **Save**.
1. Select **🚀 CREATE ISSUE** to create an issue in the detected GitHub repo. On success you will get a hyperlink dialog with the created issue URL.
![UI after creating a GitHub issue.](./media/issue-created.png)
Here's what the tool does when you select **Create issue**
- Uses GitHub GraphQL API to fetch the repository ID and call `createIssue`.
- Attempts to locate a suggested actor named like Copilot and assign the new issue to that actor.
- Includes an `AB#<work item id>` line in the issue body to trace back to the Azure work item.
- If available, calls Azure DevOps API to add a hyperlink back to the Azure work item referencing the new GitHub issue URL.
1. Select the link to navigate to the GitHub issue to do the following tasks. In your Azure DevOps work item, you should see a link to the GitHub issue in the **GitHub issue** section as well.
- Review the details
- Notice that the GitHub issue is assigned to a Copilot
- The Copilot has created a pull request.
- A link back to Azure DevOps work item
![GitHub issue UI.](./media/github-issue-copilot-pr.png)
1. Select the pull request to review changes, and sign off on the PR after a successful build and the changes look good to you.
![GitHub Pull Request UI.](./media/github-pull-request.png)
## Dry-run and logging
Dry-run prints GraphQL payloads and simulates the behavior without performing remote mutations. Inspect the "Processing Log" tab to see payloads and debug messages.
## Notes, limitations & troubleshooting
- The UI currently focuses on creating issues. There is a GraphQL `createPullRequest` function implemented, but the PR option is not exposed in the GUI at the moment.
- The work item description parsing is opinionated and expects fields like "Nature of Request:", "Link to Doc:", "Text to Change:", and "Proposed new text:". If parsing fails, the work item will be skipped.
- Document URL extraction relies on the document's HTML metadata (`original_content_git_url` or similar). If the metadata isn't present or the URL is not a GitHub URL, the tool cannot determine owner/repo.
- Common errors:
- Invalid query URL: ensure the query URL uses `_queries/query/<id>/` or includes `queryId=`
- Token permission errors: check PAT scopes
- Network timeouts: check VPN/firewall rules and increase timeouts in code if needed
## Developer notes
- File of interest: `POC/latest/devops_to_github.py` (main GUI + logic)
- Logging is visible in the GUI under "Processing Log". The app is single-process with background threads for network calls.
- Python typing uses modern syntax (requires Python 3.10+).
## Process flow (diagram)
The following diagram illustrates the end-to-end processing flow implemented by `POC/latest/devops_to_github.py`. It shows how the application fetches Azure DevOps work items, validates and parses each item, discovers the target GitHub repository from the document metadata, and then creates and (optionally) assigns and links the resulting GitHub issue back to the Azure DevOps work item.
![Hack2025 processing flow](Devops_to_GitHub_Python/latest/Hack2025_flow.png)
High-level steps shown in the diagram:
1. User launches the GUI and provides the Azure DevOps query URL and tokens.
1. The app validates the query URL and executes the query to fetch work item IDs.
1. For each work item the app fetches details and parses the description for the required fields (Nature of Request, Link to Doc, Text to Change, Proposed new text).
1. If the item contains a valid document URL, the tool requests the document HTML and extracts `original_content_git_url` and metadata such as `ms.author`.
1. The tool maps the document metadata to a GitHub `owner/repo`, builds an issue title and body, and uses the GitHub GraphQL API to create the issue (dry-run mode simulates this).
1. If available, Copilot (or another suggested actor) is assigned to the created issue; the tool will also attempt to add a hyperlink back to the Azure work item.
1. The GUI updates the item's processing status and shows a link to the created issue on success.
## Contributing ## Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a This project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visit [Contributor License Agreements](https://cla.opensource.microsoft.com). the rights to use your contribution.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide ## License
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). See [LICENSE](LICENSE) file for details.
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
## Trademarks ## Trademarks
+20 -168
View File
@@ -1,8 +1,8 @@
# MicrosoftDocFlow Tool - Setup Guide # GitHub Pulse - Setup Guide
## Overview ## Overview
This tool automates the process of converting Azure DevOps work items into GitHub pull requests with AI assistance. It fetches work items, processes documentation changes, and creates pull requests with proper diffs. A Python-based GUI application for GitHub automation workflows.
## Quick Start ## Quick Start
@@ -10,9 +10,7 @@ This tool automates the process of converting Azure DevOps work items into GitHu
- **Python 3.8+** installed on your system - **Python 3.8+** installed on your system
- **Git** installed and configured - **Git** installed and configured
- **Azure DevOps** account with work items
- **GitHub** account with repository access - **GitHub** account with repository access
- **AI Provider** account (optional, for enhanced processing)
### Installation ### Installation
@@ -20,7 +18,7 @@ This tool automates the process of converting Azure DevOps work items into GitHu
```bash ```bash
git clone https://github.com/TySP-Dev/github_automation.git git clone https://github.com/TySP-Dev/github_automation.git
cd github_automation\application cd github_automation/application
``` ```
2. **Create Virtual Environment** (Recommended) 2. **Create Virtual Environment** (Recommended)
@@ -37,11 +35,11 @@ This tool automates the process of converting Azure DevOps work items into GitHu
source venv/bin/activate source venv/bin/activate
``` ```
To deactivate the environment when done: To deactivate the environment when done:
```bash ```bash
deactivate deactivate
``` ```
3. **Install Dependencies** 3. **Install Dependencies**
@@ -81,20 +79,17 @@ The project is organized as follows:
```text ```text
github_automation/ github_automation/
├── application/ # Main application ├── application/ # Main application directory
│ ├── app.py # Application entry point │ ├── app.py # Application entry point
│ ├── requirements.txt # Python dependencies │ ├── requirements.txt # Python dependencies
│ └── app_components/ # Application modules │ └── app_components/ # Application modules
│ ├── ai_manager.py # AI provider │ ├── ai_manager.py # AI provider integration
│ ├── azure_devops_api.py # Azure DevOps API │ ├── cache_manager.py # Caching functionality
│ ├── cache_manager.py # Work item caching │ ├── config_manager.py # Configuration management
│ ├── config_manager.py # Configuration
│ ├── dataverse_api.py # Dataverse API
│ ├── github_api.py # GitHub API client │ ├── github_api.py # GitHub API client
│ ├── main_gui.py # Main GUI interface │ ├── main_gui.py # Main GUI interface
│ ├── settings_dialog.py # Settings dialog │ ├── settings_dialog.py # Settings dialog
── utils.py # Utility functions ── utils.py # Utility functions
│ └── work_item_processor.py # Work item
├── media/ # Images and assets ├── media/ # Images and assets
├── README.md # Project overview ├── README.md # Project overview
├── SETUP.md # This setup guide ├── SETUP.md # This setup guide
@@ -106,16 +101,7 @@ github_automation/
### First-Time Setup ### First-Time Setup
1. **Launch the application** and click "Settings" button 1. **Launch the application** and click "Settings" button
2. **Configure required fields** in the Settings dialog: 2. **Configure required fields** in the Settings dialog
#### Azure DevOps Configuration (Required)
- **Query URL**: Your Azure DevOps query URL
- Example: `https://dev.azure.com/yourorg/project/_queries/query/12345678-1234-1234-1234-123456789abc/`
- Get this from Azure DevOps by creating/opening a query and copying the URL
- **Personal Access Token**: Azure DevOps PAT with work item read permissions
- Create at: `https://dev.azure.com/yourorg/_usersSettings/tokens`
- Required scopes: Work Items (Read/Write)
#### GitHub Configuration (Required) #### GitHub Configuration (Required)
@@ -123,158 +109,26 @@ github_automation/
- Create at: `https://github.com/settings/tokens` - Create at: `https://github.com/settings/tokens`
- Required scopes: repo, workflow - Required scopes: repo, workflow
- **Target Repository**: Format as `owner/repository` - **Target Repository**: Format as `owner/repository`
- Example: `microsoft/fabric-docs` - Example: `microsoft/example-repo`
- **Forked Repository**: Your fork of the target repository - **Forked Repository**: Your fork of the target repository (if applicable)
- Example: `yourusername/fabric-docs` - Example: `yourusername/example-repo`
- **Local Repo Path**: Directory where repositories will be/are cloned - **Local Repo Path**: Directory where repositories will be/are cloned
- Example: `C:\Users\yourname\repos\` - Example: `C:\Users\yourname\repos\`
#### AI Provider Configuration (Optional) #### AI Provider Configuration (Optional)
- **Provider**: Choose from: - **Provider**: Choose from:
- `none` - GitHub Copilot via a automatic comment on the PR - `none` - No AI assistance
- `claude` - Anthropic Claude API - `claude` - Anthropic Claude API
- `chatgpt` - OpenAI ChatGPT API - `chatgpt` - OpenAI ChatGPT API
- `github-copilot` - GitHub Models API - `github-copilot` - GitHub Models API
- **API Keys**: Provide keys based on your chosen provider - **API Keys**: Provide keys based on your chosen provider
- GitHub Token: Auto-defaults to GitHub PAT if left empty
### Configuration Tips ### Configuration Tips
**Start Simple**: Configure only Azure DevOps and GitHub initially **Start Simple**: Configure only GitHub initially
**Test Connection**: Use "Test Connection" button to verify settings **Test Connection**: Use "Test Connection" button to verify settings
**Cache Benefits**: Work items are cached for faster subsequent loads **AI Enhancement**: Add AI provider later for automated processing
**AI Enhancement**: Add AI provider later for automated text processing
## Usage Guide
### Basic Workflow
1. **Load Work Items**
- Click "Fetch Work Items" to load work items from your query
- Items are displayed in the "All Work Items" tab
- Cached items load automatically on subsequent launches
2. **Select Work Item**
- Navigate to "All Work Items" tab
- **Double-click** any work item to select it as current
- OR click "Set as Current Item" button
- Selected item appears in "Current Work Item" tab
3. **Review Work Item Details**
- **Work Item ID**: Click to open in Azure DevOps
- **Nature of Request**: Description of required changes
- **Document URL**: Target documentation URL
- **Text to Change**: Current text that needs modification
- **Proposed New Text**: Replacement text (editable)
4. **Process Changes**
- **With AI**: If AI provider is configured and the AI provider is not `none`, click "Create PR" and the selected AI will be used
- **Manual**: Click "Create PR" or "Create Issue" for manual processing / GitHub Copilot comment
- **Dry Run**: Enable for testing without actual changes (Located in settings)
### Advanced Features
#### Work Item Navigation
- **Next/Previous**: Navigate through multiple work items
- **Edit Mode**: Click "Edit" to modify proposed text
#### Git Diff Viewer
- View file changes in the "Git Diff" tab
- Shows before/after comparison
- Automatic diff generation from repository changes
#### Processing Log
- Real-time activity logging in "Processing Log" tab
- Track API calls, file operations, and errors
- Detailed workflow visibility
## Advanced Configuration
### AI Provider Setup
#### Claude (Anthropic)
1. Get API key from [console.anthropic.com](https://console.anthropic.com)
2. Set provider to `claude`
3. Enter API key in settings
4. Cost: ~$0.01-0.05 per work item
#### ChatGPT (OpenAI)
1. Get API key from [platform.openai.com/api-keys](https://platform.openai.com/api-keys)
1. Set provider to `chatgpt`
1. Enter API key in settings
1. Cost: ~$0.01-0.05 per work item
#### GitHub Copilot
1. Ensure GitHub account has Copilot access
2. Set provider to `github-copilot`
3. GitHub Token auto-defaults to GitHub PAT
4. Uses GitHub Models API
### Repository Management
#### Local Repository Setup
- **Automatic Cloning**: Repositories are cloned automatically to Local Repo Path if AI provider is used
- **Branch Management**: Creates feature branches for each work item
- **Sync Handling**: Keeps repositories updated with upstream changes
#### Forked Repository Workflow
1. **Fork** the target repository to your GitHub account
2. **Configure** both target and forked repositories in settings
3. **Automatic**: Tool manages branch creation and PR submission
### Custom Instructions
Add custom AI instructions in settings to guide processing:
```AI Prompt
Focus on technical accuracy and clear documentation.
Ensure all code examples are properly formatted.
Include relevant cross-references to related topics.
```
## 🛠️ Troubleshooting
### Common Issues
**"No work items found"**
- Verify Azure DevOps query URL is correct
- Check PAT has work item read permissions
- Ensure query returns results in Azure DevOps
**"Repository not found"**
- Verify GitHub repository name format (`owner/repo`)
- Check GitHub PAT has repository access
- Ensure repository exists and is accessible
**"AI processing failed"**
- Verify API key is correct and has credits
- Check internet connection
- Review processing log for specific errors
**"Git operations failed"**
- Ensure Git is installed and configured
- Check Local Repo Path exists and is writable
- Verify GitHub authentication is working
### Performance Tips
**Use Cache**: Let items load from cache for faster startup
**Dry Run**: Test changes before committing
**Batch Processing**: Process multiple items in sequence
**Monitor Logs**: Watch processing log for issues
## Security Best Practices ## Security Best Practices
@@ -288,7 +142,7 @@ Include relevant cross-references to related topics.
### Repository Access ### Repository Access
- **Fork workflow**: Use personal forks for changes - **Fork workflow**: Use personal forks for changes
- **Branch isolation**: Each work item gets separate branch - **Branch isolation**: Each task gets separate branch
- **Review process**: All changes go through pull requests - **Review process**: All changes go through pull requests
## Support ## Support
@@ -298,10 +152,8 @@ Include relevant cross-references to related topics.
1. **Check logs** in Processing Log tab for detailed errors 1. **Check logs** in Processing Log tab for detailed errors
2. **Test connections** using Settings dialog test button 2. **Test connections** using Settings dialog test button
3. **Review configuration** for missing or incorrect values 3. **Review configuration** for missing or incorrect values
4. **Consult documentation** for API-specific issues
### Common Resources ### Common Resources
- [Azure DevOps PAT Documentation](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate)
- [GitHub PAT Documentation](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) - [GitHub PAT Documentation](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
- [Git Configuration Guide](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup) - [Git Configuration Guide](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup)
+5 -7
View File
@@ -1,9 +1,8 @@
""" """
MicrosoftDocFlow v3 GitHub Automation Tool
Main application entry point Main application entry point
This application processes Azure DevOps work items and UUF items, This application provides GitHub automation workflows with AI assistance.
creating GitHub issues or pull requests with AI assistance.
""" """
import sys import sys
@@ -15,7 +14,6 @@ try:
from app_components.config_manager import ConfigManager from app_components.config_manager import ConfigManager
from app_components.ai_manager import AIManager from app_components.ai_manager import AIManager
from app_components.github_api import GitHubAPI from app_components.github_api import GitHubAPI
from app_components.utils import Logger, PRNumberManager, ContentBuilders
from app_components.main_gui import MainGUI from app_components.main_gui import MainGUI
except ImportError as e: except ImportError as e:
print(f"Error importing application components: {e}") print(f"Error importing application components: {e}")
@@ -23,13 +21,13 @@ except ImportError as e:
sys.exit(1) sys.exit(1)
class AzureDevOpsToGitHubApp: class GitHubAutomationApp:
"""Main application class that orchestrates all components""" """Main application class that orchestrates all components"""
def __init__(self): def __init__(self):
"""Initialize the application""" """Initialize the application"""
self.root = tk.Tk() self.root = tk.Tk()
self.root.title("MicrosoftDocFlow v3") self.root.title("GitHub Automation Tool")
self.root.geometry("1400x1000") self.root.geometry("1400x1000")
# Initialize core managers # Initialize core managers
@@ -114,7 +112,7 @@ class AzureDevOpsToGitHubApp:
def main(): def main():
"""Main entry point""" """Main entry point"""
try: try:
app = AzureDevOpsToGitHubApp() app = GitHubAutomationApp()
app.run() app.run()
except Exception as e: except Exception as e:
print(f"Failed to start application: {e}") print(f"Failed to start application: {e}")
+8 -10
View File
@@ -1,33 +1,31 @@
""" """
Azure DevOps & UUF → GitHub Processor - Application Components GitHub Pulse - Application Components
Modular components for the application Modular components for the application
""" """
# Version info # Version info
__version__ = "3.0.0" __version__ = "1.0.0"
__author__ = "Azure DevOps to GitHub Processor" __author__ = "GitHub Pulse"
# Export main classes for easier imports # Export main classes for easier imports
from .config_manager import ConfigManager from .config_manager import ConfigManager
from .ai_manager import AIManager from .ai_manager import AIManager
from .github_api import GitHubAPI from .github_api import GitHubAPI
from .azure_devops_api import AzureDevOpsAPI
from .dataverse_api import DataverseAPI
from .work_item_processor import WorkItemProcessor
from .settings_dialog import SettingsDialog from .settings_dialog import SettingsDialog
from .main_gui import MainGUI from .main_gui import MainGUI
from .utils import Logger, PRNumberManager, ContentBuilders from .utils import Logger, PRNumberManager, ContentBuilders
from .workflow import WorkflowManager, WorkflowItem, GitHubRepoFetcher
__all__ = [ __all__ = [
'ConfigManager', 'ConfigManager',
'AIManager', 'AIManager',
'GitHubAPI', 'GitHubAPI',
'AzureDevOpsAPI',
'DataverseAPI',
'WorkItemProcessor',
'SettingsDialog', 'SettingsDialog',
'MainGUI', 'MainGUI',
'Logger', 'Logger',
'PRNumberManager', 'PRNumberManager',
'ContentBuilders' 'ContentBuilders',
'WorkflowManager',
'WorkflowItem',
'GitHubRepoFetcher'
] ]
+203 -4
View File
@@ -2709,7 +2709,7 @@ class LocalGitManager:
self.logger.log(f"❌ Error saving diff to file: {str(e)}") self.logger.log(f"❌ Error saving diff to file: {str(e)}")
def create_ai_provider(provider_name: str, api_key: str, logger: Logger) -> Optional[AIProvider]: def create_ai_provider(provider_name: str, api_key: str, logger: Logger, ollama_url: str = None, ollama_model: str = None) -> Optional[AIProvider]:
"""Factory function to create AI provider instances""" """Factory function to create AI provider instances"""
if provider_name.lower() == 'claude': if provider_name.lower() == 'claude':
return ClaudeProvider(api_key, logger) return ClaudeProvider(api_key, logger)
@@ -2717,6 +2717,9 @@ def create_ai_provider(provider_name: str, api_key: str, logger: Logger) -> Opti
return ChatGPTProvider(api_key, logger) return ChatGPTProvider(api_key, logger)
elif provider_name.lower() in ['github-copilot', 'copilot', 'github_copilot']: elif provider_name.lower() in ['github-copilot', 'copilot', 'github_copilot']:
return GitHubCopilotProvider(api_key, logger) return GitHubCopilotProvider(api_key, logger)
elif provider_name.lower() == 'ollama':
# For Ollama, api_key is optional (can be empty string)
return OllamaProvider(api_key or "", logger, ollama_url, ollama_model)
else: else:
logger.log(f"⚠️ Unknown AI provider: {provider_name}") logger.log(f"⚠️ Unknown AI provider: {provider_name}")
return None return None
@@ -2927,6 +2930,198 @@ def validate_ai_provider_setup(config: dict, parent_window=None) -> bool:
return False return False
class OllamaProvider(AIProvider):
"""Ollama AI provider for self-hosted models"""
def __init__(self, api_key: str, logger: Logger, ollama_url: str = None, model: str = None):
super().__init__(api_key, logger)
self.ollama_url = ollama_url or "http://localhost:11434"
self.model = model or "llama2"
# Normalize URL
if not self.ollama_url.startswith('http'):
self.ollama_url = f"http://{self.ollama_url}"
def make_change(self, file_content: str, old_text: str, new_text: str, file_path: str, custom_instructions: str = None) -> Optional[str]:
"""Make targeted changes using Ollama"""
# Step 1: Try direct string replacement first
if old_text and old_text.strip() in file_content:
self.logger.log("✅ Making direct string replacement (reference text found exactly)")
updated_content = file_content.replace(old_text.strip(), new_text.strip())
if updated_content != file_content:
original_lines = file_content.split('\n')
updated_lines = updated_content.split('\n')
import difflib
diff = list(difflib.unified_diff(original_lines, updated_lines, lineterm=''))
changed_lines = len([line for line in diff if line.startswith('+') or line.startswith('-')])
self.logger.log(f"✅ Direct replacement successful ({changed_lines} lines changed)")
return updated_content
# Step 2: Use Ollama to generate full document with targeted changes
self.logger.log(f"📝 Using Ollama ({self.model}) to modify the document...")
return self._generate_updated_document(file_content, old_text, new_text, file_path, custom_instructions)
def _generate_updated_document(self, file_content: str, old_text: str, new_text: str, file_path: str, custom_instructions: str = None) -> Optional[str]:
"""Generate updated document content using Ollama"""
try:
import requests
# Build custom instructions text
if custom_instructions and custom_instructions.strip():
custom_instructions_text = f"""
**Additional Custom Instructions:**
{custom_instructions.strip()}
"""
else:
custom_instructions_text = ""
# Handle case where new_text is empty or just guidance
if new_text and new_text.strip() and not new_text.strip().lower().startswith('<blank'):
# We have specific replacement text
guidance_text = f"""
**Reference text to find:**
```
{old_text}
```
**Replace with this specific content:**
```
{new_text}
```
Please find the reference text and replace it with the suggested content."""
else:
# new_text is empty or just guidance - use old_text as instructions
guidance_text = f"""
**Task Instructions:**
{old_text}
**Note:** No specific replacement text provided. Use the task instructions above to determine what changes to make to improve the document. Add appropriate content based on the instructions."""
prompt = f"""**Instructions:**
Task: Update the documentation file with the changes requested.
Steps to complete:
1. Review the current file content below
2. Follow the guidance provided to determine what changes to make
3. Make appropriate improvements while maintaining existing formatting
4. Return the complete updated file content
> [!IMPORTANT]
> OUTPUT REQUIREMENTS:
> - Return ONLY the complete file content - no explanatory text, dialog, or commentary
> - Do NOT add any text before or after the file content
> - Do NOT wrap output in markdown code blocks (```), just return the raw content
> - Return the ENTIRE document - no truncation, no placeholders like [Rest of the document here...]
> - Every single line of the original document must be present in your response
> - Preserve all markdown formatting, links, and code blocks exactly
> - Only make changes that fulfill the specified request
{custom_instructions_text}
**Current File Content:**
```
{file_content}
```
{guidance_text}
Return the complete updated file content now (NO explanatory text):"""
# Prepare request headers
headers = {
"Content-Type": "application/json"
}
if self.api_key:
headers["Authorization"] = f"Bearer {self.api_key}"
# Prepare request payload
payload = {
"model": self.model,
"prompt": prompt,
"stream": False,
"options": {
"temperature": 0.3, # Lower temperature for more consistent output
"num_predict": -1, # Generate as many tokens as needed
}
}
# Make request to Ollama
self.logger.log(f"🔄 Sending request to Ollama at {self.ollama_url}...")
response = requests.post(
f"{self.ollama_url}/api/generate",
json=payload,
headers=headers,
timeout=300 # 5 minute timeout for large documents
)
response.raise_for_status()
result = response.json()
updated_content = result.get("response", "").strip()
if not updated_content:
self.logger.log("❌ Ollama returned empty response")
return None
# Clean up response
updated_content = self._clean_ai_response(updated_content)
# Validate that we got the full document back
original_line_count = len(file_content.split('\n'))
updated_line_count = len(updated_content.split('\n'))
if updated_line_count < original_line_count * 0.5: # Less than 50% of original lines
self.logger.log(f"⚠️ Warning: Updated document seems truncated ({updated_line_count} vs {original_line_count} lines)")
self.logger.log("❌ AI may have truncated the document - using fallback")
return None
self.logger.log(f"✅ Successfully generated updated document ({updated_line_count} lines)")
return updated_content
except requests.exceptions.ConnectionError:
self.logger.log(f"❌ Could not connect to Ollama server at {self.ollama_url}")
self.logger.log(" Make sure Ollama is running and the URL is correct")
return None
except requests.exceptions.Timeout:
self.logger.log("❌ Request to Ollama server timed out")
return None
except requests.exceptions.HTTPError as e:
if e.response.status_code == 401:
self.logger.log("❌ Authentication failed - check your Ollama API key")
elif e.response.status_code == 404:
self.logger.log(f"❌ Model '{self.model}' not found on Ollama server")
self.logger.log(f" Use 'ollama pull {self.model}' to download it")
else:
self.logger.log(f"❌ HTTP error from Ollama: {e}")
return None
except Exception as e:
self.logger.log(f"❌ Error calling Ollama: {str(e)}")
import traceback
traceback.print_exc()
return None
def _clean_ai_response(self, response: str) -> str:
"""Clean up AI response by removing markdown code blocks and explanatory text"""
# Remove markdown code blocks if present
if response.startswith('```'):
lines = response.split('\n')
# Remove first line if it's a code fence
if lines[0].startswith('```'):
lines = lines[1:]
# Remove last line if it's a code fence
if lines and lines[-1].strip() == '```':
lines = lines[:-1]
response = '\n'.join(lines)
return response.strip()
# AI Providers availability flag - now always True since they're included # AI Providers availability flag - now always True since they're included
AI_PROVIDERS_AVAILABLE = True AI_PROVIDERS_AVAILABLE = True
@@ -2954,7 +3149,7 @@ class AIManager:
"""Check if AI provider modules are available and return missing packages """Check if AI provider modules are available and return missing packages
Args: Args:
provider_name: 'chatgpt', 'claude', 'anthropic', or 'github-copilot' provider_name: 'chatgpt', 'claude', 'anthropic', 'github-copilot', or 'ollama'
Returns: Returns:
tuple: (all_available, missing_packages) tuple: (all_available, missing_packages)
@@ -2971,6 +3166,8 @@ class AIManager:
required_packages = required_common + ['anthropic'] required_packages = required_common + ['anthropic']
elif provider_name.lower() in ['github-copilot', 'copilot', 'github_copilot']: elif provider_name.lower() in ['github-copilot', 'copilot', 'github_copilot']:
required_packages = required_common + ['requests'] required_packages = required_common + ['requests']
elif provider_name.lower() == 'ollama':
required_packages = required_common + ['requests']
else: else:
return True, [] # Unknown provider, assume no check needed return True, [] # Unknown provider, assume no check needed
@@ -2982,6 +3179,8 @@ class AIManager:
import openai import openai
elif package == 'anthropic': elif package == 'anthropic':
import anthropic import anthropic
elif package == 'requests':
import requests
except ImportError: except ImportError:
missing_packages.append(package) missing_packages.append(package)
@@ -3159,14 +3358,14 @@ class AIManager:
if self.logger: if self.logger:
self.logger.log(f"Error in AI modules check: {str(e)}") self.logger.log(f"Error in AI modules check: {str(e)}")
def create_ai_provider(self, provider_name: str, api_key: str): def create_ai_provider(self, provider_name: str, api_key: str, ollama_url: str = None, ollama_model: str = None):
"""Create an AI provider instance""" """Create an AI provider instance"""
if not AI_PROVIDERS_AVAILABLE: if not AI_PROVIDERS_AVAILABLE:
return None return None
try: try:
ai_logger = Logger(self.log) ai_logger = Logger(self.log)
return create_ai_provider(provider_name, api_key, ai_logger) return create_ai_provider(provider_name, api_key, ai_logger, ollama_url, ollama_model)
except Exception as e: except Exception as e:
self.log(f"Error creating AI provider: {e}") self.log(f"Error creating AI provider: {e}")
return None return None
@@ -1,213 +0,0 @@
"""
Azure DevOps API Manager
Handles Azure DevOps REST API operations for work items
"""
import base64
import json
import re
import requests
from typing import List, Dict, Any, Tuple
from urllib.parse import urlparse, parse_qs
# User agent for Azure DevOps API requests
USER_AGENT = "azure-devops-github-processor/2.0"
class AzureDevOpsAPI:
"""Azure DevOps REST API client"""
def __init__(self, organization: str, pat_token: str, logger=None):
self.organization = organization
self.pat_token = pat_token
self.logger = logger
self.base_url = f"https://dev.azure.com/{organization}"
self.api_version = "7.0"
def log(self, message: str) -> None:
"""Log a message"""
if self.logger:
self.logger.log(message)
else:
print(message)
def _headers(self):
"""Get headers for Azure DevOps API requests"""
return {
"Authorization": f"Basic {base64.b64encode(f':{self.pat_token}'.encode()).decode()}",
"Content-Type": "application/json-patch+json",
"User-Agent": USER_AGENT
}
def parse_query_url(self, url: str) -> Tuple[str, str, str]:
"""Parse Azure DevOps query URL to extract org, project, and query ID
Supports both URL formats:
1. https://dev.azure.com/organization/project/_queries/query/12345/
2. https://organization.visualstudio.com/project/_queries/query/12345/
"""
parsed_url = urlparse(url)
# Check for dev.azure.com format
if 'dev.azure.com' in parsed_url.netloc:
path_parts = parsed_url.path.strip('/').split('/')
if len(path_parts) < 5:
raise ValueError("Invalid query URL format for dev.azure.com")
organization = path_parts[0]
project = path_parts[1]
# Check for visualstudio.com format
elif 'visualstudio.com' in parsed_url.netloc:
# Extract organization from subdomain (e.g., msft-skilling.visualstudio.com)
hostname_parts = parsed_url.netloc.split('.')
if len(hostname_parts) < 3 or hostname_parts[1] != 'visualstudio':
raise ValueError("Invalid visualstudio.com URL format")
organization = hostname_parts[0]
path_parts = parsed_url.path.strip('/').split('/')
if len(path_parts) < 4:
raise ValueError("Invalid query URL format for visualstudio.com")
project = path_parts[0]
else:
raise ValueError("URL must be from dev.azure.com or visualstudio.com")
# Find query ID in the URL (same logic for both formats)
query_id = None
if '_queries/query/' in url:
# Extract query ID from path
for i, part in enumerate(path_parts):
if part == 'query' and i > 0 and path_parts[i-1] == '_queries':
if i + 1 < len(path_parts):
query_id = path_parts[i + 1]
break
elif 'queryId=' in url:
match = re.search(r'queryId=([^&]+)', url)
if match:
query_id = match.group(1)
if not query_id:
raise ValueError("Could not extract query ID from URL")
return organization, project, query_id
def execute_query(self, org: str, project: str, query_id: str, token: str) -> List[Dict[str, Any]]:
"""Execute Azure DevOps query and return work items"""
# Build API URL for query execution
api_url = f"https://dev.azure.com/{org}/{project}/_apis/wit/wiql/{query_id}?api-version=6.0"
# Prepare headers
auth_string = base64.b64encode(f":{token}".encode()).decode()
headers = {
'Authorization': f'Basic {auth_string}',
'Content-Type': 'application/json',
'User-Agent': USER_AGENT
}
# Execute query
self.log(f"Executing query at: {api_url}")
response = requests.get(api_url, headers=headers, timeout=30)
if response.status_code != 200:
raise RuntimeError(f"Query execution failed: {response.status_code} - {response.text}")
query_result = response.json()
work_item_refs = query_result.get('workItems', [])
if not work_item_refs:
self.log("No work items found in query result")
return []
# Get detailed work item data
work_item_ids = [str(item['id']) for item in work_item_refs]
ids_param = ','.join(work_item_ids)
details_url = f"https://dev.azure.com/{org}/{project}/_apis/wit/workitems?ids={ids_param}&api-version=6.0"
self.log(f"Fetching details for {len(work_item_ids)} work items")
details_response = requests.get(details_url, headers=headers, timeout=30)
if details_response.status_code != 200:
raise RuntimeError(f"Work item details fetch failed: {details_response.status_code}")
return details_response.json().get('value', [])
def _get_work_items_details(self, organization: str, work_item_ids: List[str], pat_token: str) -> List[Dict[str, Any]]:
"""Get detailed information for work items"""
try:
# Build batch request URL
ids_param = ','.join(work_item_ids)
details_url = f"https://dev.azure.com/{organization}/_apis/wit/workitems"
headers = {
"Authorization": f"Basic {self._encode_pat(pat_token)}",
"Content-Type": "application/json",
"Accept": "application/json",
"User-Agent": USER_AGENT
}
params = {
"ids": ids_param,
"api-version": self.api_version,
"$expand": "fields"
}
self.log(f"Fetching details for work items: {ids_param}")
response = requests.get(details_url, headers=headers, params=params, timeout=30)
response.raise_for_status()
result = response.json()
work_items = result.get('value', [])
self.log(f"Retrieved details for {len(work_items)} work item(s)")
return work_items
except requests.RequestException as e:
raise Exception(f"Network error fetching work item details: {str(e)}")
except Exception as e:
raise Exception(f"Error fetching work item details: {str(e)}")
def add_github_link_to_work_item(self, work_item_id: str, github_url: str, link_title: str = "GitHub Issue"):
"""Add a GitHub issue/PR link to an Azure DevOps work item"""
self.log(f"Adding GitHub link to work item #{work_item_id}: {github_url}")
url = f"{self.base_url}/_apis/wit/workitems/{work_item_id}?api-version=7.0"
patch_document = [
{
"op": "add",
"path": "/relations/-",
"value": {
"rel": "Hyperlink",
"url": github_url,
"attributes": {
"comment": link_title
}
}
}
]
try:
response = requests.patch(url, headers=self._headers(), json=patch_document, timeout=30)
if response.status_code == 200:
self.log(f"✅ Successfully linked GitHub resource to work item #{work_item_id}")
return True
else:
self.log(f"❌ Failed to link GitHub resource: {response.status_code} - {response.text}")
return False
except Exception as e:
self.log(f"❌ Exception linking GitHub resource: {str(e)}")
return False
def _encode_pat(self, pat_token: str) -> str:
"""Encode PAT token for Basic authentication"""
import base64
# For Azure DevOps, username can be empty, just use :token
credentials = f":{pat_token}"
encoded = base64.b64encode(credentials.encode()).decode()
return encoded
+11 -33
View File
@@ -17,8 +17,6 @@ class ConfigManager:
def _get_default_config(self) -> Dict[str, Any]: def _get_default_config(self) -> Dict[str, Any]:
"""Get default configuration values""" """Get default configuration values"""
return { return {
'AZURE_DEVOPS_QUERY': None,
'AZURE_DEVOPS_PAT': None,
'GITHUB_PAT': None, 'GITHUB_PAT': None,
'GITHUB_REPO': None, 'GITHUB_REPO': None,
'FORKED_REPO': None, # User's fork repository 'FORKED_REPO': None, # User's fork repository
@@ -26,13 +24,11 @@ class ConfigManager:
'CLAUDE_API_KEY': None, 'CLAUDE_API_KEY': None,
'OPENAI_API_KEY': None, 'OPENAI_API_KEY': None,
'GITHUB_TOKEN': None, # For GitHub Copilot AI Provider 'GITHUB_TOKEN': None, # For GitHub Copilot AI Provider
'OLLAMA_URL': None, # Ollama server URL
'OLLAMA_API_KEY': None, # Optional Ollama API key/password
'OLLAMA_MODEL': None, # Selected Ollama model
'LOCAL_REPO_PATH': None, 'LOCAL_REPO_PATH': None,
'DRY_RUN': 'false', 'DRY_RUN': 'false',
'DATAVERSE_ENVIRONMENT_URL': None,
'DATAVERSE_TABLE_NAME': None,
'AZURE_AD_CLIENT_ID': None,
'AZURE_AD_CLIENT_SECRET': None,
'AZURE_AD_TENANT_ID': None,
'CUSTOM_INSTRUCTIONS': None # Custom AI instructions 'CUSTOM_INSTRUCTIONS': None # Custom AI instructions
} }
@@ -120,14 +116,10 @@ class ConfigManager:
def _create_default_env_file(self, config: Dict[str, Any]) -> None: def _create_default_env_file(self, config: Dict[str, Any]) -> None:
"""Create a default .env file with empty values""" """Create a default .env file with empty values"""
try: try:
env_template = """# Azure DevOps to GitHub Tool Configuration env_template = """# GitHub Pulse Configuration
# Generated automatically - fill in your values # Generated automatically - fill in your values
# IMPORTANT: Do NOT commit this file to source control. Add it to .gitignore. # IMPORTANT: Do NOT commit this file to source control. Add it to .gitignore.
# Azure DevOps Configuration
AZURE_DEVOPS_QUERY=
AZURE_DEVOPS_PAT=
# GitHub Configuration # GitHub Configuration
GITHUB_PAT= GITHUB_PAT=
GITHUB_REPO= GITHUB_REPO=
@@ -141,15 +133,11 @@ AI_PROVIDER=
CLAUDE_API_KEY= CLAUDE_API_KEY=
OPENAI_API_KEY= OPENAI_API_KEY=
GITHUB_TOKEN= GITHUB_TOKEN=
OLLAMA_URL=
OLLAMA_API_KEY=
OLLAMA_MODEL=
LOCAL_REPO_PATH= LOCAL_REPO_PATH=
# PowerApp/Dataverse Configuration (for UUF items - optional)
DATAVERSE_ENVIRONMENT_URL=
DATAVERSE_TABLE_NAME=
AZURE_AD_CLIENT_ID=
AZURE_AD_CLIENT_SECRET=
AZURE_AD_TENANT_ID=
# Custom AI Instructions (optional) # Custom AI Instructions (optional)
CUSTOM_INSTRUCTIONS= CUSTOM_INSTRUCTIONS=
""" """
@@ -179,16 +167,11 @@ CUSTOM_INSTRUCTIONS=
# Build .env file content # Build .env file content
env_content = [] env_content = []
env_content.append("# Azure DevOps to GitHub Tool Configuration") env_content.append("# GitHub Pulse Configuration")
env_content.append("# Generated by Settings Dialog") env_content.append("# Generated by Settings Dialog")
env_content.append("# IMPORTANT: Do NOT commit this file to source control. Add it to .gitignore.") env_content.append("# IMPORTANT: Do NOT commit this file to source control. Add it to .gitignore.")
env_content.append("") env_content.append("")
env_content.append("# Azure DevOps Configuration")
env_content.append(f"AZURE_DEVOPS_QUERY={self.config.get('AZURE_DEVOPS_QUERY', '')}")
env_content.append(f"AZURE_DEVOPS_PAT={self.config.get('AZURE_DEVOPS_PAT', '')}")
env_content.append("")
env_content.append("# GitHub Configuration") env_content.append("# GitHub Configuration")
env_content.append(f"GITHUB_PAT={self.config.get('GITHUB_PAT', '')}") env_content.append(f"GITHUB_PAT={self.config.get('GITHUB_PAT', '')}")
env_content.append(f"GITHUB_REPO={self.config.get('GITHUB_REPO', '')}") env_content.append(f"GITHUB_REPO={self.config.get('GITHUB_REPO', '')}")
@@ -207,17 +190,12 @@ CUSTOM_INSTRUCTIONS=
env_content.append(f"CLAUDE_API_KEY={self.config.get('CLAUDE_API_KEY', '')}") env_content.append(f"CLAUDE_API_KEY={self.config.get('CLAUDE_API_KEY', '')}")
env_content.append(f"OPENAI_API_KEY={self.config.get('OPENAI_API_KEY', '')}") env_content.append(f"OPENAI_API_KEY={self.config.get('OPENAI_API_KEY', '')}")
env_content.append(f"GITHUB_TOKEN={self.config.get('GITHUB_TOKEN', '')}") env_content.append(f"GITHUB_TOKEN={self.config.get('GITHUB_TOKEN', '')}")
env_content.append(f"OLLAMA_URL={self.config.get('OLLAMA_URL', '')}")
env_content.append(f"OLLAMA_API_KEY={self.config.get('OLLAMA_API_KEY', '')}")
env_content.append(f"OLLAMA_MODEL={self.config.get('OLLAMA_MODEL', '')}")
env_content.append(f"LOCAL_REPO_PATH={self.config.get('LOCAL_REPO_PATH', '')}") env_content.append(f"LOCAL_REPO_PATH={self.config.get('LOCAL_REPO_PATH', '')}")
env_content.append("") env_content.append("")
env_content.append("# PowerApp/Dataverse Configuration (for UUF items - optional)")
env_content.append(f"DATAVERSE_ENVIRONMENT_URL={self.config.get('DATAVERSE_ENVIRONMENT_URL', '')}")
env_content.append(f"DATAVERSE_TABLE_NAME={self.config.get('DATAVERSE_TABLE_NAME', '')}")
env_content.append(f"AZURE_AD_CLIENT_ID={self.config.get('AZURE_AD_CLIENT_ID', '')}")
env_content.append(f"AZURE_AD_CLIENT_SECRET={self.config.get('AZURE_AD_CLIENT_SECRET', '')}")
env_content.append(f"AZURE_AD_TENANT_ID={self.config.get('AZURE_AD_TENANT_ID', '')}")
env_content.append("")
env_content.append("# Custom AI Instructions (optional)") env_content.append("# Custom AI Instructions (optional)")
env_content.append(f"CUSTOM_INSTRUCTIONS={self.config.get('CUSTOM_INSTRUCTIONS', '')}") env_content.append(f"CUSTOM_INSTRUCTIONS={self.config.get('CUSTOM_INSTRUCTIONS', '')}")
env_content.append("") env_content.append("")
-255
View File
@@ -1,255 +0,0 @@
"""
Dataverse API Manager
Handles PowerApp/Dataverse operations for UUF items
"""
import json
import re
import requests
import urllib.parse
from typing import List, Dict, Any, Optional
from urllib.parse import urlparse
# Constants
USER_AGENT = "azure-devops-github-processor/2.0"
class DataverseAPI:
"""Dataverse/PowerApp API client for UUF items"""
def __init__(self, environment_url: str, table_name: str, logger=None, config: dict = None):
self.environment_url = environment_url.rstrip('/')
self.table_name = table_name
self.logger = logger
self.config = config or {}
self.access_token = None
self.api_version = "v9.2"
def log(self, message: str) -> None:
"""Log a message"""
if self.logger:
self.logger.log(message)
else:
print(message)
def authenticate(self, client_id: str, client_secret: str, tenant_id: str) -> bool:
"""Authenticate with Azure AD and get access token"""
try:
# Azure AD token endpoint
token_url = f"https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token"
# Prepare request data
data = {
'grant_type': 'client_credentials',
'client_id': client_id,
'client_secret': client_secret,
'scope': f"{self.environment_url}/.default"
}
headers = {
'Content-Type': 'application/x-www-form-urlencoded'
}
self.log("Authenticating with Azure AD...")
response = requests.post(token_url, data=data, headers=headers, timeout=30)
response.raise_for_status()
token_data = response.json()
self.access_token = token_data['access_token']
self.log("✅ Successfully authenticated with Azure AD")
return True
except requests.RequestException as e:
self.log(f"❌ Network error during authentication: {str(e)}")
return False
except KeyError as e:
self.log(f"❌ Invalid token response: {str(e)}")
return False
except Exception as e:
self.log(f"❌ Authentication error: {str(e)}")
return False
def _headers(self):
"""Get headers for Dataverse API requests"""
return {
"Authorization": f"Bearer {self.access_token}",
"OData-MaxVersion": "4.0",
"OData-Version": "4.0",
"Accept": "application/json",
"Content-Type": "application/json",
"User-Agent": USER_AGENT
}
def fetch_uuf_items(self, filter_query: Optional[str] = None) -> List[Dict[str, Any]]:
"""Fetch UUF items from Dataverse"""
try:
if not self.access_token:
raise RuntimeError("Not authenticated. Call authenticate() first.")
self.log(f"Fetching UUF items from table: {self.table_name}")
# Build API URL
api_url = f"{self.environment_url}/api/data/{self.api_version}/{self.table_name}"
# Add filter if provided
if filter_query:
api_url += f"?$filter={urllib.parse.quote(filter_query)}"
response = requests.get(api_url, headers=self._headers(), timeout=60)
if response.status_code != 200:
raise RuntimeError(f"Failed to fetch UUF items: {response.status_code} - {response.text}")
data = response.json()
items = data.get('value', [])
self.log(f"✅ Fetched {len(items)} UUF items from Dataverse")
return items
except Exception as e:
self.log(f"❌ Error fetching UUF items: {str(e)}")
raise
def process_uuf_item(self, uuf_item: dict) -> dict | None:
"""Process a single UUF item from Dataverse/PowerApp
UUF items may have different field names than Azure DevOps work items.
Adjust the field mapping based on your actual Dataverse table schema.
"""
try:
# Extract UUF item ID (adjust field names as needed)
uuf_id = uuf_item.get('cr4af_uufid') or uuf_item.get('cr4af_name') or uuf_item.get('cr_uufitemid') or 'unknown'
# Extract title
title = uuf_item.get('cr4af_title') or uuf_item.get('cr4af_subject') or uuf_item.get('cr_title') or 'No Title'
# Extract description/details
description = uuf_item.get('cr4af_description') or uuf_item.get('cr4af_details') or uuf_item.get('cr_description') or ''
if not description:
self.log(f"UUF item {uuf_id} has no description, skipping")
return None
# Extract document URL
doc_url = uuf_item.get('cr4af_documenturl') or uuf_item.get('cr4af_docurl') or uuf_item.get('cr_documenturl') or ''
if not doc_url:
self.log(f"UUF item {uuf_id} has no document URL, skipping")
return None
# Extract text to change and new text
text_to_change = uuf_item.get('cr4af_texttochange') or uuf_item.get('cr4af_currenttext') or uuf_item.get('cr_currenttext') or ''
new_text = uuf_item.get('cr4af_proposednewtext') or uuf_item.get('cr4af_newtext') or uuf_item.get('cr_newtext') or ''
if not text_to_change or not new_text:
self.log(f"UUF item {uuf_id} missing text fields, skipping")
return None
# Extract GitHub info from document URL
github_info = self._extract_github_info(doc_url)
# If the document does not include an original_content_git_url, skip this item
if not github_info.get('original_content_git_url'):
self.log(f"UUF item {uuf_id} skipped: original_content_git_url not found in document {doc_url}")
return None
processed_item = {
'id': uuf_id,
'title': title,
'nature_of_request': 'UUF Item - Modify existing docs',
'mydoc_url': doc_url,
'text_to_change': text_to_change,
'new_text': new_text,
'github_info': github_info,
'status': 'Ready',
'original_new_text': new_text,
'source': 'UUF' # Mark as UUF item
}
self.log(f"Successfully processed UUF item {uuf_id}")
return processed_item
except Exception as e:
self.log(f"Error processing UUF item {uuf_item.get('cr4af_uufid', 'unknown')}: {str(e)}")
return None
def _extract_github_info(self, doc_url: str) -> dict:
"""Extract GitHub repository info and ms.author from document URL
If GITHUB_REPO is configured in .env, it will be used instead of the repo
extracted from the document metadata. This allows you to create PRs in your
fork while preserving the file path and ms.author from the original document.
"""
try:
# Fetch the document
headers = {'User-Agent': USER_AGENT}
response = requests.get(doc_url, headers=headers, timeout=30)
response.raise_for_status()
html = response.text
# Extract ms.author
ms_author = self._extract_meta_tag(html, 'ms.author')
# Extract original_content_git_url
original_content_git_url = self._extract_meta_tag(html, 'original_content_git_url')
if not original_content_git_url:
# Try alternative extraction method
match = re.search(r"original_content_git_url[\"\']?\s*[:=]\s*[\"\']([^\"']+)[\"']", html, re.IGNORECASE)
if match:
original_content_git_url = match.group(1).strip()
if not original_content_git_url:
raise ValueError("original_content_git_url not found in document")
# Check if GITHUB_REPO is configured in .env
# If it is, use that instead of the repo from the document
configured_repo = self.config.get('GITHUB_REPO')
if configured_repo and '/' in configured_repo:
# Use the configured repository (e.g., "b-tsammons/fabric-docs-pr")
parts = configured_repo.split('/', 1)
owner = parts[0].strip()
repo = parts[1].strip()
self.log(f"Using configured GITHUB_REPO: {owner}/{repo} (overriding document metadata)")
else:
# Parse GitHub owner/repo from original_content_git_url (fallback to document metadata)
owner, repo = self._parse_github_url(original_content_git_url)
self.log(f"Using repository from document metadata: {owner}/{repo}")
return {
'ms_author': ms_author,
'original_content_git_url': original_content_git_url,
'owner': owner,
'repo': repo
}
except Exception as e:
self.log(f"Error extracting GitHub info from {doc_url}: {str(e)}")
return {
'ms_author': None,
'original_content_git_url': None,
'owner': None,
'repo': None,
'error': str(e)
}
def _extract_meta_tag(self, html: str, name: str) -> str | None:
"""Extract content from meta tag"""
pattern = rf'<meta\s+(?:[^>]*?\s)?(?:name|property)\s*=\s*["\'](?P<n>{re.escape(name)})["\']\s+[^>]*?\bcontent\s*=\s*["\'](?P<content>[^"\']+)["\'][^>]*?>'
match = re.search(pattern, html, re.IGNORECASE)
if match:
return match.group('content').strip()
return None
def _parse_github_url(self, url: str) -> tuple[str, str]:
"""Parse GitHub URL to extract owner and repo"""
parsed = urlparse(url)
if "github.com" not in parsed.netloc.lower():
raise ValueError(f"Not a GitHub URL: {url}")
parts = [p for p in parsed.path.split("/") if p]
if len(parts) < 2:
raise ValueError(f"Unable to parse owner/repo from: {url}")
return parts[0], parts[1]
+16 -21
View File
@@ -12,7 +12,7 @@ from urllib.parse import urlparse
# Constants # Constants
GITHUB_GRAPHQL_ENDPOINT = "https://api.github.com/graphql" GITHUB_GRAPHQL_ENDPOINT = "https://api.github.com/graphql"
USER_AGENT = "azure-devops-github-processor/2.0" USER_AGENT = "github-automation-tool/1.0"
class GitHubGQL: class GitHubGQL:
@@ -552,7 +552,7 @@ class GitHubGQL:
raise raise
def create_pull_request(self, repository_id: str, title: str, body: str, head_ref: str, base_ref: str = "main") -> tuple[str, str, int]: def create_pull_request(self, repository_id: str, title: str, body: str, head_ref: str, base_ref: str = "main") -> tuple[str, str, int]:
"""Create a pull request with AB# linking""" """Create a pull request"""
self.log(f"Creating pull request with createPullRequest mutation from {head_ref} to {base_ref}...") self.log(f"Creating pull request with createPullRequest mutation from {head_ref} to {base_ref}...")
mutation = """ mutation = """
mutation($repositoryId:ID!, $title:String!, $body:String!, $headRefName:String!, $baseRefName:String!) { mutation($repositoryId:ID!, $title:String!, $body:String!, $headRefName:String!, $baseRefName:String!) {
@@ -664,8 +664,8 @@ class GitHubGQL:
old_text: Text to find and replace old_text: Text to find and replace
new_text: New text to replace with new_text: New text to replace with
branch_name: Branch name for this PR branch_name: Branch name for this PR
work_item_id: Work item or UUF issue ID work_item_id: Reference ID for tracking (optional)
item_source: Source of the item ('UUF' or 'Azure DevOps') item_source: Source of the item (optional)
Returns True if successful, False otherwise. Returns True if successful, False otherwise.
""" """
@@ -680,25 +680,22 @@ class GitHubGQL:
"User-Agent": USER_AGENT "User-Agent": USER_AGENT
} }
# Build work item reference # Build reference ID if provided
if work_item_id: if work_item_id:
if item_source == 'UUF': reference_id = f"**Reference ID:** {work_item_id}\n"
work_item_ref = f"**UUF Issue:** {work_item_id}\n"
else:
work_item_ref = f"**Azure DevOps Work Item:** AB#{work_item_id}\n"
else: else:
work_item_ref = "" reference_id = ""
# Build document reference # Build document reference
if file_path and not file_path.startswith("See work item") and not file_path.startswith("File path not specified"): if file_path and not file_path.startswith("File path not specified"):
doc_ref = f"**Document to modify:** `{file_path}`\n" doc_ref = f"**Document to modify:** `{file_path}`\n"
file_instruction = f"2. Locate the file: `{file_path}`" file_instruction = f"2. Locate the file: `{file_path}`"
elif doc_url: elif doc_url:
doc_ref = f"**Document URL:** {doc_url}\n" doc_ref = f"**Document URL:** {doc_url}\n"
file_instruction = f"2. Locate the file from this document URL: {doc_url}" file_instruction = f"2. Locate the file from this document URL: {doc_url}"
else: else:
doc_ref = "**Note:** File path not specified in work item\n" doc_ref = "**Note:** File path not specified\n"
file_instruction = "2. Review the PR description and work item details to identify the file(s) that need to be modified" file_instruction = "2. Review the PR description to identify the file(s) that need to be modified"
# Build custom instructions section # Build custom instructions section
if custom_instructions and custom_instructions.strip(): if custom_instructions and custom_instructions.strip():
@@ -713,25 +710,24 @@ class GitHubGQL:
# Create a comment mentioning @copilot with VERY explicit instructions # Create a comment mentioning @copilot with VERY explicit instructions
comment_body = f"""@copilot comment_body = f"""@copilot
{work_item_ref}{doc_ref} {reference_id}{doc_ref}
**Instructions:** **Instructions:**
Task: Update the documentation file with the changes requested above. Task: Update the file with the changes requested above.
Steps to complete: Steps to complete:
Locate the file containing the reference shown below. Locate the file containing the reference shown below.
Find the reference text within the file Find the reference text within the file
Replace it with the 'Proposed New Text' shown above or use the reference as guidance Replace it with the 'Proposed New Text' shown above or use the reference as guidance
Maintain the existing formatting, indentation, and markdown structure Maintain the existing formatting, indentation, and structure
Ensure no other content in the file is modified Ensure no other content in the file is modified
> [!IMPORTANT] > [!IMPORTANT]
> Only replace the specified text - do not make additional changes. > Only replace the specified text - do not make additional changes.
> Preserve all markdown formatting, links, and code blocks. > Preserve all formatting, links, and code blocks.
> If the current text cannot be found exactly, search for similar text. > If the current text cannot be found exactly, search for similar text.
> Please ensure the changes align with Microsoft documentation standards.
> Do not remove any text unless the reference or suggested guidance indicates to do so, if the text is obsolete or incorrect. > Do not remove any text unless the reference or suggested guidance indicates to do so, if the text is obsolete or incorrect.
1. Make changes to `{branch_name}` branch for this pull request. 1. Make changes to `{branch_name}` branch for this pull request.
@@ -748,14 +744,13 @@ Ensure no other content in the file is modified
{new_text} {new_text}
``` ```
5. Ensure the changes align with the context of the work item. 5. Ensure the changes align with the context provided.
6. Do a freshness check to ensure the file content is up-to-date before making changes. 6. Do a freshness check to ensure the file content is up-to-date before making changes.
7. Commit the changes to the `{branch_name}` branch 7. Commit the changes to the `{branch_name}` branch
> [!NOTE] > [!NOTE]
> This documentation is maintained by spelluru.
> If guidance is empty, follow the reference to make changes. > If guidance is empty, follow the reference to make changes.
{custom_instructions_section} {custom_instructions_section}
@@ -848,7 +843,7 @@ Thank you!
{new_text} {new_text}
``` ```
**Automated Suggestion:** This change was requested in Azure DevOps work item. **Automated Suggestion:** This change was requested.
Click "Commit suggestion" above to apply this change directly to the PR.""" Click "Commit suggestion" above to apply this change directly to the PR."""
+636 -46
View File
@@ -12,9 +12,8 @@ from typing import List, Dict, Any, Optional
from .utils import Logger from .utils import Logger
from .settings_dialog import SettingsDialog from .settings_dialog import SettingsDialog
from .work_item_processor import WorkItemProcessor # Removed imports: WorkItemProcessor, AzureDevOpsAPI, DataverseAPI
from .azure_devops_api import AzureDevOpsAPI # These were specific to Azure DevOps integration
from .dataverse_api import DataverseAPI
class HyperlinkDialog: class HyperlinkDialog:
@@ -136,8 +135,9 @@ class MainGUI:
# Initialize logger after GUI is created # Initialize logger after GUI is created
self.logger = Logger(self.log_text) self.logger = Logger(self.log_text)
# Initialize work item processor # Initialize work item processor - REMOVED (was Azure DevOps specific)
self.work_item_processor = WorkItemProcessor(self.logger, self.config_manager.get_config()) # self.work_item_processor = WorkItemProcessor(self.logger, self.config_manager.get_config())
self.work_item_processor = None # Placeholder for future implementation
# Initialize cache manager # Initialize cache manager
from .cache_manager import CacheManager from .cache_manager import CacheManager
@@ -222,7 +222,7 @@ class MainGUI:
title_frame.columnconfigure(0, weight=1) title_frame.columnconfigure(0, weight=1)
# Title # Title
title_label = ttk.Label(title_frame, text="MicrosoftDocFlow v3", title_label = ttk.Label(title_frame, text="GitHub Pulse",
font=('Arial', 16, 'bold')) font=('Arial', 16, 'bold'))
title_label.grid(row=0, column=0, sticky=tk.W) title_label.grid(row=0, column=0, sticky=tk.W)
@@ -237,54 +237,178 @@ class MainGUI:
self.settings_button.grid(row=0, column=2, sticky=tk.E, padx=(5, 0)) self.settings_button.grid(row=0, column=2, sticky=tk.E, padx=(5, 0))
def _create_controls_section(self, parent): def _create_controls_section(self, parent):
"""Create work item controls section""" """Create GitHub Tools section"""
# Work Item Details group frame # GitHub Tools group frame
workitem_frame = ttk.LabelFrame(parent, text="📋 Work Item Details", tools_frame = ttk.LabelFrame(parent, text="🔧 GitHub Tools",
style='WorkItem.TLabelframe', padding="15") style='WorkItem.TLabelframe', padding="15")
workitem_frame.grid(row=1, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=(0, 15), padx=5) tools_frame.grid(row=1, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=(0, 15), padx=5)
workitem_frame.columnconfigure(2, weight=1) tools_frame.columnconfigure(1, weight=1)
# Controls row # Initialize workflow data
controls_row = ttk.Frame(workitem_frame) self.target_repos = []
controls_row.grid(row=0, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=(0, 10)) self.forked_repos = []
controls_row.columnconfigure(6, weight=1) self.workflow_items = []
self.current_workflow_items = []
# Fetch buttons # Get current config
self.fetch_button = ttk.Button(controls_row, text="📥 Fetch Work Items", config = self.config_manager.get_config()
command=self.start_fetch_work_items)
self.fetch_button.grid(row=0, column=0, padx=(0, 10))
self.fetch_uuf_button = ttk.Button(controls_row, text="📋 Fetch UUF Items", # Row 0: Mode Selection
command=self.start_fetch_uuf_items) mode_frame = ttk.Frame(tools_frame)
self.fetch_uuf_button.grid(row=0, column=1, padx=(0, 15)) mode_frame.grid(row=0, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=(0, 10), padx=5)
# Navigation buttons ttk.Label(mode_frame, text="Mode:", font=('Arial', 10, 'bold')).grid(
self.prev_button = ttk.Button(controls_row, text="← Previous", row=0, column=0, sticky=tk.W, padx=(0, 10))
command=self.previous_item, state='disabled')
self.prev_button.grid(row=0, column=2, padx=(0, 5))
self.next_button = ttk.Button(controls_row, text="Next →", self.tools_mode_var = tk.StringVar(value="action")
command=self.next_item, state='disabled') create_radio = ttk.Radiobutton(mode_frame, text="✏️ Create PR/Issue", variable=self.tools_mode_var,
self.next_button.grid(row=0, column=3, padx=(0, 15)) value="create", command=self._on_mode_changed)
create_radio.grid(row=0, column=1, padx=(0, 15))
# Action type dropdown action_radio = ttk.Radiobutton(mode_frame, text="📋 Action Existing PR/Issue", variable=self.tools_mode_var,
self.action_type_var = tk.StringVar(value="Create Issue") value="action", command=self._on_mode_changed)
self.action_type_dropdown = ttk.Combobox(controls_row, textvariable=self.action_type_var, action_radio.grid(row=0, column=2, padx=(0, 10))
values=["Create Issue", "Create PR"],
state="readonly", width=15)
self.action_type_dropdown.grid(row=0, column=4, padx=(0, 5))
self.action_type_dropdown.bind("<<ComboboxSelected>>", lambda e: self.update_action_button_text())
# GO button # Separator
self.go_button = ttk.Button(controls_row, text="🚀 GO", ttk.Separator(tools_frame, orient='horizontal').grid(row=1, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=(5, 10))
command=self.create_github_resource, state='disabled')
self.go_button.grid(row=0, column=5, padx=(0, 20)) # Row 2: Target Repository
self.target_repo_label = ttk.Label(tools_frame, text="Target Repository:", font=('Arial', 10, 'bold'))
self.target_repo_label.grid(row=2, column=0, sticky=tk.W, pady=5, padx=5)
target_frame = ttk.Frame(tools_frame)
target_frame.grid(row=2, column=1, sticky=(tk.W, tk.E), pady=5, padx=5)
target_frame.columnconfigure(0, weight=1)
self.target_repo_var = tk.StringVar(value=config.get('GITHUB_REPO', ''))
self.target_repo_dropdown = ttk.Combobox(target_frame, textvariable=self.target_repo_var,
values=[''], width=60)
self.target_repo_dropdown.grid(row=0, column=0, sticky=(tk.W, tk.E), padx=(0, 5))
self.target_repo_dropdown.bind('<KeyRelease>', self._on_target_repo_search)
self.target_repo_dropdown.bind('<<ComboboxSelected>>', lambda e: self._on_repo_selection_changed())
refresh_target_btn = ttk.Button(target_frame, text="🔄", width=3,
command=self._refresh_target_repos)
refresh_target_btn.grid(row=0, column=1, padx=(0, 2))
search_target_btn = ttk.Button(target_frame, text="🔍", width=3,
command=self._search_target_repos)
search_target_btn.grid(row=0, column=2)
# Row 3: Forked Repository
self.forked_repo_label = ttk.Label(tools_frame, text="Forked Repository:", font=('Arial', 10, 'bold'))
self.forked_repo_label.grid(row=3, column=0, sticky=tk.W, pady=5, padx=5)
self.fork_frame = ttk.Frame(tools_frame)
self.fork_frame.grid(row=3, column=1, sticky=(tk.W, tk.E), pady=5, padx=5)
self.fork_frame.columnconfigure(0, weight=1)
self.forked_repo_var = tk.StringVar(value=config.get('FORKED_REPO', ''))
self.forked_repo_dropdown = ttk.Combobox(self.fork_frame, textvariable=self.forked_repo_var,
values=[''], width=60)
self.forked_repo_dropdown.grid(row=0, column=0, sticky=(tk.W, tk.E), padx=(0, 5))
self.forked_repo_dropdown.bind('<<ComboboxSelected>>', lambda e: self._on_repo_selection_changed())
refresh_fork_btn = ttk.Button(self.fork_frame, text="🔄", width=3,
command=self._refresh_forked_repos)
refresh_fork_btn.grid(row=0, column=1, padx=(0, 2))
clone_fork_btn = ttk.Button(self.fork_frame, text="📥", width=3,
command=self._clone_forked_repo)
clone_fork_btn.grid(row=0, column=2)
# Row 4: Action Mode Controls (View toggles and load button)
self.action_controls_row = ttk.Frame(tools_frame)
self.action_controls_row.grid(row=4, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=(10, 5), padx=5)
self.action_controls_row.columnconfigure(2, weight=1)
# Repo source toggle
ttk.Label(self.action_controls_row, text="View:", font=('Arial', 10, 'bold')).grid(
row=0, column=0, sticky=tk.W, padx=(0, 5))
self.repo_source_var = tk.StringVar(value="target")
target_radio = ttk.Radiobutton(self.action_controls_row, text="Target", variable=self.repo_source_var,
value="target", command=self._filter_workflow_items)
target_radio.grid(row=0, column=1, padx=(0, 5))
fork_radio = ttk.Radiobutton(self.action_controls_row, text="Fork", variable=self.repo_source_var,
value="fork", command=self._filter_workflow_items)
fork_radio.grid(row=0, column=2, padx=(0, 15))
# Item type toggle
self.item_type_var = tk.StringVar(value="pull_request")
pr_radio = ttk.Radiobutton(self.action_controls_row, text="PRs", variable=self.item_type_var,
value="pull_request", command=self._filter_workflow_items)
pr_radio.grid(row=0, column=3, padx=(0, 5))
issue_radio = ttk.Radiobutton(self.action_controls_row, text="Issues", variable=self.item_type_var,
value="issue", command=self._filter_workflow_items)
issue_radio.grid(row=0, column=4, padx=(0, 15))
# Fetch button
self.fetch_workflow_btn = ttk.Button(self.action_controls_row, text="📥 Load Items",
command=self._load_workflow_items)
self.fetch_workflow_btn.grid(row=0, column=5, padx=(0, 10))
# Item counter # Item counter
self.item_counter_label = ttk.Label(controls_row, text="No items loaded", self.item_counter_label = ttk.Label(self.action_controls_row, text="No items loaded",
font=('Arial', 9, 'italic')) font=('Arial', 9, 'italic'))
self.item_counter_label.grid(row=0, column=6, sticky=tk.E) self.item_counter_label.grid(row=0, column=6, sticky=tk.E)
# Row 5: Workflow items dropdown (Action Mode)
self.action_item_label = ttk.Label(tools_frame, text="Select Item:", font=('Arial', 10, 'bold'))
self.action_item_label.grid(row=5, column=0, sticky=tk.W, pady=5, padx=5)
action_item_dropdown_frame = ttk.Frame(tools_frame)
action_item_dropdown_frame.grid(row=5, column=1, sticky=(tk.W, tk.E), pady=5, padx=5)
action_item_dropdown_frame.columnconfigure(0, weight=1)
self.workflow_item_var = tk.StringVar()
self.workflow_item_dropdown = ttk.Combobox(action_item_dropdown_frame, textvariable=self.workflow_item_var,
values=[''], width=60, state='readonly')
self.workflow_item_dropdown.grid(row=0, column=0, sticky=(tk.W, tk.E))
self.workflow_item_dropdown.bind('<<ComboboxSelected>>', self._on_workflow_item_selected)
# Row 4-5: Create Mode Controls (hidden by default)
self.create_controls_row = ttk.Frame(tools_frame)
self.create_controls_row.grid(row=4, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=(10, 5), padx=5)
self.create_controls_row.columnconfigure(1, weight=1)
# Create type selection
ttk.Label(self.create_controls_row, text="Create:", font=('Arial', 10, 'bold')).grid(
row=0, column=0, sticky=tk.W, padx=(0, 5))
self.create_type_var = tk.StringVar(value="pull_request")
create_pr_radio = ttk.Radiobutton(self.create_controls_row, text="📝 Pull Request",
variable=self.create_type_var, value="pull_request")
create_pr_radio.grid(row=0, column=1, padx=(0, 15), sticky=tk.W)
create_issue_radio = ttk.Radiobutton(self.create_controls_row, text="🐛 Issue",
variable=self.create_type_var, value="issue")
create_issue_radio.grid(row=0, column=2, padx=(0, 15), sticky=tk.W)
# Create button
self.create_item_btn = ttk.Button(self.create_controls_row, text="✏️ Create New",
command=self._create_new_item)
self.create_item_btn.grid(row=0, column=3, padx=(0, 10))
# Store references for show/hide
self.action_mode_widgets = [
self.action_controls_row,
self.action_item_label,
action_item_dropdown_frame
]
self.create_mode_widgets = [
self.create_controls_row
]
# Initialize mode (show action, hide create)
self._on_mode_changed()
# Start loading repos
self.root.after(100, self._init_load_repos)
def _create_status_section(self, parent): def _create_status_section(self, parent):
"""Create progress and status section""" """Create progress and status section"""
# Progress bar # Progress bar
@@ -664,7 +788,10 @@ class MainGUI:
# Process items # Process items
self.current_work_items = [] self.current_work_items = []
for item in work_items: for item in work_items:
processed_item = self.work_item_processor.process_work_item(item) # REMOVED: Azure DevOps specific processing
# processed_item = self.work_item_processor.process_work_item(item)
# TODO: Implement custom item processing here
processed_item = item # Placeholder
if processed_item: if processed_item:
self.current_work_items.append(processed_item) self.current_work_items.append(processed_item)
@@ -723,7 +850,10 @@ class MainGUI:
# Process items # Process items
self.current_work_items = [] self.current_work_items = []
for item in uuf_items: for item in uuf_items:
processed_item = self.work_item_processor.process_uuf_item(item) # REMOVED: UUF/Dataverse specific processing
# processed_item = self.work_item_processor.process_uuf_item(item)
# TODO: Implement custom item processing here
processed_item = item # Placeholder
if processed_item: if processed_item:
self.current_work_items.append(processed_item) self.current_work_items.append(processed_item)
@@ -1850,14 +1980,28 @@ Proposed new text:
api_key = config.get('OPENAI_API_KEY', '').strip() api_key = config.get('OPENAI_API_KEY', '').strip()
elif ai_provider in ['github-copilot', 'copilot', 'github_copilot']: elif ai_provider in ['github-copilot', 'copilot', 'github_copilot']:
api_key = config.get('GITHUB_TOKEN', '').strip() api_key = config.get('GITHUB_TOKEN', '').strip()
elif ai_provider == 'ollama':
api_key = config.get('OLLAMA_API_KEY', '').strip() # Optional for Ollama
else: else:
api_key = '' api_key = ''
github_token = config.get('GITHUB_PAT', '').strip() github_token = config.get('GITHUB_PAT', '').strip()
local_repo_path = config.get('LOCAL_REPO_PATH', '').strip() or None local_repo_path = config.get('LOCAL_REPO_PATH', '').strip() or None
if not api_key: # Validate API key (except for Ollama where it's optional)
if not api_key and ai_provider != 'ollama':
raise ValueError(f"No API key configured for {ai_provider}. Please configure in Settings.") raise ValueError(f"No API key configured for {ai_provider}. Please configure in Settings.")
# Get Ollama-specific configuration
ollama_url = None
ollama_model = None
if ai_provider == 'ollama':
ollama_url = config.get('OLLAMA_URL', '').strip()
ollama_model = config.get('OLLAMA_MODEL', '').strip()
if not ollama_url:
raise ValueError("Ollama Server URL not configured. Please configure in Settings.")
if not ollama_model:
raise ValueError("Ollama Model not selected. Please configure in Settings.")
self.logger.log(f"Using AI Provider: {ai_provider.upper()}") self.logger.log(f"Using AI Provider: {ai_provider.upper()}")
# Create AI manager # Create AI manager
@@ -1865,7 +2009,7 @@ Proposed new text:
ai_manager = AIManager(self.logger) ai_manager = AIManager(self.logger)
# Create AI provider instance # Create AI provider instance
ai_provider_instance = ai_manager.create_ai_provider(ai_provider, api_key) ai_provider_instance = ai_manager.create_ai_provider(ai_provider, api_key, ollama_url, ollama_model)
if not ai_provider_instance: if not ai_provider_instance:
raise ValueError(f"Failed to create {ai_provider} provider") raise ValueError(f"Failed to create {ai_provider} provider")
@@ -2175,6 +2319,452 @@ Proposed new text:
except Exception as e: except Exception as e:
self.logger.log(f"Error checking AI provider setup: {str(e)}") self.logger.log(f"Error checking AI provider setup: {str(e)}")
# ===== GitHub Tools Methods =====
def _init_load_repos(self):
"""Initialize loading of repositories"""
self._load_target_repos_async()
self._load_forked_repos_async()
def _load_target_repos_async(self):
"""Load target repositories asynchronously"""
def load_repos():
try:
config = self.config_manager.get_config()
github_token = config.get('GITHUB_PAT', '')
if not github_token:
return
from .workflow import GitHubRepoFetcher
repo_fetcher = GitHubRepoFetcher(github_token, self.logger)
repos = repo_fetcher.fetch_repos_with_permissions(min_permission='push')
self.target_repos = repo_fetcher.get_repo_names(repos)
# Update dropdown on main thread
self.root.after(0, self._update_target_dropdown)
except Exception as e:
self.logger.log(f"Error loading target repos: {e}")
threading.Thread(target=load_repos, daemon=True).start()
def _update_target_dropdown(self):
"""Update the target repository dropdown"""
try:
current_values = [''] # Start with empty option
# Add user's repos with edit access
if self.target_repos:
current_values.append('--- Your Repos (with edit access) ---')
current_values.extend(self.target_repos)
self.target_repo_dropdown['values'] = current_values
except Exception as e:
self.logger.log(f"Error updating target dropdown: {e}")
def _refresh_target_repos(self):
"""Refresh target repositories"""
self._load_target_repos_async()
def _search_target_repos(self):
"""Search for repositories on GitHub"""
query = self.target_repo_var.get().strip()
if not query:
return
def search_repos():
try:
config = self.config_manager.get_config()
github_token = config.get('GITHUB_PAT', '')
if not github_token:
return
from .workflow import GitHubRepoFetcher
repo_fetcher = GitHubRepoFetcher(github_token, self.logger)
repos = repo_fetcher.search_repositories(query, per_page=50)
search_results = repo_fetcher.get_repo_names(repos)
# Update dropdown on main thread
self.root.after(0, lambda: self._update_target_search_results(search_results))
except Exception as e:
self.logger.log(f"Error searching repos: {e}")
threading.Thread(target=search_repos, daemon=True).start()
def _update_target_search_results(self, search_results):
"""Update target dropdown with search results"""
try:
current_values = ['']
# Add user's repos
if self.target_repos:
current_values.append('--- Your Repos (with edit access) ---')
current_values.extend(self.target_repos)
# Add search results
if search_results:
current_values.append('--- Search Results ---')
current_values.extend(search_results)
self.target_repo_dropdown['values'] = current_values
except Exception as e:
self.logger.log(f"Error updating search results: {e}")
def _on_target_repo_search(self, _event):
"""Handle typing in target repo field for auto-search"""
# Debounce: only search after user stops typing for 500ms
if hasattr(self, '_search_timer'):
self.root.after_cancel(self._search_timer)
query = self.target_repo_var.get().strip()
if len(query) >= 3: # Only search if at least 3 characters
self._search_timer = self.root.after(500, self._search_target_repos)
def _load_forked_repos_async(self):
"""Load forked repositories asynchronously"""
def load_forks():
try:
config = self.config_manager.get_config()
github_token = config.get('GITHUB_PAT', '')
local_repo_path = config.get('LOCAL_REPO_PATH', '')
# Load local repos
local_repos = []
if local_repo_path:
try:
from .utils import LocalRepositoryScanner
local_repos = LocalRepositoryScanner.scan_local_repos(local_repo_path)
except Exception as e:
self.logger.log(f"Error scanning local repos: {e}")
# Load GitHub repos
github_repos = []
if github_token:
try:
from .workflow import GitHubRepoFetcher
repo_fetcher = GitHubRepoFetcher(github_token, self.logger)
repos = repo_fetcher.fetch_user_repos(repo_type='owner')
github_repos = repo_fetcher.get_repo_names(repos)
except Exception as e:
self.logger.log(f"Error loading GitHub repos: {e}")
self.forked_repos = {'local': local_repos, 'github': github_repos}
# Update dropdown on main thread
self.root.after(0, self._update_forked_dropdown)
except Exception as e:
self.logger.log(f"Error loading forked repos: {e}")
threading.Thread(target=load_forks, daemon=True).start()
def _update_forked_dropdown(self):
"""Update the forked repository dropdown"""
try:
current_values = [''] # Start with empty option
# Add local repos
if self.forked_repos.get('local'):
current_values.append('--- Local Repositories ---')
current_values.extend(self.forked_repos['local'])
# Add GitHub repos
if self.forked_repos.get('github'):
current_values.append('--- Your GitHub Forks ---')
current_values.extend(self.forked_repos['github'])
self.forked_repo_dropdown['values'] = current_values
except Exception as e:
self.logger.log(f"Error updating forked dropdown: {e}")
def _refresh_forked_repos(self):
"""Refresh forked repositories"""
self._load_forked_repos_async()
def _clone_forked_repo(self):
"""Clone the selected forked repository"""
selected_repo = self.forked_repo_var.get().strip()
# Validate selection
if not selected_repo:
messagebox.showwarning("No Repository Selected",
"Please select a repository to clone.")
return
# Check if it's a section header
if selected_repo.startswith('---'):
messagebox.showwarning("Invalid Selection",
"Please select a repository, not a section header.")
return
config = self.config_manager.get_config()
local_repo_path = config.get('LOCAL_REPO_PATH', '').strip()
if not local_repo_path:
messagebox.showwarning("Local Path Not Configured",
"Please configure the Local Repository Path in settings first.")
return
# Clone logic (similar to settings_dialog.py)
import subprocess
import os
try:
os.makedirs(local_repo_path, exist_ok=True)
except Exception as e:
messagebox.showerror("Directory Error",
f"Could not create local repository directory:\n{str(e)}")
return
# Extract repo name
repo_name = selected_repo
if '/' not in repo_name:
messagebox.showerror("Invalid Repository",
"Repository must be in 'owner/repo' format.")
return
folder_name = repo_name.split('/')[-1]
target_path = os.path.join(local_repo_path, folder_name)
if os.path.exists(target_path):
response = messagebox.askyesno("Directory Exists",
f"The directory '{folder_name}' already exists.\n\n"
f"Do you want to continue anyway?")
if not response:
return
clone_url = f"https://github.com/{repo_name}.git"
def clone_repo():
try:
result = subprocess.run(
['git', 'clone', clone_url, target_path],
capture_output=True,
text=True,
timeout=300
)
if result.returncode == 0:
self.root.after(0, lambda: messagebox.showinfo(
"Clone Successful",
f"Successfully cloned {repo_name}!"))
self.root.after(0, self._refresh_forked_repos)
else:
error_msg = result.stderr if result.stderr else result.stdout
self.root.after(0, lambda: messagebox.showerror(
"Clone Failed",
f"Failed to clone {repo_name}.\n\n{error_msg}"))
except subprocess.TimeoutExpired:
self.root.after(0, lambda: messagebox.showerror(
"Clone Timeout",
f"Cloning {repo_name} timed out after 5 minutes."))
except FileNotFoundError:
self.root.after(0, lambda: messagebox.showerror(
"Git Not Found",
"Git is not installed or not found in PATH."))
except Exception as e:
self.root.after(0, lambda: messagebox.showerror(
"Clone Error",
f"An error occurred while cloning:\n{str(e)}"))
messagebox.showinfo("Cloning Repository",
f"Cloning {repo_name} to:\n{target_path}\n\n"
f"This may take a few moments...")
threading.Thread(target=clone_repo, daemon=True).start()
def _on_mode_changed(self):
"""Handle mode change between Create and Action"""
mode = self.tools_mode_var.get()
if mode == "action":
# Show action mode widgets
for widget in self.action_mode_widgets:
widget.grid()
# Hide create mode widgets
for widget in self.create_mode_widgets:
widget.grid_remove()
else: # create mode
# Hide action mode widgets
for widget in self.action_mode_widgets:
widget.grid_remove()
# Show create mode widgets
for widget in self.create_mode_widgets:
widget.grid()
def _create_new_item(self):
"""Handle creating a new PR or Issue"""
create_type = self.create_type_var.get()
target_repo = self.target_repo_var.get().strip()
# Skip section headers
if target_repo.startswith('---'):
target_repo = ''
if not target_repo:
messagebox.showwarning("No Repository Selected",
"Please select a target repository.")
return
if create_type == "pull_request":
# TODO: Implement PR creation workflow
messagebox.showinfo("Create Pull Request",
f"PR creation workflow for {target_repo} will be implemented here.\n\n"
"This will open the PR creation interface in the tabs below.")
else: # issue
# TODO: Implement Issue creation workflow
messagebox.showinfo("Create Issue",
f"Issue creation workflow for {target_repo} will be implemented here.\n\n"
"This will open the Issue creation interface in the tabs below.")
def _on_repo_selection_changed(self):
"""Handle repository selection change"""
# Clear workflow items when repos change
self.workflow_items = []
self.current_workflow_items = []
self.workflow_item_dropdown['values'] = ['']
self.workflow_item_var.set('')
self.item_counter_label.config(text="No items loaded")
def _load_workflow_items(self):
"""Load workflow items from selected repositories"""
target_repo = self.target_repo_var.get().strip()
forked_repo = self.forked_repo_var.get().strip()
# Skip section headers
if target_repo.startswith('---'):
target_repo = ''
if forked_repo.startswith('---'):
forked_repo = ''
if not target_repo and not forked_repo:
messagebox.showwarning("No Repositories Selected",
"Please select at least one repository.")
return
self.progress.start()
self.update_status("Loading workflow items...")
def load_items():
try:
config = self.config_manager.get_config()
github_token = config.get('GITHUB_PAT', '')
from .workflow import WorkflowManager
workflow_manager = WorkflowManager(github_token, self.logger)
# Fetch all items
results = workflow_manager.fetch_all_workflow_items(
target_repo=target_repo if target_repo else None,
fork_repo=forked_repo if forked_repo else None,
include_issues=True,
include_prs=True,
state='open' # Only load open items
)
self.workflow_items = results
# Update UI on main thread
self.root.after(0, self._on_workflow_items_loaded)
except Exception as e:
self.logger.log(f"Error loading workflow items: {e}")
self.root.after(0, lambda: self.update_status("Failed to load workflow items"))
self.root.after(0, lambda: messagebox.showerror(
"Load Error",
f"Failed to load workflow items:\n{str(e)}"))
finally:
self.root.after(0, self.progress.stop)
threading.Thread(target=load_items, daemon=True).start()
def _on_workflow_items_loaded(self):
"""Handle workflow items loaded"""
total_items = sum(len(items) for items in self.workflow_items.values())
self.logger.log(f"Loaded {total_items} workflow items")
self.update_status(f"Loaded {total_items} workflow items")
# Apply current filters
self._filter_workflow_items()
def _filter_workflow_items(self):
"""Filter workflow items based on current selections"""
if not self.workflow_items:
return
repo_source = self.repo_source_var.get() # 'target' or 'fork'
item_type = self.item_type_var.get() # 'pull_request' or 'issue'
# Get the appropriate list
# WorkflowManager returns keys like: 'target_prs', 'target_issues', 'fork_prs', 'fork_issues'
key = f"{repo_source}_prs" if item_type == 'pull_request' else f"{repo_source}_issues"
filtered_items = self.workflow_items.get(key, [])
# Update dropdown
self.current_workflow_items = filtered_items
item_options = [
f"#{item.number} - {item.title}" for item in filtered_items
]
self.workflow_item_dropdown['values'] = item_options if item_options else ['']
self.workflow_item_var.set('')
# Update counter
count = len(filtered_items)
source_name = "Target" if repo_source == "target" else "Fork"
type_name = "PRs" if item_type == "pull_request" else "Issues"
self.item_counter_label.config(text=f"{count} {source_name} {type_name}")
def _on_workflow_item_selected(self, _event):
"""Handle workflow item selection"""
selected = self.workflow_item_var.get()
if not selected:
return
# Extract item number from selection
try:
item_number = int(selected.split('#')[1].split(' ')[0])
# Find the item
for item in self.current_workflow_items:
if item.number == item_number:
self._display_workflow_item(item)
break
except Exception as e:
self.logger.log(f"Error selecting workflow item: {e}")
def _display_workflow_item(self, item):
"""Display workflow item details"""
# Update Current Work Item tab
self.work_item_id_label.config(text=f"{item.item_type.upper()} #{item.number}")
# Update nature text
self.nature_text.config(state='normal')
self.nature_text.delete('1.0', tk.END)
self.nature_text.insert('1.0', item.title)
self.nature_text.config(state='disabled')
# Update URL
self.doc_url_text.config(state='normal')
self.doc_url_text.delete('1.0', tk.END)
self.doc_url_text.insert('1.0', item.url)
self.doc_url_text.config(state='disabled')
# Update description
self.description_text.config(state='normal')
self.description_text.delete('1.0', tk.END)
self.description_text.insert('1.0', item.body or 'No description')
self.description_text.config(state='disabled')
self.logger.log(f"Displaying {item.item_type} #{item.number}: {item.title}")
def display_current_item(self): def display_current_item(self):
"""Display current work item (public method for compatibility)""" """Display current work item (public method for compatibility)"""
return self._display_current_item() return self._display_current_item()
+436 -92
View File
@@ -5,6 +5,7 @@ GUI for configuring application settings
import tkinter as tk import tkinter as tk
import threading import threading
import subprocess
from tkinter import ttk, messagebox, scrolledtext from tkinter import ttk, messagebox, scrolledtext
from typing import Dict, Any, Optional from typing import Dict, Any, Optional
import sys import sys
@@ -46,7 +47,7 @@ class SettingsDialog:
# Create tabs # Create tabs
self._create_general_tab(notebook) self._create_general_tab(notebook)
self._create_ai_tab(notebook) self._create_ai_tab(notebook)
self._create_dataverse_tab(notebook) # Removed: self._create_dataverse_tab(notebook) - Azure DevOps/Dataverse specific
# Buttons frame # Buttons frame
buttons_frame = ttk.Frame(main_frame) buttons_frame = ttk.Frame(main_frame)
@@ -84,15 +85,8 @@ class SettingsDialog:
current_row = 0 current_row = 0
# Azure DevOps section # REMOVED: Azure DevOps Configuration section
self._create_section_header(scrollable_frame, current_row, "🔷 Azure DevOps Configuration") # This was specific to Azure DevOps integration
current_row += 1
self._create_label_entry(scrollable_frame, current_row, "Query URL:", 'AZURE_DEVOPS_QUERY', width=60, multiline=True)
current_row += 1
self._create_label_entry(scrollable_frame, current_row, "Personal Access Token:", 'AZURE_DEVOPS_PAT', password=True, width=60)
current_row += 1
# GitHub section # GitHub section
self._create_section_header(scrollable_frame, current_row, "🐙 GitHub Configuration") self._create_section_header(scrollable_frame, current_row, "🐙 GitHub Configuration")
@@ -101,7 +95,7 @@ class SettingsDialog:
self._create_label_entry(scrollable_frame, current_row, "Personal Access Token:", 'GITHUB_PAT', password=True, width=60) self._create_label_entry(scrollable_frame, current_row, "Personal Access Token:", 'GITHUB_PAT', password=True, width=60)
current_row += 1 current_row += 1
self._create_label_entry(scrollable_frame, current_row, "Target Repository (owner/repo):", 'GITHUB_REPO', width=60) self._create_target_repo_dropdown(scrollable_frame, current_row)
current_row += 1 current_row += 1
self._create_forked_repo_dropdown(scrollable_frame, current_row) self._create_forked_repo_dropdown(scrollable_frame, current_row)
@@ -148,14 +142,13 @@ class SettingsDialog:
# Help text # Help text
help_text = ttk.Label(scrollable_frame, text="💡 Getting Started:\n" help_text = ttk.Label(scrollable_frame, text="💡 Getting Started:\n"
"1. Set your Azure DevOps Query URL (copy from browser)\n" "1. Create a GitHub Personal Access Token\n"
"2. Create Personal Access Tokens for both services\n" "2. Configure GitHub repositories:\n"
"3. Configure GitHub repositories:\n"
" • Target Repository: Where PRs will be created\n" " • Target Repository: Where PRs will be created\n"
" • Forked Repository: Your fork where changes are made\n" " • Forked Repository: Your fork where changes are made\n"
"4. Set Local Repo Path for automatic repository detection\n" "3. Set Local Repo Path for automatic repository detection\n"
"5. Configure AI provider in the AI tab (optional)\n" "4. Configure AI provider in the AI tab (optional)\n"
"6. Test your connection before processing items", "5. Test your connection before processing items",
font=('Arial', 9), foreground='blue', justify=tk.LEFT, wraplength=850) font=('Arial', 9), foreground='blue', justify=tk.LEFT, wraplength=850)
help_text.grid(row=current_row, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=(20, 30), padx=10) help_text.grid(row=current_row, column=0, columnspan=2, sticky=(tk.W, tk.E), pady=(20, 30), padx=10)
@@ -193,7 +186,7 @@ class SettingsDialog:
self.ai_provider_var = tk.StringVar(value=self.config.get('AI_PROVIDER', 'none')) self.ai_provider_var = tk.StringVar(value=self.config.get('AI_PROVIDER', 'none'))
provider_dropdown = ttk.Combobox(scrollable_frame, textvariable=self.ai_provider_var, provider_dropdown = ttk.Combobox(scrollable_frame, textvariable=self.ai_provider_var,
values=['none', 'claude', 'chatgpt', 'github-copilot'], state='readonly', width=47) values=['none', 'claude', 'chatgpt', 'github-copilot', 'ollama'], state='readonly', width=47)
provider_dropdown.grid(row=1, column=1, sticky=(tk.W, tk.E), pady=5, padx=10) provider_dropdown.grid(row=1, column=1, sticky=(tk.W, tk.E), pady=5, padx=10)
self.entries['AI_PROVIDER'] = self.ai_provider_var self.entries['AI_PROVIDER'] = self.ai_provider_var
@@ -202,6 +195,13 @@ class SettingsDialog:
self._create_label_entry(scrollable_frame, 3, "ChatGPT API Key:", 'OPENAI_API_KEY', password=True) self._create_label_entry(scrollable_frame, 3, "ChatGPT API Key:", 'OPENAI_API_KEY', password=True)
self._create_label_entry(scrollable_frame, 4, "GitHub Token (for Copilot) [defaults to GitHub PAT]:", 'GITHUB_TOKEN', password=True) self._create_label_entry(scrollable_frame, 4, "GitHub Token (for Copilot) [defaults to GitHub PAT]:", 'GITHUB_TOKEN', password=True)
# Ollama Configuration
self._create_label_entry(scrollable_frame, 5, "Ollama Server URL:", 'OLLAMA_URL')
self._create_label_entry(scrollable_frame, 6, "Ollama API Key (optional):", 'OLLAMA_API_KEY', password=True)
# Ollama Model Dropdown
self._create_ollama_model_dropdown(scrollable_frame, 7)
# Help text # Help text
help_text = ttk.Label(scrollable_frame, text="\n💡 Tips:\n" help_text = ttk.Label(scrollable_frame, text="\n💡 Tips:\n"
"• Provider: Choose 'none' to disable AI (uses Copilot workflow)\n" "• Provider: Choose 'none' to disable AI (uses Copilot workflow)\n"
@@ -209,58 +209,21 @@ class SettingsDialog:
"• ChatGPT: Get key at platform.openai.com/api-keys\n" "• ChatGPT: Get key at platform.openai.com/api-keys\n"
"• GitHub Copilot: Uses GitHub Models API (requires GitHub token)\n" "• GitHub Copilot: Uses GitHub Models API (requires GitHub token)\n"
"• GitHub Token: Auto-defaults to GitHub PAT if left empty\n" "• GitHub Token: Auto-defaults to GitHub PAT if left empty\n"
"Cost: ~$0.01-0.05 per PR with AI, free with 'none'\n" "Ollama: Self-hosted AI (requires Ollama server running)\n"
"• Cost: ~$0.01-0.05 per PR with AI, free with 'none' and Ollama\n"
"• AI providers clone repos locally to make changes before pushing", "• AI providers clone repos locally to make changes before pushing",
font=('Arial', 9), foreground='blue', justify=tk.LEFT, wraplength=800) font=('Arial', 9), foreground='blue', justify=tk.LEFT, wraplength=800)
help_text.grid(row=5, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=20, padx=10) help_text.grid(row=8, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=20, padx=10)
# Pack canvas and scrollbar # Pack canvas and scrollbar
canvas.pack(side="left", fill="both", expand=True) canvas.pack(side="left", fill="both", expand=True)
scrollbar.pack(side="right", fill="y") scrollbar.pack(side="right", fill="y")
def _create_dataverse_tab(self, notebook): # REMOVED: _create_dataverse_tab method
"""Create Dataverse/PowerApp settings tab""" # This was specific to Azure DevOps/Dataverse integration
dataverse_frame = ttk.Frame(notebook) # def _create_dataverse_tab(self, notebook):
notebook.add(dataverse_frame, text="UUF/Dataverse") # """Create Dataverse/PowerApp settings tab"""
# ...
# Scrollable frame
canvas = tk.Canvas(dataverse_frame)
scrollbar = ttk.Scrollbar(dataverse_frame, orient="vertical", command=canvas.yview)
scrollable_frame = ttk.Frame(canvas)
scrollable_frame.bind(
"<Configure>",
lambda e: canvas.configure(scrollregion=canvas.bbox("all"))
)
canvas.create_window((0, 0), window=scrollable_frame, anchor="nw")
canvas.configure(yscrollcommand=scrollbar.set)
# Dataverse section
self._create_section_header(scrollable_frame, 0, "📊 PowerApp/Dataverse Configuration")
self._create_label_entry(scrollable_frame, 1, "Environment URL:", 'DATAVERSE_ENVIRONMENT_URL', width=60, multiline=True)
self._create_label_entry(scrollable_frame, 2, "Table Name:", 'DATAVERSE_TABLE_NAME')
# Azure AD section
self._create_section_header(scrollable_frame, 3, "🔐 Azure AD Configuration")
self._create_label_entry(scrollable_frame, 4, "Client ID:", 'AZURE_AD_CLIENT_ID', width=60)
self._create_label_entry(scrollable_frame, 5, "Client Secret:", 'AZURE_AD_CLIENT_SECRET', password=True, width=60)
self._create_label_entry(scrollable_frame, 6, "Tenant ID:", 'AZURE_AD_TENANT_ID', width=60)
# Help text
help_text = ttk.Label(scrollable_frame, text="\n💡 UUF Integration:\n"
"• This section is only needed if you want to fetch UUF items\n"
"• UUF items are processed differently than Azure DevOps work items\n"
"• Environment URL: Your Dataverse environment\n"
"• Azure AD app must have appropriate permissions\n"
"• Contact your PowerApp administrator for these values\n"
"• Leave blank if not using UUF integration",
font=('Arial', 9), foreground='blue', justify=tk.LEFT, wraplength=800)
help_text.grid(row=7, column=0, columnspan=3, sticky=(tk.W, tk.E), pady=20, padx=10)
# Pack canvas and scrollbar
canvas.pack(side="left", fill="both", expand=True)
scrollbar.pack(side="right", fill="y")
def _create_section_header(self, parent, row: int, text: str): def _create_section_header(self, parent, row: int, text: str):
"""Create a section header""" """Create a section header"""
@@ -319,6 +282,48 @@ class SettingsDialog:
self.entries[config_key] = entry self.entries[config_key] = entry
parent.columnconfigure(1, weight=1) parent.columnconfigure(1, weight=1)
def _create_target_repo_dropdown(self, parent, row: int):
"""Create target repository dropdown with search functionality"""
ttk.Label(parent, text="Target Repository:", font=('Arial', 10, 'bold')).grid(
row=row, column=0, sticky=tk.W, pady=5, padx=10)
# Frame for dropdown, search entry, and buttons
dropdown_frame = ttk.Frame(parent)
dropdown_frame.grid(row=row, column=1, sticky=(tk.W, tk.E), pady=5, padx=10)
dropdown_frame.columnconfigure(0, weight=1)
# Placeholder for target repos
self.target_repos = []
# Combobox for target repo (searchable)
self.target_repo_var = tk.StringVar(value=self.config.get('GITHUB_REPO', ''))
self.target_repo_dropdown = ttk.Combobox(dropdown_frame, textvariable=self.target_repo_var,
values=[''], width=50)
self.target_repo_dropdown.grid(row=0, column=0, sticky=(tk.W, tk.E), padx=(0, 5))
self.entries['GITHUB_REPO'] = self.target_repo_var
# Bind typing event for search
self.target_repo_dropdown.bind('<KeyRelease>', self._on_target_repo_search)
# Refresh button
refresh_btn = ttk.Button(dropdown_frame, text="🔄", width=3,
command=self._refresh_target_repos)
refresh_btn.grid(row=0, column=1, padx=(0, 2))
# Search button
search_btn = ttk.Button(dropdown_frame, text="🔍", width=3,
command=self._search_target_repos)
search_btn.grid(row=0, column=2)
# Help text for target repo
help_label = ttk.Label(parent,
text=" ️ Upstream repo where PRs will be created. Type to search all GitHub repos.",
font=('Arial', 9), foreground='gray')
help_label.grid(row=row+1, column=0, columnspan=3, sticky=tk.W, padx=10)
# Start async loading of user's repos with edit access
self.dialog.after(100, self._load_target_repos_async)
def _create_forked_repo_dropdown(self, parent, row: int): def _create_forked_repo_dropdown(self, parent, row: int):
"""Create forked repository dropdown with local repo detection""" """Create forked repository dropdown with local repo detection"""
ttk.Label(parent, text="Forked Repository:", font=('Arial', 10, 'bold')).grid( ttk.Label(parent, text="Forked Repository:", font=('Arial', 10, 'bold')).grid(
@@ -356,7 +361,12 @@ class SettingsDialog:
# Refresh button # Refresh button
refresh_btn = ttk.Button(dropdown_frame, text="🔄", width=3, refresh_btn = ttk.Button(dropdown_frame, text="🔄", width=3,
command=self._refresh_forked_repos) command=self._refresh_forked_repos)
refresh_btn.grid(row=0, column=1) refresh_btn.grid(row=0, column=1, padx=(0, 2))
# Clone button
clone_btn = ttk.Button(dropdown_frame, text="📥", width=3,
command=self._clone_forked_repo)
clone_btn.grid(row=0, column=2)
# Help text for forked repo # Help text for forked repo
help_label = ttk.Label(parent, help_label = ttk.Label(parent,
@@ -405,6 +415,127 @@ class SettingsDialog:
except Exception as e: except Exception as e:
print(f"Error refreshing local repos: {e}") print(f"Error refreshing local repos: {e}")
def _clone_forked_repo(self):
"""Clone the selected forked repository to the local repo path"""
# Get selected repository
selected_repo = self.forked_repo_var.get().strip()
# Validate selection
if not selected_repo:
messagebox.showwarning("No Repository Selected",
"Please select a repository to clone.")
return
# Check if it's a section header
if selected_repo.startswith('---'):
messagebox.showwarning("Invalid Selection",
"Please select a repository, not a section header.")
return
# Get local repo path
local_repo_path = self.config.get('LOCAL_REPO_PATH', '').strip()
if not local_repo_path:
messagebox.showwarning("Local Path Not Configured",
"Please configure the Local Repository Path in settings first.")
return
# Create directory if it doesn't exist
try:
os.makedirs(local_repo_path, exist_ok=True)
except Exception as e:
messagebox.showerror("Directory Error",
f"Could not create local repository directory:\n{str(e)}")
return
# Extract repo name (handle both "owner/repo" and URLs)
repo_name = selected_repo
if repo_name.startswith('http'):
# Extract from URL
parts = repo_name.rstrip('/').split('/')
if len(parts) >= 2:
repo_name = f"{parts[-2]}/{parts[-1]}"
else:
messagebox.showerror("Invalid Repository",
"Could not parse repository name from URL.")
return
# Validate format "owner/repo"
if '/' not in repo_name:
messagebox.showerror("Invalid Repository",
"Repository must be in 'owner/repo' format.")
return
# Extract just the repo name for the folder
folder_name = repo_name.split('/')[-1]
target_path = os.path.join(local_repo_path, folder_name)
# Check if directory already exists
if os.path.exists(target_path):
response = messagebox.askyesno("Directory Exists",
f"The directory '{folder_name}' already exists.\n\n"
f"Do you want to continue anyway?\n"
f"(This may fail if it's already a git repository)")
if not response:
return
# Construct clone URL
clone_url = f"https://github.com/{repo_name}.git"
# Clone in background thread
def clone_repo():
try:
# Run git clone
result = subprocess.run(
['git', 'clone', clone_url, target_path],
capture_output=True,
text=True,
timeout=300 # 5 minute timeout
)
# Update UI on main thread
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
self.dialog.after(0, lambda: self._handle_clone_result(result, repo_name, folder_name))
except subprocess.TimeoutExpired:
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
self.dialog.after(0, lambda: messagebox.showerror(
"Clone Timeout",
f"Cloning {repo_name} timed out after 5 minutes."))
except FileNotFoundError:
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
self.dialog.after(0, lambda: messagebox.showerror(
"Git Not Found",
"Git is not installed or not found in PATH.\n\n"
"Please install Git from: https://git-scm.com/downloads"))
except Exception as e:
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
self.dialog.after(0, lambda: messagebox.showerror(
"Clone Error",
f"An error occurred while cloning:\n{str(e)}"))
# Show progress message
messagebox.showinfo("Cloning Repository",
f"Cloning {repo_name} to:\n{target_path}\n\n"
f"This may take a few moments...")
# Start clone in background
thread = threading.Thread(target=clone_repo, daemon=True)
thread.start()
def _handle_clone_result(self, result, repo_name: str, folder_name: str):
"""Handle the result of a git clone operation"""
if result.returncode == 0:
messagebox.showinfo("Clone Successful",
f"Successfully cloned {repo_name}!\n\n"
f"Location: {folder_name}/")
# Refresh the dropdown to show the newly cloned repo
self._refresh_forked_repos()
else:
error_msg = result.stderr if result.stderr else result.stdout
messagebox.showerror("Clone Failed",
f"Failed to clone {repo_name}.\n\n"
f"Error:\n{error_msg}")
def _load_user_forks_async(self): def _load_user_forks_async(self):
"""Load user's GitHub forks asynchronously""" """Load user's GitHub forks asynchronously"""
def load_forks(): def load_forks():
@@ -413,9 +544,10 @@ class SettingsDialog:
if not github_token: if not github_token:
return return
from .github_api import GitHubGQL from .workflow import GitHubRepoFetcher
github_api = GitHubGQL(github_token, dry_run=False) repo_fetcher = GitHubRepoFetcher(github_token)
self.forked_repos = github_api.get_user_forks() repos = repo_fetcher.fetch_user_repos(repo_type='owner')
self.forked_repos = repo_fetcher.get_repo_names(repos)
# Update dropdown on main thread # Update dropdown on main thread
if hasattr(self, 'dialog') and self.dialog.winfo_exists(): if hasattr(self, 'dialog') and self.dialog.winfo_exists():
@@ -437,14 +569,14 @@ class SettingsDialog:
current_values = list(self.forked_repo_dropdown['values']) current_values = list(self.forked_repo_dropdown['values'])
# Remove old GitHub forks section if exists # Remove old GitHub repos section if exists
if '--- Your GitHub Forks ---' in current_values: if '--- Your GitHub Repos ---' in current_values:
start_idx = current_values.index('--- Your GitHub Forks ---') start_idx = current_values.index('--- Your GitHub Repos ---')
current_values = current_values[:start_idx] current_values = current_values[:start_idx]
# Add GitHub forks section # Add GitHub repos section
if self.forked_repos: if self.forked_repos:
current_values.append('--- Your GitHub Forks ---') current_values.append('--- Your GitHub Repos ---')
current_values.extend(self.forked_repos) current_values.extend(self.forked_repos)
self.forked_repo_dropdown['values'] = current_values self.forked_repo_dropdown['values'] = current_values
@@ -452,6 +584,231 @@ class SettingsDialog:
except Exception as e: except Exception as e:
print(f"Error updating forked dropdown: {e}") print(f"Error updating forked dropdown: {e}")
def _load_target_repos_async(self):
"""Load target repos (with push/admin access) asynchronously"""
def load_repos():
try:
github_token = self.config.get('GITHUB_PAT', '')
if not github_token:
return
from .workflow import GitHubRepoFetcher
repo_fetcher = GitHubRepoFetcher(github_token)
repos = repo_fetcher.fetch_repos_with_permissions(min_permission='push')
self.target_repos = repo_fetcher.get_repo_names(repos)
# Update dropdown on main thread
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
self.dialog.after(0, self._update_target_dropdown)
except Exception as e:
print(f"Error loading target repos: {e}")
threading.Thread(target=load_repos, daemon=True).start()
def _update_target_dropdown(self):
"""Update the target repository dropdown"""
try:
if not hasattr(self, 'dialog') or not self.dialog.winfo_exists():
return
if not hasattr(self, 'target_repo_dropdown') or not self.target_repo_dropdown.winfo_exists():
return
current_values = [''] # Start with empty option
# Add user's repos with edit access
if self.target_repos:
current_values.append('--- Your Repos (with edit access) ---')
current_values.extend(self.target_repos)
self.target_repo_dropdown['values'] = current_values
except Exception as e:
print(f"Error updating target dropdown: {e}")
def _refresh_target_repos(self):
"""Refresh target repositories"""
self._load_target_repos_async()
def _search_target_repos(self):
"""Search for repositories on GitHub"""
query = self.target_repo_var.get().strip()
if not query:
return
def search_repos():
try:
github_token = self.config.get('GITHUB_PAT', '')
if not github_token:
return
from .workflow import GitHubRepoFetcher
repo_fetcher = GitHubRepoFetcher(github_token)
repos = repo_fetcher.search_repositories(query, per_page=50)
search_results = repo_fetcher.get_repo_names(repos)
# Update dropdown on main thread
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
self.dialog.after(0, lambda: self._update_target_dropdown_with_search(search_results, query))
except Exception as e:
print(f"Error searching repos: {e}")
threading.Thread(target=search_repos, daemon=True).start()
def _update_target_dropdown_with_search(self, search_results, query):
"""Update target dropdown with search results"""
try:
if not hasattr(self, 'target_repo_dropdown') or not self.target_repo_dropdown.winfo_exists():
return
current_values = ['']
# Add user's repos
if self.target_repos:
current_values.append('--- Your Repos (with edit access) ---')
current_values.extend(self.target_repos)
# Add search results
if search_results:
current_values.append(f'--- Search Results for "{query}" ---')
current_values.extend(search_results)
self.target_repo_dropdown['values'] = current_values
except Exception as e:
print(f"Error updating target dropdown with search: {e}")
def _on_target_repo_search(self, _event):
"""Handle typing in target repo field for auto-search"""
# Debounce: only search after user stops typing for 500ms
if hasattr(self, '_search_timer'):
self.dialog.after_cancel(self._search_timer)
query = self.target_repo_var.get().strip()
if len(query) >= 3: # Only search if at least 3 characters
self._search_timer = self.dialog.after(500, self._search_target_repos)
def _create_ollama_model_dropdown(self, parent, row: int):
"""Create Ollama model dropdown with scan button"""
ttk.Label(parent, text="Ollama Model:", font=('Arial', 10, 'bold')).grid(
row=row, column=0, sticky=tk.W, pady=5, padx=10)
# Frame for dropdown and scan button
dropdown_frame = ttk.Frame(parent)
dropdown_frame.grid(row=row, column=1, sticky=(tk.W, tk.E), pady=5, padx=10)
dropdown_frame.columnconfigure(0, weight=1)
# Model dropdown
self.ollama_model_var = tk.StringVar(value=self.config.get('OLLAMA_MODEL', ''))
self.ollama_model_dropdown = ttk.Combobox(dropdown_frame, textvariable=self.ollama_model_var,
values=[''], width=47)
self.ollama_model_dropdown.grid(row=0, column=0, sticky=(tk.W, tk.E), padx=(0, 5))
self.entries['OLLAMA_MODEL'] = self.ollama_model_var
# Scan button
scan_btn = ttk.Button(dropdown_frame, text="🔍", width=3,
command=self._scan_ollama_models)
scan_btn.grid(row=0, column=1)
# Help text for Ollama model
help_label = ttk.Label(parent,
text=" ️ Click 🔍 to scan available models from your Ollama server.",
font=('Arial', 9), foreground='gray')
help_label.grid(row=row+1, column=0, columnspan=3, sticky=tk.W, padx=10)
def _scan_ollama_models(self):
"""Scan Ollama server for available models"""
ollama_url = self.entries.get('OLLAMA_URL').get().strip() if 'OLLAMA_URL' in self.entries else ''
if not ollama_url:
messagebox.showwarning("Ollama URL Required",
"Please enter the Ollama Server URL first.")
return
# Normalize URL
if not ollama_url.startswith('http'):
ollama_url = f"http://{ollama_url}"
# Scan in background thread
def scan_models():
try:
import requests
# Get API key if provided
ollama_api_key = self.entries.get('OLLAMA_API_KEY').get().strip() if 'OLLAMA_API_KEY' in self.entries else ''
headers = {}
if ollama_api_key:
headers['Authorization'] = f'Bearer {ollama_api_key}'
# Query Ollama API for models
response = requests.get(f"{ollama_url}/api/tags", headers=headers, timeout=10)
response.raise_for_status()
data = response.json()
models = data.get('models', [])
model_names = [model.get('name', '') for model in models if model.get('name')]
# Update UI on main thread
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
self.dialog.after(0, lambda: self._update_ollama_models(model_names))
except requests.exceptions.ConnectionError:
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
self.dialog.after(0, lambda: messagebox.showerror(
"Connection Error",
f"Could not connect to Ollama server at:\n{ollama_url}\n\n"
f"Make sure Ollama is running and the URL is correct."))
except requests.exceptions.Timeout:
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
self.dialog.after(0, lambda: messagebox.showerror(
"Timeout",
f"Connection to Ollama server timed out."))
except requests.exceptions.HTTPError as e:
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
if e.response.status_code == 401:
self.dialog.after(0, lambda: messagebox.showerror(
"Authentication Error",
"Invalid API key. Please check your Ollama API Key."))
else:
self.dialog.after(0, lambda: messagebox.showerror(
"HTTP Error",
f"Error from Ollama server:\n{e}"))
except Exception as e:
if hasattr(self, 'dialog') and self.dialog.winfo_exists():
self.dialog.after(0, lambda: messagebox.showerror(
"Scan Error",
f"An error occurred while scanning for models:\n{str(e)}"))
# Start scan in background
threading.Thread(target=scan_models, daemon=True).start()
def _update_ollama_models(self, model_names):
"""Update the Ollama model dropdown with scanned models"""
if not model_names:
messagebox.showinfo("No Models Found",
"No models found on the Ollama server.\n\n"
"Use 'ollama pull <model>' to download models.")
return
try:
if hasattr(self, 'ollama_model_dropdown') and self.ollama_model_dropdown.winfo_exists():
current_value = self.ollama_model_var.get()
self.ollama_model_dropdown['values'] = model_names
# Keep current selection if it's still in the list
if current_value not in model_names and model_names:
self.ollama_model_var.set(model_names[0])
messagebox.showinfo("Models Found",
f"Found {len(model_names)} model(s):\n\n" +
"\n".join(f"{name}" for name in model_names[:10]) +
(f"\n\n...and {len(model_names) - 10} more" if len(model_names) > 10 else ""))
except Exception as e:
print(f"Error updating Ollama models: {e}")
def _create_dry_run_checkbox(self, parent, row: int): def _create_dry_run_checkbox(self, parent, row: int):
"""Create dry run checkbox""" """Create dry run checkbox"""
self.dry_run_var = tk.BooleanVar() self.dry_run_var = tk.BooleanVar()
@@ -502,21 +859,8 @@ class SettingsDialog:
results = [] results = []
# Test Azure DevOps # REMOVED: Azure DevOps test connection
if config_values.get('AZURE_DEVOPS_QUERY') and config_values.get('AZURE_DEVOPS_PAT'): # This was specific to Azure DevOps integration
try:
# Try to import and test Azure DevOps API
from .azure_devops_api import AzureDevOpsAPI
api = AzureDevOpsAPI(config_values.get('AZURE_DEVOPS_PAT'))
# Basic connection test (this would need actual implementation)
results.append("Azure DevOps: ✅ Configuration looks valid")
except ImportError:
results.append("Azure DevOps: ⚠️ Configuration set (API module not available)")
except Exception as e:
results.append(f"Azure DevOps: ❌ Error - {str(e)}")
elif config_values.get('AZURE_DEVOPS_QUERY') or config_values.get('AZURE_DEVOPS_PAT'):
results.append("Azure DevOps: ⚠️ Incomplete configuration")
# Test GitHub # Test GitHub
if config_values.get('GITHUB_PAT'): if config_values.get('GITHUB_PAT'):
@@ -605,7 +949,7 @@ class SettingsDialog:
config_values = self._get_config_values() config_values = self._get_config_values()
# Validate required fields # Validate required fields
required_for_basic = ['AZURE_DEVOPS_QUERY', 'AZURE_DEVOPS_PAT', 'GITHUB_PAT'] required_for_basic = ['GITHUB_PAT']
missing_basic = [field for field in required_for_basic if not config_values.get(field)] missing_basic = [field for field in required_for_basic if not config_values.get(field)]
if missing_basic: if missing_basic:
@@ -613,7 +957,7 @@ class SettingsDialog:
"Missing Configuration", "Missing Configuration",
f"The following required fields are missing:\n\n" f"The following required fields are missing:\n\n"
f"{', '.join(missing_basic)}\n\n" f"{', '.join(missing_basic)}\n\n"
f"These are required for basic functionality." f"GitHub Personal Access Token is required for basic functionality."
) )
return return
@@ -686,7 +1030,7 @@ class SettingsDialog:
import os import os
# Create .env content # Create .env content
env_content = "# Azure DevOps to GitHub Tool Configuration\n" env_content = "# GitHub Pulse Configuration\n"
env_content += "# Generated by Settings Dialog\n\n" env_content += "# Generated by Settings Dialog\n\n"
# Add all configuration values # Add all configuration values
@@ -822,11 +1166,11 @@ class SettingsDialog:
self.dialog.destroy() self.dialog.destroy()
def _clear_cache(self): def _clear_cache(self):
"""Clear all cached work items""" """Clear all cached items"""
result = messagebox.askyesno( result = messagebox.askyesno(
"Clear Cache", "Clear Cache",
"Are you sure you want to clear all cached items?\n\n" "Are you sure you want to clear all cached items?\n\n"
"Cached work items and UUF items will be removed.\n" "All cached data will be removed.\n"
"The next time you open the app, it will auto-load fresh data." "The next time you open the app, it will auto-load fresh data."
) )
if result: if result:
+22 -97
View File
@@ -192,11 +192,11 @@ class GitHubInfoExtractor:
class WorkItemFieldExtractor: class WorkItemFieldExtractor:
"""Extracts and processes work item fields""" """Extracts and processes item fields (placeholder for future implementation)"""
@staticmethod @staticmethod
def extract_work_item_fields(work_item: Dict[str, Any]) -> Dict[str, Any]: def extract_work_item_fields(work_item: Dict[str, Any]) -> Dict[str, Any]:
"""Extract and process fields from Azure DevOps work item""" """Extract and process fields from work item (placeholder)"""
fields = work_item.get('fields', {}) fields = work_item.get('fields', {})
# Extract basic fields # Extract basic fields
@@ -243,12 +243,12 @@ class WorkItemFieldExtractor:
'new_text': new_text, 'new_text': new_text,
'github_info': github_info, 'github_info': github_info,
'status': 'Ready', 'status': 'Ready',
'source': 'Azure DevOps' 'source': 'Generic'
} }
@staticmethod @staticmethod
def extract_uuf_item_fields(uuf_item: Dict[str, Any]) -> Dict[str, Any]: def extract_uuf_item_fields(uuf_item: Dict[str, Any]) -> Dict[str, Any]:
"""Extract and process fields from UUF item""" """Extract and process fields from custom item (placeholder)"""
# UUF items have different field structure # UUF items have different field structure
item_id = uuf_item.get('cr_uufitemid', 'Unknown') item_id = uuf_item.get('cr_uufitemid', 'Unknown')
title = uuf_item.get('cr_title', 'No Title') title = uuf_item.get('cr_title', 'No Title')
@@ -296,8 +296,10 @@ class ContentBuilders:
@staticmethod @staticmethod
def build_issue_title(item: Dict[str, Any]) -> str: def build_issue_title(item: Dict[str, Any]) -> str:
"""Build GitHub issue title""" """Build GitHub issue title"""
source_prefix = "UUF" if item.get('source') == 'UUF' else "AB" item_id = item.get('id', '')
return f"[{source_prefix}#{item['id']}] {item['title']}" if item_id:
return f"[#{item_id}] {item['title']}"
return f"{item['title']}"
@staticmethod @staticmethod
def build_issue_body(item: Dict[str, Any], github_info: Dict[str, Any]) -> str: def build_issue_body(item: Dict[str, Any], github_info: Dict[str, Any]) -> str:
@@ -305,8 +307,7 @@ class ContentBuilders:
body_parts = [] body_parts = []
# Header # Header
source_name = "UUF Item" if item.get('source') == 'UUF' else "Azure DevOps Work Item" body_parts.append("## Item Details")
body_parts.append(f"## {source_name} Details")
body_parts.append("") body_parts.append("")
# Make ID a hyperlink if source URL is available # Make ID a hyperlink if source URL is available
@@ -372,15 +373,17 @@ class ContentBuilders:
body_parts.append("5. Close this issue when complete") body_parts.append("5. Close this issue when complete")
body_parts.append("") body_parts.append("")
body_parts.append("---") body_parts.append("---")
body_parts.append("*Created automatically by Azure DevOps → GitHub Processor*") body_parts.append("*Created automatically by GitHub Pulse*")
return "\n".join(body_parts) return "\n".join(body_parts)
@staticmethod @staticmethod
def build_pr_title(item: Dict[str, Any]) -> str: def build_pr_title(item: Dict[str, Any]) -> str:
"""Build GitHub PR title""" """Build GitHub PR title"""
source_prefix = "UUF" if item.get('source') == 'UUF' else "AB" item_id = item.get('id', '')
return f"[{source_prefix}#{item['id']}] {item['title']}" if item_id:
return f"[#{item_id}] {item['title']}"
return f"{item['title']}"
@staticmethod @staticmethod
def build_pr_body(item: Dict[str, Any], github_info: Dict[str, Any]) -> str: def build_pr_body(item: Dict[str, Any], github_info: Dict[str, Any]) -> str:
@@ -388,8 +391,7 @@ class ContentBuilders:
body_parts = [] body_parts = []
# Header # Header
source_name = "UUF Item" if item.get('source') == 'UUF' else "Azure DevOps Work Item" body_parts.append("## Documentation Update")
body_parts.append(f"## {source_name} Documentation Update")
body_parts.append("") body_parts.append("")
# Make ID a hyperlink if source URL is available # Make ID a hyperlink if source URL is available
@@ -448,7 +450,7 @@ class ContentBuilders:
body_parts.append("") body_parts.append("")
body_parts.append("---") body_parts.append("---")
body_parts.append("*Created automatically by Azure DevOps → GitHub Processor*") body_parts.append("*Created automatically by GitHub Pulse*")
return "\n".join(body_parts) return "\n".join(body_parts)
@@ -612,17 +614,14 @@ class ConfigurationHelpers:
def create_default_env_file() -> bool: def create_default_env_file() -> bool:
"""Create a default .env file with all settings blank""" """Create a default .env file with all settings blank"""
try: try:
default_config = """# Azure DevOps to GitHub Tool Configuration default_config = """# GitHub Pulse Configuration
# Generated automatically - fill in your values # Generated automatically - fill in your values
# IMPORTANT: Do NOT commit this file to source control. Add it to .gitignore. # IMPORTANT: Do NOT commit this file to source control. Add it to .gitignore.
# Azure DevOps Configuration
AZURE_DEVOPS_QUERY=
AZURE_DEVOPS_PAT=
# GitHub Configuration # GitHub Configuration
GITHUB_PAT= GITHUB_PAT=
GITHUB_REPO= GITHUB_REPO=
FORKED_REPO=
# Application Settings # Application Settings
DRY_RUN=false DRY_RUN=false
@@ -634,12 +633,8 @@ OPENAI_API_KEY=
GITHUB_TOKEN= GITHUB_TOKEN=
LOCAL_REPO_PATH= LOCAL_REPO_PATH=
# PowerApp/Dataverse Configuration (for UUF items - optional) # Custom AI Instructions (optional)
DATAVERSE_ENVIRONMENT_URL= CUSTOM_INSTRUCTIONS=
DATAVERSE_TABLE_NAME=
AZURE_AD_CLIENT_ID=
AZURE_AD_CLIENT_SECRET=
AZURE_AD_TENANT_ID=
""" """
with open('.env', 'w', encoding='utf-8') as f: with open('.env', 'w', encoding='utf-8') as f:
f.write(default_config) f.write(default_config)
@@ -652,78 +647,8 @@ AZURE_AD_TENANT_ID=
return False return False
class EnhancedContentBuilders(ContentBuilders): # Removed EnhancedContentBuilders class - was specific to Azure DevOps
"""Enhanced content builders with Azure DevOps specific methods""" # Use ContentBuilders class for generic GitHub automation instead
@staticmethod
def build_pr_title_for_azure_devops(item: Dict[str, Any]) -> str:
"""Build GitHub PR title for Azure DevOps items"""
return f"Docs update: {item['title'][:80]} (AB#{item['id']})"
@staticmethod
def build_pr_body_for_azure_devops(item: Dict[str, Any], github_info: Dict[str, Any]) -> str:
"""Build GitHub PR body for Azure DevOps items with enhanced Copilot instructions"""
now = datetime.datetime.utcnow().strftime("%Y-%m-%d %H:%M UTC")
lines = [
f"**Automated documentation update from Azure DevOps (created on {now})**",
"",
f"**Work Item ID:** AB#{item['id']}",
f"**Document URL:** {item['mydoc_url']}",
]
# Add file path information if available
if github_info.get('original_content_git_url'):
lines.append(f"**File Path:** {github_info['original_content_git_url']}")
# Add ms.author metadata if available
if github_info.get('ms_author'):
lines.append(f"**ms.author:** `{github_info['ms_author']}`")
# Add nature of request for context
lines.extend([
"",
"## Change Type",
f"{item['nature_of_request']}",
"",
])
lines.extend([
"## Changes Requested",
"",
"### Current Text to Replace",
"```",
item['text_to_change'],
"```",
"",
"### Proposed New Text",
"```",
item['new_text'],
"```",
"",
"---",
"",
"## Instructions for GitHub Copilot",
"",
"**Task:** Update the documentation file with the changes requested above.",
"",
"**Steps to complete:**",
"1. Locate the file containing the 'Current Text to Replace' shown above",
"2. Find the exact text that needs to be updated",
"3. Replace it with the 'Proposed New Text'",
"4. Ensure no other changes are made to the file",
"5. Commit the changes with a descriptive message",
"",
"**Important Notes:**",
"- Only change the specific text shown above",
"- Do not modify formatting, links, or other content",
"- Verify the replacement text fits naturally in context",
"",
"---",
"*This PR was created automatically from Azure DevOps work item AB#" + str(item['id']) + "*"
])
return "\n".join(lines)
# Compatibility functions for direct function access # Compatibility functions for direct function access
@@ -1,291 +0,0 @@
"""
Work Item Processor
Handles processing of Azure DevOps work items and UUF items
"""
import re
import html
import requests
from typing import Dict, Any, Optional, Tuple
from urllib.parse import urlparse
from .utils import WorkItemFieldExtractor
# User agent for web requests
USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
class WorkItemProcessor:
"""Processor for extracting and validating work item data with advanced parsing"""
def __init__(self, logger, config: Dict[str, Any] = None):
self.logger = logger
self.log = logger.log if hasattr(logger, 'log') else logger
self.config = config or {}
def process_work_item(self, work_item: Dict[str, Any]) -> Optional[Dict[str, Any]]:
"""Process a single work item to extract required fields with advanced validation"""
try:
work_item_id = work_item['id']
title = work_item.get('fields', {}).get('System.Title', 'No Title')
description = work_item.get('fields', {}).get('System.Description', '')
if not description:
self.log(f"Work item {work_item_id} has no description, skipping")
return None
# Parse description for required fields
parsed_data = self._parse_description(description)
if not parsed_data:
self.log(f"Work item {work_item_id} doesn't contain required fields, skipping")
return None
# Validate nature of request (check for both variations)
nature_lower = parsed_data['nature_of_request'].lower()
if not ("modify existing docs" in nature_lower or "modifying existing docs" in nature_lower):
self.log(f"Work item {work_item_id} nature of request doesn't contain 'modify existing docs', skipping")
return None
# Extract GitHub info from document URL
github_info = self._extract_github_info(parsed_data['mydoc_url'])
# If the document does not include an original_content_git_url, skip this work item
if not github_info.get('original_content_git_url'):
self.log(f"Work item {work_item_id} skipped: original_content_git_url not found in document {parsed_data['mydoc_url']}")
return None
# Construct proper web URL for work item
# The API returns something like: https://dev.azure.com/org/project/_apis/wit/workItems/123
# We need to convert it to: https://dev.azure.com/org/project/_workitems/edit/123
work_item_url = ''
api_url = work_item.get('url', '')
if api_url:
# Convert API URL to web URL
# Replace /_apis/wit/workItems/ with /_workitems/edit/
work_item_url = api_url.replace('/_apis/wit/workItems/', '/_workitems/edit/')
processed_item = {
'id': work_item_id,
'title': title,
'nature_of_request': parsed_data['nature_of_request'],
'mydoc_url': parsed_data['mydoc_url'],
'text_to_change': parsed_data['text_to_change'],
'new_text': parsed_data['new_text'],
'github_info': github_info,
'status': 'Ready',
'source': 'Azure DevOps',
'source_url': work_item_url, # URL to Azure DevOps work item
'original_new_text': parsed_data['new_text'] # Keep original for reference
}
self.log(f"Successfully processed work item {work_item_id}")
return processed_item
except Exception as e:
self.log(f"Error processing work item {work_item.get('id', 'unknown')}: {str(e)}")
return None
def process_uuf_item(self, uuf_item: Dict[str, Any]) -> Optional[Dict[str, Any]]:
"""Process a single UUF item from Dataverse/PowerApp with enhanced field mapping"""
try:
# Extract UUF item ID (adjust field name as needed)
uuf_id = uuf_item.get('cr4af_uufid') or uuf_item.get('cr4af_name') or 'unknown'
# Extract title
title = uuf_item.get('cr4af_title') or uuf_item.get('cr4af_subject') or 'No Title'
# Extract description/details
description = uuf_item.get('cr4af_description') or uuf_item.get('cr4af_details') or ''
if not description:
self.log(f"UUF item {uuf_id} has no description, skipping")
return None
# Extract document URL
doc_url = uuf_item.get('cr4af_documenturl') or uuf_item.get('cr4af_docurl') or ''
if not doc_url:
self.log(f"UUF item {uuf_id} has no document URL, skipping")
return None
# Extract text to change and new text
text_to_change = uuf_item.get('cr4af_texttochange') or uuf_item.get('cr4af_currenttext') or ''
new_text = uuf_item.get('cr4af_proposednewtext') or uuf_item.get('cr4af_newtext') or ''
if not text_to_change or not new_text:
self.log(f"UUF item {uuf_id} missing text fields, skipping")
return None
# Extract GitHub info from document URL
github_info = self._extract_github_info(doc_url)
# If the document does not include an original_content_git_url, skip this item
if not github_info.get('original_content_git_url'):
self.log(f"UUF item {uuf_id} skipped: original_content_git_url not found in document {doc_url}")
return None
# Get UUF item URL if available (e.g., from Dataverse)
uuf_url = uuf_item.get('cr4af_itemurl', '') or uuf_item.get('cr4af_url', '')
processed_item = {
'id': uuf_id,
'title': title,
'nature_of_request': 'UUF Item - Modify existing docs',
'mydoc_url': doc_url,
'text_to_change': text_to_change,
'new_text': new_text,
'github_info': github_info,
'status': 'Ready',
'source': 'UUF', # Mark as UUF item
'source_url': uuf_url, # URL to UUF item (if available)
'original_new_text': new_text
}
self.log(f"Successfully processed UUF item {uuf_id}")
return processed_item
except Exception as e:
self.log(f"Error processing UUF item {uuf_item.get('cr4af_uufid', 'unknown')}: {str(e)}")
return None
def _parse_description(self, description: str) -> Optional[Dict[str, Any]]:
"""Parse work item description to extract required fields using enhanced regex patterns"""
# Enhanced regex patterns from regex_V5
patterns = {
'nature_of_request': r'nature\s+of\s+request[:\s]*([^\)]*\))',
'link_to_doc': r'link\s+to\s+doc[:\s]*([^\s&]+)',
'text_to_change': r'text\s+to\s+change[:\s]*([\s\S]*?)(?=\n*-+\s*Proposed new text|If adding brand new docs:|$)',
'proposed_new_text': r'proposed\s+new\s+text[:\s]*([\s\S]+?)(?=\s*If\s+adding\s+brand\s+new\s+docs:)'
}
# Clean HTML tags if present
clean_description = re.sub(r'<[^>]+>', '', description)
# Convert HTML entities to characters (e.g., &quot; to ", &amp; to &)
clean_description = html.unescape(clean_description)
extracted = {}
for field, pattern in patterns.items():
match = re.search(pattern, clean_description, re.IGNORECASE | re.DOTALL)
if match:
value = match.group(1).strip()
if field == 'nature_of_request':
extracted['nature_of_request'] = value
elif field == 'link_to_doc':
extracted['mydoc_url'] = value.rstrip('-')
elif field == 'text_to_change':
extracted['text_to_change'] = value
elif field == 'proposed_new_text':
extracted['new_text'] = value
# If enhanced patterns don't work, fall back to basic patterns
if not all(field in extracted for field in ['nature_of_request', 'mydoc_url', 'text_to_change', 'new_text']):
basic_patterns = {
'nature_of_request': r'nature\s+of\s+request[:\s]*([^\n]+)',
'link_to_doc': r'link\s+to\s+doc[:\s]*([^\s]+)',
'text_to_change': r'text\s+to\s+change[:\s]*(.+?)(?=proposed\s+new\s+text|$)',
'proposed_new_text': r'proposed\s+new\s+text[:\s]*(.+?)(?=\n\n|$)'
}
extracted = {}
for field, pattern in basic_patterns.items():
match = re.search(pattern, clean_description, re.IGNORECASE | re.DOTALL)
if match:
value = match.group(1).strip()
if field == 'nature_of_request':
extracted['nature_of_request'] = value
elif field == 'link_to_doc':
extracted['mydoc_url'] = value
elif field == 'text_to_change':
extracted['text_to_change'] = value
elif field == 'proposed_new_text':
extracted['new_text'] = value
# Validate all required fields are present
required_fields = ['nature_of_request', 'mydoc_url', 'text_to_change', 'new_text']
if not all(field in extracted for field in required_fields):
return None
return extracted
def _extract_github_info(self, doc_url: str) -> Dict[str, Any]:
"""Extract GitHub repository info and ms.author from document URL
If GITHUB_REPO is configured in .env, it will be used instead of the repo
extracted from the document metadata. This allows you to create PRs in your
fork while preserving the file path and ms.author from the original document.
"""
try:
# Fetch the document
headers = {'User-Agent': USER_AGENT}
response = requests.get(doc_url, headers=headers, timeout=30)
response.raise_for_status()
html_content = response.text
# Extract ms.author
ms_author = self._extract_meta_tag(html_content, 'ms.author')
# Extract original_content_git_url
original_content_git_url = self._extract_meta_tag(html_content, 'original_content_git_url')
if not original_content_git_url:
# Try alternative extraction method
match = re.search(r"original_content_git_url[\"\']?\s*[:=]\s*[\"\']([^\"']+)[\"']", html_content, re.IGNORECASE)
if match:
original_content_git_url = match.group(1).strip()
if not original_content_git_url:
raise ValueError("original_content_git_url not found in document")
# Check if GITHUB_REPO is configured in .env
# If it is, use that instead of the repo from the document
configured_repo = self.config.get('GITHUB_REPO')
if configured_repo and '/' in configured_repo:
# Use the configured repository (e.g., "b-tsammons/fabric-docs-pr")
parts = configured_repo.split('/', 1)
owner = parts[0].strip()
repo = parts[1].strip()
self.log(f"Using configured GITHUB_REPO: {owner}/{repo} (overriding document metadata)")
else:
# Parse GitHub owner/repo from original_content_git_url (fallback to document metadata)
owner, repo = self._parse_github_url(original_content_git_url)
self.log(f"Using repository from document metadata: {owner}/{repo}")
return {
'ms_author': ms_author,
'original_content_git_url': original_content_git_url,
'owner': owner,
'repo': repo
}
except Exception as e:
self.log(f"Error extracting GitHub info from {doc_url}: {str(e)}")
return {
'ms_author': None,
'original_content_git_url': None,
'owner': None,
'repo': None,
'error': str(e)
}
def _extract_meta_tag(self, html_content: str, name: str) -> Optional[str]:
"""Extract content from meta tag"""
pattern = rf'<meta\s+(?:[^>]*?\s)?(?:name|property)\s*=\s*["\'](?P<n>{re.escape(name)})["\']\s+[^>]*?\bcontent\s*=\s*["\'](?P<content>[^"\']+)["\'][^>]*?>'
match = re.search(pattern, html_content, re.IGNORECASE)
if match:
return match.group('content').strip()
return None
def _parse_github_url(self, url: str) -> Tuple[str, str]:
"""Parse GitHub URL to extract owner and repo"""
parsed = urlparse(url)
if "github.com" not in parsed.netloc.lower():
raise ValueError(f"Not a GitHub URL: {url}")
parts = [p for p in parsed.path.split("/") if p]
if len(parts) < 2:
raise ValueError(f"Unable to parse owner/repo from: {url}")
return parts[0], parts[1]
+518
View File
@@ -0,0 +1,518 @@
"""
Workflow Manager
Manages GitHub workflow items (Issues and Pull Requests) from target and fork repositories
"""
import requests
from typing import List, Dict, Any, Optional, Tuple
class WorkflowItem:
"""Represents a GitHub workflow item (Issue or PR)"""
def __init__(self, item_type: str, data: Dict[str, Any], repo_source: str):
"""
Initialize a workflow item
Args:
item_type: 'issue' or 'pull_request'
data: Raw data from GitHub API
repo_source: 'target' or 'fork'
"""
self.item_type = item_type
self.repo_source = repo_source
self.data = data
# Extract common fields
self.number = data.get('number')
self.title = data.get('title', 'No Title')
self.state = data.get('state', 'unknown')
self.created_at = data.get('created_at', '')
self.updated_at = data.get('updated_at', '')
self.body = data.get('body', '')
self.url = data.get('html_url', '')
self.api_url = data.get('url', '')
# Author information
user = data.get('user', {})
self.author = user.get('login', 'unknown') if user else 'unknown'
self.author_url = user.get('html_url', '') if user else ''
# Labels
self.labels = [label.get('name', '') for label in data.get('labels', [])]
# Assignees
assignees = data.get('assignees', [])
self.assignees = [a.get('login', '') for a in assignees if a]
# PR-specific fields
if item_type == 'pull_request':
self.is_draft = data.get('draft', False)
self.mergeable_state = data.get('mergeable_state', 'unknown')
self.merged = data.get('merged', False)
self.base_ref = data.get('base', {}).get('ref', '')
self.head_ref = data.get('head', {}).get('ref', '')
else:
self.is_draft = False
self.mergeable_state = None
self.merged = False
self.base_ref = None
self.head_ref = None
# Comments count
self.comments_count = data.get('comments', 0)
def __repr__(self):
return f"<WorkflowItem {self.item_type} #{self.number}: {self.title[:50]}>"
def to_dict(self) -> Dict[str, Any]:
"""Convert to dictionary for easy serialization"""
return {
'item_type': self.item_type,
'repo_source': self.repo_source,
'number': self.number,
'title': self.title,
'state': self.state,
'created_at': self.created_at,
'updated_at': self.updated_at,
'body': self.body,
'url': self.url,
'author': self.author,
'author_url': self.author_url,
'labels': self.labels,
'assignees': self.assignees,
'is_draft': self.is_draft,
'mergeable_state': self.mergeable_state,
'merged': self.merged,
'base_ref': self.base_ref,
'head_ref': self.head_ref,
'comments_count': self.comments_count
}
class GitHubRepoFetcher:
"""Fetches repository information from GitHub"""
def __init__(self, github_token: str, logger=None):
"""
Initialize the repo fetcher
Args:
github_token: GitHub Personal Access Token
logger: Optional logger instance
"""
self.token = github_token
self.logger = logger
self.headers = {
"Authorization": f"Bearer {github_token}",
"Accept": "application/vnd.github+json",
"User-Agent": "github-automation-tool/1.0"
}
def log(self, message: str):
"""Log a message"""
if self.logger:
self.logger.log(message)
else:
print(message)
def get_authenticated_user(self) -> Optional[Dict[str, Any]]:
"""
Get information about the authenticated user
Returns:
Dictionary with user information or None if error
"""
try:
url = "https://api.github.com/user"
response = requests.get(url, headers=self.headers, timeout=30)
response.raise_for_status()
return response.json()
except Exception as e:
self.log(f"❌ Error fetching authenticated user: {str(e)}")
return None
def fetch_user_repos(self, repo_type: str = 'owner', per_page: int = 100) -> List[Dict[str, Any]]:
"""
Fetch repositories for the authenticated user
Args:
repo_type: 'owner', 'member', or 'all'
per_page: Number of repos per page (max 100)
Returns:
List of repository dictionaries
"""
try:
url = "https://api.github.com/user/repos"
params = {
'type': repo_type,
'per_page': min(per_page, 100),
'sort': 'updated',
'direction': 'desc'
}
response = requests.get(url, headers=self.headers, params=params, timeout=30)
response.raise_for_status()
repos = response.json()
self.log(f"✅ Found {len(repos)} repositories ({repo_type})")
return repos
except Exception as e:
self.log(f"❌ Error fetching user repos: {str(e)}")
return []
def fetch_repos_with_permissions(self, min_permission: str = 'push') -> List[Dict[str, Any]]:
"""
Fetch repositories where user has specific permissions
Args:
min_permission: Minimum permission level ('pull', 'push', 'admin')
Returns:
List of repository dictionaries with sufficient permissions
"""
try:
# Fetch all repos user has access to
all_repos = self.fetch_user_repos(repo_type='all')
# Filter by permission level
filtered_repos = []
permission_levels = {'pull': 0, 'push': 1, 'admin': 2}
min_level = permission_levels.get(min_permission, 1)
for repo in all_repos:
permissions = repo.get('permissions', {})
# Check permission level
if permissions.get('admin'):
level = 2
elif permissions.get('push'):
level = 1
elif permissions.get('pull'):
level = 0
else:
level = -1
if level >= min_level:
filtered_repos.append(repo)
self.log(f"✅ Found {len(filtered_repos)} repos with '{min_permission}' permission or higher")
return filtered_repos
except Exception as e:
self.log(f"❌ Error fetching repos with permissions: {str(e)}")
return []
def search_repositories(self, query: str, per_page: int = 30) -> List[Dict[str, Any]]:
"""
Search for repositories on GitHub
Args:
query: Search query string
per_page: Number of results per page (max 100)
Returns:
List of repository dictionaries
"""
if not query or not query.strip():
return []
try:
url = "https://api.github.com/search/repositories"
params = {
'q': query.strip(),
'per_page': min(per_page, 100),
'sort': 'updated',
'order': 'desc'
}
response = requests.get(url, headers=self.headers, params=params, timeout=30)
response.raise_for_status()
data = response.json()
repos = data.get('items', [])
total_count = data.get('total_count', 0)
self.log(f"✅ Search found {total_count} repositories (showing {len(repos)})")
return repos
except Exception as e:
self.log(f"❌ Error searching repositories: {str(e)}")
return []
def get_repo_names(self, repos: List[Dict[str, Any]]) -> List[str]:
"""
Extract repository names in 'owner/repo' format
Args:
repos: List of repository dictionaries
Returns:
List of repository name strings
"""
return [repo.get('full_name', '') for repo in repos if repo.get('full_name')]
class WorkflowManager:
"""Manages workflow items from GitHub repositories"""
def __init__(self, github_token: str, logger=None):
"""
Initialize the workflow manager
Args:
github_token: GitHub Personal Access Token
logger: Optional logger instance
"""
self.token = github_token
self.logger = logger
self.headers = {
"Authorization": f"Bearer {github_token}",
"Accept": "application/vnd.github+json",
"User-Agent": "github-automation-tool/1.0"
}
# Initialize repo fetcher
self.repo_fetcher = GitHubRepoFetcher(github_token, logger)
def log(self, message: str):
"""Log a message"""
if self.logger:
self.logger.log(message)
else:
print(message)
def _parse_repo(self, repo_str: str) -> Optional[Tuple[str, str]]:
"""
Parse a repository string into owner and name
Args:
repo_str: Repository string in format "owner/repo"
Returns:
Tuple of (owner, repo) or None if invalid
"""
if not repo_str or '/' not in repo_str:
return None
parts = repo_str.strip().split('/')
if len(parts) != 2:
return None
return parts[0], parts[1]
def fetch_issues(self, repo_str: str, repo_source: str = 'target',
state: str = 'all', per_page: int = 100) -> List[WorkflowItem]:
"""
Fetch issues from a repository
Args:
repo_str: Repository string in format "owner/repo"
repo_source: 'target' or 'fork' to identify source
state: 'open', 'closed', or 'all'
per_page: Number of items per page (max 100)
Returns:
List of WorkflowItem objects
"""
parsed = self._parse_repo(repo_str)
if not parsed:
self.log(f"L Invalid repository format: {repo_str}")
return []
owner, repo = parsed
self.log(f"Fetching issues from {owner}/{repo} ({repo_source})...")
try:
url = f"https://api.github.com/repos/{owner}/{repo}/issues"
params = {
'state': state,
'per_page': min(per_page, 100),
'sort': 'updated',
'direction': 'desc'
}
response = requests.get(url, headers=self.headers, params=params, timeout=30)
response.raise_for_status()
items_data = response.json()
# Filter out pull requests (GitHub's issues endpoint includes PRs)
issues_data = [item for item in items_data if 'pull_request' not in item]
issues = [WorkflowItem('issue', data, repo_source) for data in issues_data]
self.log(f" Found {len(issues)} issues in {owner}/{repo}")
return issues
except requests.HTTPError as e:
self.log(f"L HTTP Error fetching issues from {owner}/{repo}: {e}")
if e.response.status_code == 401:
self.log(" Check your GitHub Personal Access Token")
elif e.response.status_code == 404:
self.log(" Repository not found or no access")
return []
except Exception as e:
self.log(f"L Error fetching issues from {owner}/{repo}: {str(e)}")
return []
def fetch_pull_requests(self, repo_str: str, repo_source: str = 'target',
state: str = 'all', per_page: int = 100) -> List[WorkflowItem]:
"""
Fetch pull requests from a repository
Args:
repo_str: Repository string in format "owner/repo"
repo_source: 'target' or 'fork' to identify source
state: 'open', 'closed', or 'all'
per_page: Number of items per page (max 100)
Returns:
List of WorkflowItem objects
"""
parsed = self._parse_repo(repo_str)
if not parsed:
self.log(f"L Invalid repository format: {repo_str}")
return []
owner, repo = parsed
self.log(f"Fetching pull requests from {owner}/{repo} ({repo_source})...")
try:
url = f"https://api.github.com/repos/{owner}/{repo}/pulls"
params = {
'state': state,
'per_page': min(per_page, 100),
'sort': 'updated',
'direction': 'desc'
}
response = requests.get(url, headers=self.headers, params=params, timeout=30)
response.raise_for_status()
prs_data = response.json()
prs = [WorkflowItem('pull_request', data, repo_source) for data in prs_data]
self.log(f" Found {len(prs)} pull requests in {owner}/{repo}")
return prs
except requests.HTTPError as e:
self.log(f"L HTTP Error fetching PRs from {owner}/{repo}: {e}")
if e.response.status_code == 401:
self.log(" Check your GitHub Personal Access Token")
elif e.response.status_code == 404:
self.log(" Repository not found or no access")
return []
except Exception as e:
self.log(f"L Error fetching PRs from {owner}/{repo}: {str(e)}")
return []
def fetch_all_workflow_items(self, target_repo: str, fork_repo: str = None,
include_issues: bool = True,
include_prs: bool = True,
state: str = 'all') -> Dict[str, List[WorkflowItem]]:
"""
Fetch all workflow items from both target and fork repositories
Args:
target_repo: Target repository string "owner/repo"
fork_repo: Fork repository string "owner/repo" (optional)
include_issues: Whether to fetch issues
include_prs: Whether to fetch pull requests
state: 'open', 'closed', or 'all'
Returns:
Dictionary with keys 'target_issues', 'target_prs', 'fork_issues', 'fork_prs'
"""
results = {
'target_issues': [],
'target_prs': [],
'fork_issues': [],
'fork_prs': []
}
# Fetch from target repository
if target_repo:
if include_issues:
results['target_issues'] = self.fetch_issues(target_repo, 'target', state)
if include_prs:
results['target_prs'] = self.fetch_pull_requests(target_repo, 'target', state)
# Fetch from fork repository
if fork_repo:
if include_issues:
results['fork_issues'] = self.fetch_issues(fork_repo, 'fork', state)
if include_prs:
results['fork_prs'] = self.fetch_pull_requests(fork_repo, 'fork', state)
# Log summary
total = sum(len(items) for items in results.values())
self.log(f"\n= Summary: Fetched {total} total items")
self.log(f" Target Issues: {len(results['target_issues'])}")
self.log(f" Target PRs: {len(results['target_prs'])}")
if fork_repo:
self.log(f" Fork Issues: {len(results['fork_issues'])}")
self.log(f" Fork PRs: {len(results['fork_prs'])}")
return results
def get_combined_items(self, workflow_items: Dict[str, List[WorkflowItem]],
sort_by: str = 'updated') -> List[WorkflowItem]:
"""
Combine and sort all workflow items
Args:
workflow_items: Dictionary from fetch_all_workflow_items()
sort_by: 'updated', 'created', or 'number'
Returns:
Sorted list of all workflow items
"""
all_items = []
for items_list in workflow_items.values():
all_items.extend(items_list)
# Sort items
if sort_by == 'updated':
all_items.sort(key=lambda x: x.updated_at, reverse=True)
elif sort_by == 'created':
all_items.sort(key=lambda x: x.created_at, reverse=True)
elif sort_by == 'number':
all_items.sort(key=lambda x: x.number, reverse=True)
return all_items
def filter_items(self, items: List[WorkflowItem], **filters) -> List[WorkflowItem]:
"""
Filter workflow items based on criteria
Args:
items: List of WorkflowItem objects
**filters: Filter criteria (state, item_type, repo_source, author, labels)
Returns:
Filtered list of items
"""
filtered = items
if 'state' in filters and filters['state']:
filtered = [item for item in filtered if item.state == filters['state']]
if 'item_type' in filters and filters['item_type']:
filtered = [item for item in filtered if item.item_type == filters['item_type']]
if 'repo_source' in filters and filters['repo_source']:
filtered = [item for item in filtered if item.repo_source == filters['repo_source']]
if 'author' in filters and filters['author']:
filtered = [item for item in filtered if item.author == filters['author']]
if 'labels' in filters and filters['labels']:
label_filter = filters['labels']
if isinstance(label_filter, str):
label_filter = [label_filter]
filtered = [item for item in filtered
if any(label in item.labels for label in label_filter)]
return filtered