Skip to content

Conversation

devin-ai-integration[bot]
Copy link
Contributor

feat: restore AgentOps documentation and integration

Summary

This PR restores the AgentOps integration that was removed in PR #3334, addressing user concerns raised in issue #3348. The restoration includes:

  • AgentOpsListener implementation: Event listener that integrates with CrewAI's event system to track crew kickoffs, tool usage, task evaluations, and errors
  • Optional dependency handling: Graceful degradation when AgentOps package is not installed
  • Comprehensive documentation: AgentOps integration docs in English, Portuguese, and Korean
  • Complete test coverage: Unit tests for all AgentOps listener functionality with proper mocking
  • Configuration updates: Added AgentOps to optional dependencies and updated navigation

The integration automatically captures CrewAI events when AgentOps is installed, providing observability for crew executions, tool usage, and performance metrics.

Review & Testing Checklist for Human

  • Test AgentOps integration when installed: Install agentops==0.3.18 and verify that CrewAI automatically integrates with AgentOps during crew execution
  • Test graceful degradation when AgentOps not installed: Ensure CrewAI works normally without AgentOps package installed (no import errors or crashes)
  • Verify event system functionality: Run existing CrewAI tests to ensure no regressions were introduced in the event system
  • Run AgentOps tests: Execute pytest tests/utilities/events/third_party/test_agentops_listener.py -v to verify all test cases pass
  • Review documentation accuracy: Check that AgentOps documentation examples and setup instructions are correct and consistent across languages

Recommended test plan: Create a simple CrewAI script, run it with and without AgentOps installed, and verify proper behavior in both scenarios.


Diagram

%%{ init : { "theme" : "default" }}%%
graph TD
    pyproject["pyproject.toml<br/>Optional Dependencies"]:::major-edit
    AgentOpsListener["src/crewai/utilities/events/<br/>third_party/agentops_listener.py"]:::major-edit
    EventInit["src/crewai/utilities/events/<br/>__init__.py"]:::minor-edit
    ThirdPartyInit["src/crewai/utilities/events/<br/>third_party/__init__.py"]:::major-edit
    
    DocsEN["docs/en/observability/<br/>agentops.mdx"]:::major-edit
    DocsPT["docs/pt-BR/observability/<br/>agentops.mdx"]:::major-edit
    DocsKO["docs/ko/observability/<br/>agentops.mdx"]:::major-edit
    DocsJSON["docs/docs.json<br/>Navigation Config"]:::minor-edit
    
    Tests["tests/utilities/events/<br/>third_party/test_agentops_listener.py"]:::major-edit
    
    EventBus["CrewAI Event Bus"]:::context
    
    pyproject --> AgentOpsListener
    AgentOpsListener --> EventInit
    AgentOpsListener --> ThirdPartyInit
    EventBus --> AgentOpsListener
    DocsJSON --> DocsEN
    DocsJSON --> DocsPT
    DocsJSON --> DocsKO
    Tests --> AgentOpsListener
    
    subgraph Legend
        L1[Major Edit]:::major-edit
        L2[Minor Edit]:::minor-edit
        L3[Context/No Edit]:::context
    end
    
    classDef major-edit fill:#90EE90
    classDef minor-edit fill:#87CEEB
    classDef context fill:#FFFFFF
Loading

Notes

  • Local testing limitation: Could not run tests locally due to pytest environment conflict (pytest-recording vs pytest-vcr), so CI testing is critical
  • Dependency approach: Used exact version pinning (agentops==0.3.18) matching the previous implementation - may want to consider version ranges
  • Event system integration: The AgentOpsListener follows the same pattern as other event listeners in the codebase
  • Documentation scope: Created comprehensive docs in 3 languages based on the previous implementation, but native speaker review recommended

Session info: Requested by João (joao@crewai.com) for issue #3348
Devin session: https://app.devin.ai/sessions/c2ffef3fbc21466396d1a2401255f9ce

- Add AgentOps optional dependency to pyproject.toml
- Implement AgentOpsListener with event handling for crew kickoff, tool usage, and task evaluation
- Add comprehensive AgentOps documentation in English, Portuguese, and Korean
- Update docs.json navigation to include AgentOps in observability sections
- Add agentops.log to .gitignore
- Include comprehensive tests for AgentOps integration with mocking
- Ensure graceful handling when AgentOps package is not installed

Addresses issue #3348 by restoring AgentOps integration removed in PR #3334

Co-Authored-By: João <joao@crewai.com>
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

- Fix event handler signatures to match expected (source, event) pattern
- Change 'arguments' to 'tool_args' for tool usage events
- Update TaskEvaluationEvent to use correct attributes (evaluation_type, task)
- Fix corrupted tests/__init__.py file
- Add explicit re-export in src third_party __init__.py

Co-Authored-By: João <joao@crewai.com>
Copy link
Contributor Author

Closing due to inactivity for more than 7 days. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

0 participants