Building Features with AI-Powered PR Review

Now that you’ve implemented AI monitoring in your application, let’s take it a step further! We’ll add a new AI-powered course creation feature to Sentry Academy, then use Sentry’s brand new AI-powered PR review functionality to ensure our code is production-ready before merging.

Sentry Prevent AI in action on a GitHub pull request

Sentry recently released Prevent AI, which gives you AI-powered code review and test generation capabilities directly in your GitHub workflow. This feature gives you several new capabilities:

Direct integration with GitHub pull requests through the Seer by Sentry app
AI-powered code review that analyzes your changes and suggests improvements
Automatic test generation for uncovered code paths
Security vulnerability detection and performance optimization suggestions
Only runs when explicitly triggered, ensuring it doesn’t interfere with your development flow

Learning Objectives

By the end of this module, you will:

Implement a new AI feature using OpenAI function calling for course creation
Create a pull request for your new feature
Install and configure Sentry Prevent AI for automated PR review
Use AI-powered code review and test generation to improve code quality
Understand how to integrate AI development tools into your workflow

Getting started with AI-powered PR review is straightforward. We need to…

Implement a new feature that adds value to our application
Create a pull request with our changes
Install the Seer by Sentry GitHub app and enable the necessary features
Test the AI review capabilities in action

Implementing the Course Creation AI Tool

We already have a basic course creation feature in Sentry Academy, but let’s enhance it with better OpenAI function calling and proper Sentry instrumentation.

Generate our new tool call

We’re going to have a little bit of fun with code generation! First, provide the following prompt in your IDE like Cursor…

 I need to add a new tool call to my application to handle course generation. This tool call should use structure like the following, and create a new course in my project using the chat interface.

And paste the following tool call below your prompt. We do this to give the LLM a structure to understand how to use. In a few moments, it should produce a new chat endpoint.

// Add this enhanced function calling definition
const tools = [
  {
    type: 'function' as const,
    function: {
      name: 'create_structured_course',
      description: 'Create a comprehensive course with detailed modules and lessons',
      parameters: {
        type: 'object',
        properties: {
          title: {
            type: 'string',
            description: 'Engaging course title that clearly describes the learning outcome'
          },
          description: {
            type: 'string',
            description: 'Comprehensive course description explaining what students will learn'
          },
          category: {
            type: 'string',
            description: 'Course category (e.g., Programming, Design, Business)'
          },
          estimatedHours: {
            type: 'number',
            description: 'Total estimated hours to complete the course'
          },
          modules: {
            type: 'array',
            description: 'Array of course modules',
            items: {
              type: 'object',
              properties: {
                title: { type: 'string', description: 'Module title' },
                description: { type: 'string', description: 'Module overview' },
                lessons: {
                  type: 'array',
                  items: {
                    type: 'object',
                    properties: {
                      title: { type: 'string', description: 'Lesson title' },
                      objectives: {
                        type: 'array',
                        items: { type: 'string' },
                        description: 'Learning objectives for this lesson'
                      },
                      estimatedMinutes: { type: 'number', description: 'Estimated completion time' }
                    },
                    required: ['title', 'objectives', 'estimatedMinutes']
                  }
                }
              },
              required: ['title', 'description', 'lessons']
            }
          }
        },
        required: ['title', 'description', 'category', 'estimatedHours', 'modules']
      }
    }
  }
];

Test the Enhanced Course Creation

Navigate to the Course Builder in Sentry Academy and create a new course with a prompt like:

“Building scalable REST APIs with Node.js and TypeScript”

Setting Up Sentry Prevent AI

Before we can use Sentry’s AI-powered PR review, we need to install and configure the Sentry GitHub app.

Enable AI Features in Sentry Organization Settings

In your Sentry organization settings, ensure the feature below is enabled:
- Enable PR Review and Test Generation
Navigate to your organization settings and look for the AI features section.
Install the Seer by Sentry GitHub App

Navigate to the Seer by Sentry GitHub App and select Configure on it in your repository.

If you’re not an organization admin, copy the installation link and share it with your admin to install it.
Confirm the installation and its permissions

Select the install option and confirm the permissions.
Verify the installation

If everything went right, you should see the following screen once installation is complete.

Creating a Pull Request for Code Review

We’ve got Seer’s PR Review and Test Generation installed, now we’re ready to test it out! Let’s create a pull request to merge our course creation feature into the main branch.

Commit Your Changes

First, stage and commit your enhanced course creation feature (and while we’re at it, everything else). From the root of the project, run the following commands (you can also do this in your IDE from the source control tab):
Terminal window
```
git add .
git commit -m "Enhance course creation with structured function calling and improved Sentry instrumentation"
```

Push to a Feature Branch

Create and push a feature branch for our improvements:

git checkout -b feature/enhanced-course-creation
git push -u origin feature/enhanced-course-creation

Open a Pull Request

Navigate to your GitHub repository and create a new pull request from the feature/enhanced-course-creation branch to main. Include a detailed description of your changes:

## Enhanced Course Creation with Better Function Calling

This PR improves our AI course creation feature with:

- Structured OpenAI function calling with detailed schemas
- Enhanced Sentry instrumentation with metadata
- Better error handling and monitoring
- More comprehensive course structure generation

It also inclues several other fixes throughout my project.

Testing Sentry Prevent AI in Action

Now let’s see Sentry Prevent AI in action by requesting a code review of our enhanced course creation feature.

Request an AI Code Review

In your pull request, add a comment with the following command:
```
@sentry review my PR
```
Sentry Prevent AI will acknowledge your request and begin analyzing your code changes. The AI will review:
- Code structure and implementation patterns
- Potential bugs or edge cases
- Security considerations for API endpoints
- Performance implications of function calling
- TypeScript usage and type safety
Review AI-Generated Feedback

The AI will add review comments directly to your pull request, providing specific suggestions such as:

Common suggestions might include:
- Adding input validation for the prompt parameter
- Implementing rate limiting for the OpenAI API calls
- Adding error handling for malformed function call responses
- Improving TypeScript interfaces for the course structure
- Adding timeout configurations for long-running AI requests
Generate Comprehensive Tests

Request AI-generated tests by commenting:
```
@sentry generate-test
```
Sentry Prevent AI will analyze your course creation feature and generate tests covering:
- Unit tests for the enhanced function calling logic
- Integration tests for the OpenAI API interaction
- Error handling scenarios (API failures, malformed responses)
- Performance tests for the enhanced telemetry
- Edge cases for various prompt inputs

Review Generated Test Code

The AI will provide a link to view the generated tests, which might include:

describe('Enhanced Course Creation', () => {
  it('should create structured course with function calling', async () => {
    // Test implementation for successful course creation
  });

  it('should handle OpenAI API errors gracefully', async () => {
    // Test error handling scenarios
  });

  it('should validate input prompts properly', async () => {
    // Test input validation
  });
});

Summing Up AI-Powered PR Review

AI-powered PR review with Sentry Prevent AI gives you a comprehensive approach to maintaining code quality and accelerating development. You can use it to:

Catch Issues Early: Identify potential problems before they reach production
Improve Code Quality: Get specific suggestions for better implementations
Enhance Test Coverage: Generate comprehensive tests for new features
Learn Best Practices: Use AI feedback to continuously improve coding skills
Streamline Reviews: Reduce the manual effort required for thorough code reviews

The combination of enhanced Sentry instrumentation with AI-powered development tools creates a powerful workflow for building reliable, well-monitored applications. Continue using @sentry review and @sentry generate-test commands on your pull requests to maintain high standards as your application evolves.