The PR Review patchflow aims to automatically summarize and comment on PRs in your repository.

​
How to run?

You can run it as follows:

patchwork PRReview pr_url=<Link_to_your_PR>

by default you will need to provide the openai_api_key and the github_api_key you can pass them as arguments:

patchwork PRReview openai_api_key=<Your_API_KEY> github_api_key=<Your_GH_Token> pr_url=<Link_to_your_PR>

​
What it does?

The PRReview patchflow will first read the PR. It will then extract the diff from the PR and use it to create a prompt to be sent to gpt-3.5-turbo to create a summary. You can check the default prompt template. The PR summary is then added as a comment to the PR.

​
Configuration

The following are the default configurations that can be modified by the user to adapt the PRReview patchflow to their needs. All the options can be set both via CLI arguments and and the yaml config file.

​
Model

You can choose any LLM API as long as it has an OpenAI API compatible chat completions endpoint. Just update the default values of the following options:

- model: gpt-3.5-turbo
- client_base_url: https://api.openai.com/v1

E.g. to use Meta’s CodeLlama model from HuggingFace you can set:

client_base_url: https://api-inference.huggingface.co/models/codellama/CodeLlama-70b-Instruct-hf/v1
model: codellama/CodeLlama-70b-Instruct-hf
model_temperature: 0.2
model_top_p: 0.95
model_max_tokens: 2000

and pass your HuggingFace token in the openai_api_key option.

You can also use llama.cpp to run inference on CPU locally. Just install the llama-cpp-python package and run their OpenAI compatible web server as described here with the command:

python3 -m llama_cpp.server --hf_model_repo_id TheBloke/deepseek-coder-6.7B-instruct-GGUF --model 'deepseek-coder-6.7b-instruct.Q4_0.gguf'

Once the local server is running you can set:

client_base_url: http://localhost:8000/v1
model: TheBloke/deepseek-coder-6.7B-instruct-GGUF
model_temperature: 0.2
model_top_p: 0.95
model_max_tokens: 1000

and use the local model for inference.

​
Diff suggestion

When the diff_suggestion option is set, the PRReview patchflow will generate suggestions to improve the PR along with the summary.

diff_suggestion: true

​
Diff summary

You can configure the length of the summary using the diff_summary option. Acceptable values are ‘long’, ‘short’ and ‘none’.

diff_summary: short

​
Prompt template

You can update the default prompt template. The basic prompt that comments on the PR is with "id": "diffreview". Note the use of variables {{other_fields}}, {{path}} and {{diff}}. They are generated by the steps within the PRReview patchflow and are replaced by the actual values during the execution. Also, remember to keep the output format as given in the default prompt since it is used to extract information that is used by the steps after the model response is processed. The following output format is expected at the moment:

A. Summary: 
<summary of the diff>

when the diff_suggestion option is set the expected output format is:

B. Suggestion:
<suggestion to improve the PR>

A. Summary:
<summary of the diff>

​
Examples

Here are some example comments generated with the PRReview patchflow:

OverviewAutoFix