Automate to Keep Up: How to Stop Drowning in GitHub Issues

You maintain an open source project. You love it. You also have a job, a life, and approximately zero spare hours.

Issues pile up untriaged. A contributor opens a PR, gets no response for two weeks, and goes quiet. Someone files a bug with a one-line description. The good-first-issue label never gets applied because you forgot. None of this is intentional neglect. It's just the math of OSS maintenance.

What if you could delegate the boring-but-critical parts to an autonomous agent that runs on a schedule?

This post walks through setting up three agent tasks on Eyevinn Open Source Cloud that handle triage, implementation, and PR follow-up for your project, every day, without you being in the loop.

We'll use https://github.com/birme/vacay-planner as the running example throughout.

What is My Agent Tasks?

My Agent Tasks is a feature of Eyevinn Open Source Cloud (OSC) that lets you schedule autonomous AI agents as cloud-based cron jobs. You write a prompt describing what the agent should do, point it at a Git repository, set a schedule, and OSC handles the rest: spinning up the agent, injecting your credentials from a secure parameter store, running the task, and tearing down the container.

No servers to manage. No queues to wire up. No CI minutes consumed on your repo.

The agent runs as Claude Code under the hood, with full tool access: it can read files, run shell commands, call GitHub APIs, push commits, and open PRs. My Agent Tasks is not Anthropic-only: you can run tasks with Anthropic Claude (the default) or OpenAI Codex and other supported providers, depending on which API key you configure in Step 1.

My Agent Tasks is available on the paid plan, starting at 15 EUR/month with a 14-day free trial. See pricing.

Step 1: Store your GitHub PAT securely

The agent needs a GitHub Personal Access Token to read issues and push code. You never put a raw token in a prompt (prompts are stored in task definitions and could appear in logs). Instead, you use OSC's parameter store.

Connect Claude Desktop or your AI assistant to the OSC MCP server and run:

Create a parameter store called "vacay-planner-agent-store" with secrets enabled.
Then store my GitHub PAT (ghp_...) as the key GITHUB_TOKEN, marked as secret.
Save the config API key — I'll need it when creating tasks.

The MCP tool calls that happen behind the scenes:

1. setup-parameter-store with name: "vacay-planner-agent-store" and enableSecrets: true. This returns a CONFIG_API_KEY once. Copy it.
2. set-parameter with key: "GITHUB_TOKEN", value: "<your PAT>", secret: true, and the config API key from step 1.

At runtime, the agent reads $GITHUB_TOKEN from the environment as a plain variable. The raw value never appears in any log or conversation history.

You also need to store your AI provider key so OSC can run the agent on your behalf. If you want to use Anthropic Claude, store your Anthropic key with credentialType: "anthropicapikey"; if you want to use OpenAI Codex, store your OpenAI key with credentialType: "openaiapikey":

Set up my agent credentials using my Anthropic API key (sk-ant-...).

This calls setup-agent once and applies to all your agent tasks.

Step 2: The triage agent (runs at 08:00 UTC)

This agent runs every morning, finds all untriaged issues, reads them, labels them, and posts a friendly acknowledgment comment.

Create it with:

Create an agent task called "Vacay Planner Daily Triage" that runs every day at 08:00 UTC,
working in the repository https://github.com/birme/vacay-planner.
Use the parameter store "vacay-planner-agent-store" with the config API key I saved earlier.
Set the git token to $GITHUB_TOKEN from the parameter store.

Here is the prompt to use for this task:

You are a maintainer assistant for the GitHub repository github.com/birme/vacay-planner.

Your job is to triage new issues. Run this every time you are invoked:

1. Use the GitHub API (authenticated with the GITHUB_TOKEN environment variable)
   to fetch all open issues that do NOT have the label "triaged".

2. For each untriaged issue:
   a. Read the full issue title, body, and any existing comments carefully.
   b. Categorize it as one of: bug, feature-request, question, or docs.
   c. Add the "triaged" label.
   d. Add a type label: "bug" for bugs, "enhancement" for feature requests,
      "question" for usage questions, "documentation" for docs issues.
   e. If the issue describes a straightforward, well-scoped bug fix or small
      improvement, also add "good-first-issue" and "ready-for-dev".
   f. If the issue is a large feature request, break it down: create sub-issues
      in the same repo for each logical piece of work. Reference the parent
      issue in each sub-issue body.
   g. Post a comment on the issue. Keep it short and human. Acknowledge what
      they reported, tell them what label was applied and why, and for
      good-first-issue items, mention that a contributor may pick it up soon.
      Do not use corporate boilerplate. Do not say "Thank you for your
      contribution to our project."

3. Do not modify issues that already have the "triaged" label.

4. Do not push any code or open any PRs. This task is read-and-label only.

Use gh CLI commands for GitHub API calls where possible. Fall back to curl
with the Authorization header if needed.

Set cronExpression to "0 8 * * *" (daily at 08:00 UTC).

Set disallowedTools to "Write,Edit" to prevent the agent from touching files at all. It only needs Bash for GitHub API calls.

Step 3: The implementation agent (runs at 09:00 UTC)

This agent picks up issues labeled ready-for-dev, clones the repo, implements the fix, and opens a PR. It runs an hour after triage so the triage agent has already labeled anything from the night before.

The prompt:

You are an implementation agent for the GitHub repository github.com/birme/vacay-planner.

Your job is to implement ready-to-fix issues. Run this every time you are invoked:

1. Use the GitHub API (authenticated with GITHUB_TOKEN) to fetch all open issues
   that have the label "ready-for-dev" AND do NOT already have an associated
   pull request.

2. For each qualifying issue (process at most 3 per run to stay within budget):
   a. Read the full issue title, body, and all comments. Understand exactly
      what is being asked before writing any code.
   b. Create a new branch from the default branch. Name it
      "fix/issue-{number}-{short-slug}".
   c. Implement the fix or feature described in the issue.
      - Write clean, idiomatic code that matches the existing style.
      - Add or update tests where the project has them.
      - Do not refactor unrelated code.
   d. Commit your changes with a clear commit message referencing the issue:
      "fix: <description> (closes #<number>)".
   e. Push the branch to origin.
   f. Open a pull request from that branch to the default branch.
      - Title: same as the commit message.
      - Body: brief description of what changed and why, followed by
        "Closes #<number>".
      - Never push directly to main or the default branch.
   g. Post a comment on the issue linking to the PR.

3. If an issue is ambiguous or requires architectural decisions beyond a
   straightforward fix, skip it and post a comment explaining what clarification
   is needed before it can be implemented.

Always prefer small, focused PRs. If a fix is larger than ~200 lines of change,
consider whether it should be split, and comment on the issue instead of opening
a partial PR.

Set cronExpression to "0 9 * * *".

Set maxTurns to "50" to give it enough steps for non-trivial implementations without runaway costs.

Step 4: The PR comment responder (runs at 17:00 UTC)

Reviewers leave feedback on PRs. Without a response, those PRs stall. This agent runs at the end of the day, reads new review comments on open PRs, and either updates the code or explains why the current approach is correct.

The prompt:

You are a PR response agent for the GitHub repository github.com/birme/vacay-planner.

Your job is to address review feedback on open pull requests. Run this every
time you are invoked:

1. Fetch all open pull requests for the repository using GITHUB_TOKEN.

2. For each open PR:
   a. Fetch all review comments and review threads.
   b. Identify comments that have not yet been resolved or replied to.
      Specifically, look for:
      - Inline review comments requesting code changes.
      - Review comments asking questions about the implementation.
      - Pending change requests that have not been addressed.

3. For each unresolved comment:
   a. If the reviewer is asking for a code change:
      - Check out the PR branch.
      - Make the requested change.
      - Commit with a message like "address review: <short description>".
      - Push to the PR branch.
      - Reply to the review comment confirming what was changed.
   b. If the reviewer is asking a clarifying question (no code change needed):
      - Reply to the comment with a clear, concise explanation.
      - Do not push code just to close a thread.

4. Do not force-push or rewrite history on the PR branch. Only add new commits.
5. Do not merge pull requests. That decision belongs to the human maintainer.
6. If a reviewer's request is unclear, ask for clarification in a reply comment
   rather than guessing at what they want.

Keep replies short and factual. No apologetics, no "Great suggestion!".

Set cronExpression to "0 17 * * 1-5" (weekdays at 17:00 UTC).

Putting it all together

You now have three tasks on a staggered schedule:

You wake up to: labeled issues, PRs in flight, and review threads moving forward. You still own the merge button. The agent never merges anything. It contributes; you ship.

To trigger a manual test run before the first scheduled execution:

Run the "Vacay Planner Daily Triage" agent task now.

The MCP tool run-agent-task fires an immediate run. Check progress with get-agent-task-logs.

Tips and gotchas

Scope permissions carefully. The triage agent only needs repo:read and issues:write on your PAT. The implementation agent needs repo:write to push branches. Give the PR responder the same. Principle of least privilege applies even to your own agents.

Review PRs before merging. The agent implements what the issue says. If the issue was underspecified, the implementation may be wrong. Treat agent-opened PRs the same as any contributor PR: read the diff, run the tests, merge only when you are satisfied.

Run with enabled: false first. Create tasks with enabled: false, trigger one manual run with run-agent-task, inspect the logs, and check what the agent did on GitHub before enabling the schedule. This is cheaper than watching it label 40 issues incorrectly on day one.

The agent is a contributor, not an architect. Complex feature design, breaking API changes, and security-sensitive fixes should not be labeled ready-for-dev without a human design pass first. The triage agent knows to skip issues that need architectural clarification. The implementation agent should too, and both prompts above say as much explicitly.

Costs. Each agent task run consumes Anthropic API credits from the key you stored with setup-agent. A typical triage run over 10 issues costs a few cents. An implementation run that writes and tests a patch costs more. Set maxTurns conservatively until you know what your project's runs actually cost.

The three tasks above are what the OSC team uses internally to keep pace with issues across our own platform repos. Triage went from a backlog that accumulated over weekends to same-day labeling. PRs from the implementation agent still need review, but they exist, which is more than could be said when the only person who could open them was busy.

If your backlog could use a night shift, sign up for Eyevinn Open Source Cloud and set up your first agent task today. Paid plans start at 15 EUR per month and include a 14-day free trial.