Compare commits

..

1 Commits

527 changed files with 20244 additions and 50422 deletions
+4 -4
View File
@@ -1,9 +1,9 @@
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and WebStorm # Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and WebStorm
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839 # Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
# github actions # github acions
.git
.github/ .github/
.*ignore .*ignore
.git/
# User-specific stuff # User-specific stuff
.idea/ .idea/
# Byte-compiled / optimized / DLL files # Byte-compiled / optimized / DLL files
@@ -15,10 +15,10 @@ env/
venv*/ venv*/
ENV/ ENV/
.conda/ .conda/
README*.md
dashboard/ dashboard/
data/ data/
changelogs/ changelogs/
tests/ tests/
.ruff_cache/ .ruff_cache/
.astrbot .astrbot
astrbot.lock
+4 -5
View File
@@ -16,7 +16,7 @@ body:
请将插件信息填写到下方的 JSON 代码块中。其中 `tags`(插件标签)和 `social_link`(社交链接)选填。 请将插件信息填写到下方的 JSON 代码块中。其中 `tags`(插件标签)和 `social_link`(社交链接)选填。
不熟悉 JSON ?可以从 [此站](https://plugins.astrbot.app) 右下角提交。 不熟悉 JSON 现在可以从 [这里](https://plugins.astrbot.app/#/submit) 获取你的 JSON 啦!获取到了记得复制粘贴过来哦!
- type: textarea - type: textarea
id: plugin-info id: plugin-info
@@ -26,13 +26,12 @@ body:
value: | value: |
```json ```json
{ {
"name": "插件名,请以 astrbot_plugin_ 开头", "name": "插件名",
"display_name": "用于展示的插件名,方便人类阅读", "desc": "插件介绍",
"desc": "插件的简短介绍",
"author": "作者名", "author": "作者名",
"repo": "插件仓库链接", "repo": "插件仓库链接",
"tags": [], "tags": [],
"social_link": "", "social_link": ""
} }
``` ```
validations: validations:
+23 -21
View File
@@ -1,44 +1,46 @@
name: '🐛 Report Bug / 报告 Bug' name: '🐛 报告 Bug'
title: '[Bug]' title: '[Bug]'
description: Submit bug report to help us improve. / 提交报告帮助我们改进。 description: 提交报告帮助我们改进。
labels: [ 'bug' ] labels: [ 'bug' ]
body: body:
- type: markdown - type: markdown
attributes: attributes:
value: | value: |
Thank you for taking the time to report this issue! Please describe your problem accurately. If possible, please provide a reproducible snippet (this will help resolve the issue more quickly). Please note that issues that are not detailed or have no logs will be closed immediately. Thank you for your understanding. / 感谢您抽出时间报告问题!请准确解释您的问题。如果可能,请提供一个可复现的片段(这有助于更快地解决问题)。请注意,不详细 / 没有日志的 issue 会被直接关闭,谢谢理解。 感谢您抽出时间报告问题!请准确解释您的问题。如果可能,请提供一个可复现的片段(这有助于更快地解决问题)。
- type: textarea - type: textarea
attributes: attributes:
label: What happened / 发生了什么 label: 发生了什么
description: Description description: 描述你遇到的异常
placeholder: > placeholder: >
Please provide a clear and specific description of what this exception is. Please note that issues that are not detailed or have no logs will be closed immediately. Thank you for your understanding. / 一个清晰且具体的描述这个异常是什么。请注意,不详细 / 没有日志的 issue 会被直接关闭,谢谢理解 一个清晰且具体的描述这个异常是什么
validations: validations:
required: true required: true
- type: textarea - type: textarea
attributes: attributes:
label: Reproduce / 如何复现? label: 如何复现?
description: > description: >
The steps to reproduce the issue. / 复现该问题的步骤 复现该问题的步骤
placeholder: > placeholder: >
Example: 1. Open '...' : 1. 打开 '...'
validations: validations:
required: true required: true
- type: textarea - type: textarea
attributes: attributes:
label: AstrBot version, deployment method (e.g., Windows Docker Desktop deployment), provider used, and messaging platform used. / AstrBot 版本、部署方式(如 Windows Docker Desktop 部署)、使用的提供商、使用的消息平台适配器 label: AstrBot 版本、部署方式(如 Windows Docker Desktop 部署)、使用的提供商、使用的消息平台适配器
description: >
请提供您的 AstrBot 版本和部署方式。
placeholder: > placeholder: >
Example: 4.5.7 Docker, 3.1.7 Windows Launcher 如: 3.1.8 Docker, 3.1.7 Windows启动器
validations: validations:
required: true required: true
- type: dropdown - type: dropdown
attributes: attributes:
label: OS label: 操作系统
description: | description: |
On which operating system did you encounter this problem? / 你在哪个操作系统上遇到了这个问题? 你在哪个操作系统上遇到了这个问题?
multiple: false multiple: false
options: options:
- 'Windows' - 'Windows'
@@ -51,30 +53,30 @@ body:
- type: textarea - type: textarea
attributes: attributes:
label: Logs / 报错日志 label: 报错日志
description: > description: >
Please provide complete Debug-level logs, such as error logs and screenshots. Don't worry if they're long! Please note that issues with insufficient details or no logs will be closed immediately. Thank you for your understanding. / 如报错日志、截图等。请提供完整的 Debug 级别的日志,不要介意它很长!请注意,不详细 / 没有日志的 issue 会被直接关闭,谢谢理解。 如报错日志、截图等。请提供完整的 Debug 级别的日志,不要介意它很长!
placeholder: > placeholder: >
Please provide a complete error log or screenshot. / 请提供完整的报错日志或截图。 请提供完整的报错日志或截图。
validations: validations:
required: true required: true
- type: checkboxes - type: checkboxes
attributes: attributes:
label: Are you willing to submit a PR? / 你愿意提交 PR 吗? label: 你愿意提交 PR 吗?
description: > description: >
This is not required, but we would be happy to provide guidance during the contribution process, especially if you already have a good understanding of how to implement the fix. / 这不是必需的,但我们很乐意在贡献过程中为您提供指导特别是如果你已经很好地理解了如何实现修复。 这不是必需的,但我们很乐意在贡献过程中为您提供指导特别是如果你已经很好地理解了如何实现修复。
options: options:
- label: Yes! - label: 是的,我愿意提交 PR!
- type: checkboxes - type: checkboxes
attributes: attributes:
label: Code of Conduct label: Code of Conduct
options: options:
- label: > - label: >
I have read and agree to abide by the project's [Code of Conduct](https://docs.github.com/zh/site-policy/github-terms/github-community-code-of-conduct)。 我已阅读并同意遵守该项目的 [行为准则](https://docs.github.com/zh/site-policy/github-terms/github-community-code-of-conduct)。
required: true required: true
- type: markdown - type: markdown
attributes: attributes:
value: "Thank you for filling out our form! / 感谢您填写我们的表单!" value: "感谢您填写我们的表单!"
+12 -20
View File
@@ -1,27 +1,19 @@
<!--Please describe the motivation for this change: What problem does it solve? (e.g., Fixes XX issue, adds YY feature)--> <!-- 如果有的话,指定这个 PR 要解决的 ISSUE -->
<!--请描述此项更改的动机:它解决了什么问题?(例如:修复了 XX issue,添加了 YY 功能)--> 解决了 #XYZ
### Modifications / 改动点 ### Motivation
<!--请总结你的改动:哪些核心文件被修改了?实现了什么功能?--> <!--解释为什么要改动-->
<!--Please summarize your changes: What core files were modified? What functionality was implemented?-->
- [x] This is NOT a breaking change. / 这不是一个破坏性变更。 ### Modifications
<!-- If your changes is a breaking change, please uncheck the checkbox above -->
### Screenshots or Test Results / 运行截图或测试结果 <!--简单解释你的改动-->
<!--Please paste screenshots, GIFs, or test logs here as evidence of executing the "Verification Steps" to prove this change is effective.--> ### Check
<!--请粘贴截图、GIF 或测试日志,作为执行“验证步骤”的证据,证明此改动有效。-->
--- <!--如果分支被合并,您的代码将服务于数万名用户!在提交前,请核查一下几点内容-->
### Checklist / 检查清单 - [ ] 😊 我的 Commit Message 符合良好的[规范](https://www.conventionalcommits.org/en/v1.0.0/#summary)
- [ ] 👀 我的更改经过良好的测试
<!--If merged, your code will serve tens of thousands of users! Please double-check the following items before submitting.--> - [ ] 🤓 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到了 `requirements.txt``pyproject.toml` 文件相应位置。
<!--如果分支被合并,您的代码将服务于数万名用户!在提交前,请核查一下几点内容。--> - [ ] 😮 我的更改没有引入恶意代码
- [ ] 😊 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。/ If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
- [ ] 👀 我的更改经过了良好的测试,**并已在上方提供了“验证步骤”和“运行截图”**。/ My changes have been well-tested, **and "Verification Steps" and "Screenshots" have been provided above**.
- [ ] 🤓 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到了 `requirements.txt``pyproject.toml` 文件相应位置。/ I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in `requirements.txt` and `pyproject.toml`.
- [ ] 😮 我的更改没有引入恶意代码。/ My changes do not introduce malicious code.
-38
View File
@@ -1,38 +0,0 @@
# Set to true to add reviewers to pull requests
addReviewers: true
# Set to true to add assignees to pull requests
addAssignees: false
# A list of reviewers to be added to pull requests (GitHub user name)
reviewers:
- Soulter
- Raven95676
- Larch-C
- anka-afk
- advent259141
- Fridemn
- LIghtJUNction
# - zouyonghe
# A number of reviewers added to the pull request
# Set 0 to add all the reviewers (default: 0)
numberOfReviewers: 2
# A list of assignees, overrides reviewers if set
# assignees:
# - assigneeA
# A number of assignees to add to the pull request
# Set to 0 to add all of the assignees.
# Uses numberOfReviewers if unset.
# numberOfAssignees: 2
# A list of keywords to be skipped the process that add reviewers if pull requests include it
skipKeywords:
- wip
- draft
# A list of users to be skipped by both the add reviewers and add assignees processes
# skipUsers:
# - dependabot[bot]
+3 -3
View File
@@ -13,7 +13,7 @@ jobs:
contents: write contents: write
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v6 uses: actions/checkout@v5
- name: Dashboard Build - name: Dashboard Build
run: | run: |
@@ -70,10 +70,10 @@ jobs:
needs: build-and-publish-to-github-release needs: build-and-publish-to-github-release
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v6 uses: actions/checkout@v5
- name: Set up Python - name: Set up Python
uses: actions/setup-python@v6 uses: actions/setup-python@v5
with: with:
python-version: '3.10' python-version: '3.10'
-34
View File
@@ -1,34 +0,0 @@
name: Code Format Check
on:
pull_request:
branches: [ master ]
push:
branches: [ master ]
jobs:
format-check:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.10'
- name: Install UV
run: pip install uv
- name: Install dependencies
run: uv sync
- name: Check code formatting with ruff
run: |
uv run ruff format --check .
- name: Check code style with ruff
run: |
uv run ruff check .
+3 -3
View File
@@ -56,11 +56,11 @@ jobs:
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages # your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v6 uses: actions/checkout@v5
# Initializes the CodeQL tools for scanning. # Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL - name: Initialize CodeQL
uses: github/codeql-action/init@v4 uses: github/codeql-action/init@v3
with: with:
languages: ${{ matrix.language }} languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }} build-mode: ${{ matrix.build-mode }}
@@ -88,6 +88,6 @@ jobs:
exit 1 exit 1
- name: Perform CodeQL Analysis - name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4 uses: github/codeql-action/analyze@v3
with: with:
category: "/language:${{matrix.language}}" category: "/language:${{matrix.language}}"
+2 -2
View File
@@ -17,12 +17,12 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v6 uses: actions/checkout@v5
with: with:
fetch-depth: 0 fetch-depth: 0
- name: Set up Python - name: Set up Python
uses: actions/setup-python@v6 uses: actions/setup-python@v5
- name: Install dependencies - name: Install dependencies
run: | run: |
+4 -12
View File
@@ -11,20 +11,13 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@v6 uses: actions/checkout@v5
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: 'latest'
- name: npm install, build - name: npm install, build
run: | run: |
cd dashboard cd dashboard
npm install pnpm -g npm install
pnpm install npm run build
pnpm i --save-dev @types/markdown-it
pnpm run build
- name: Inject Commit SHA - name: Inject Commit SHA
id: get_sha id: get_sha
@@ -36,7 +29,7 @@ jobs:
zip -r dist.zip dist zip -r dist.zip dist
- name: Archive production artifacts - name: Archive production artifacts
uses: actions/upload-artifact@v5 uses: actions/upload-artifact@v4
with: with:
name: dist-without-markdown name: dist-without-markdown
path: | path: |
@@ -44,7 +37,6 @@ jobs:
!dist/**/*.md !dist/**/*.md
- name: Create GitHub Release - name: Create GitHub Release
if: github.event_name == 'push'
uses: ncipollo/release-action@v1 uses: ncipollo/release-action@v1
with: with:
tag: release-${{ github.sha }} tag: release-${{ github.sha }}
+12 -148
View File
@@ -3,125 +3,18 @@ name: Docker Image CI/CD
on: on:
push: push:
tags: tags:
- "v*" - 'v*'
schedule:
# Run at 00:00 UTC every day
- cron: "0 0 * * *"
workflow_dispatch: workflow_dispatch:
jobs: jobs:
build-nightly-image: publish-docker:
if: github.event_name == 'schedule'
runs-on: ubuntu-latest runs-on: ubuntu-latest
env:
DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }}
GHCR_OWNER: soulter
HAS_GHCR_TOKEN: ${{ secrets.GHCR_GITHUB_TOKEN != '' }}
steps: steps:
- name: Checkout - name: Pull The Codes
uses: actions/checkout@v6 uses: actions/checkout@v5
with: with:
fetch-depth: 1 fetch-depth: 0 # Must be 0 so we can fetch tags
fetch-tag: true
- name: Check for new commits today
if: github.event_name == 'schedule'
id: check-commits
run: |
# Get commits from the last 24 hours
commits=$(git log --since="24 hours ago" --oneline)
if [ -z "$commits" ]; then
echo "No commits in the last 24 hours, skipping build"
echo "has_commits=false" >> $GITHUB_OUTPUT
else
echo "Found commits in the last 24 hours:"
echo "$commits"
echo "has_commits=true" >> $GITHUB_OUTPUT
fi
- name: Exit if no commits
if: github.event_name == 'schedule' && steps.check-commits.outputs.has_commits == 'false'
run: exit 0
- name: Build Dashboard
run: |
cd dashboard
npm install
npm run build
mkdir -p dist/assets
echo $(git rev-parse HEAD) > dist/assets/version
cd ..
mkdir -p data
cp -r dashboard/dist data/
- name: Determine test image tags
id: test-meta
run: |
short_sha=$(echo "${GITHUB_SHA}" | cut -c1-12)
build_date=$(date +%Y%m%d)
echo "short_sha=$short_sha" >> $GITHUB_OUTPUT
echo "build_date=$build_date" >> $GITHUB_OUTPUT
- name: Set QEMU
uses: docker/setup-qemu-action@v3
- name: Set Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_PASSWORD }}
- name: Login to GitHub Container Registry
if: env.HAS_GHCR_TOKEN == 'true'
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ env.GHCR_OWNER }}
password: ${{ secrets.GHCR_GITHUB_TOKEN }}
- name: Build nightly image tags list
id: test-tags
run: |
TAGS="${{ env.DOCKER_HUB_USERNAME }}/astrbot:nightly-latest
${{ env.DOCKER_HUB_USERNAME }}/astrbot:nightly-${{ steps.test-meta.outputs.build_date }}-${{ steps.test-meta.outputs.short_sha }}"
if [ "${{ env.HAS_GHCR_TOKEN }}" = "true" ]; then
TAGS="$TAGS
ghcr.io/${{ env.GHCR_OWNER }}/astrbot:nightly-latest
ghcr.io/${{ env.GHCR_OWNER }}/astrbot:nightly-${{ steps.test-meta.outputs.build_date }}-${{ steps.test-meta.outputs.short_sha }}"
fi
echo "tags<<EOF" >> $GITHUB_OUTPUT
echo "$TAGS" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Build and Push Nightly Image
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.test-tags.outputs.tags }}
- name: Post build notifications
run: echo "Test Docker image has been built and pushed successfully"
build-release-image:
if: github.event_name == 'workflow_dispatch' || (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v'))
runs-on: ubuntu-latest
env:
DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }}
GHCR_OWNER: soulter
HAS_GHCR_TOKEN: ${{ secrets.GHCR_GITHUB_TOKEN != '' }}
steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 1
fetch-tag: true
- name: Get latest tag (only on manual trigger) - name: Get latest tag (only on manual trigger)
id: get-latest-tag id: get-latest-tag
@@ -134,34 +27,6 @@ jobs:
if: github.event_name == 'workflow_dispatch' if: github.event_name == 'workflow_dispatch'
run: git checkout ${{ steps.get-latest-tag.outputs.latest_tag }} run: git checkout ${{ steps.get-latest-tag.outputs.latest_tag }}
- name: Compute release metadata
id: release-meta
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
version="${{ steps.get-latest-tag.outputs.latest_tag }}"
else
version="${GITHUB_REF#refs/tags/}"
fi
if [[ "$version" == *"beta"* ]] || [[ "$version" == *"alpha"* ]]; then
echo "is_prerelease=true" >> $GITHUB_OUTPUT
echo "Version $version marked as pre-release"
else
echo "is_prerelease=false" >> $GITHUB_OUTPUT
echo "Version $version marked as stable"
fi
echo "version=$version" >> $GITHUB_OUTPUT
- name: Build Dashboard
run: |
cd dashboard
npm install
npm run build
mkdir -p dist/assets
echo $(git rev-parse HEAD) > dist/assets/version
cd ..
mkdir -p data
cp -r dashboard/dist data/
- name: Set QEMU - name: Set QEMU
uses: docker/setup-qemu-action@v3 uses: docker/setup-qemu-action@v3
@@ -175,24 +40,23 @@ jobs:
password: ${{ secrets.DOCKER_HUB_PASSWORD }} password: ${{ secrets.DOCKER_HUB_PASSWORD }}
- name: Login to GitHub Container Registry - name: Login to GitHub Container Registry
if: env.HAS_GHCR_TOKEN == 'true'
uses: docker/login-action@v3 uses: docker/login-action@v3
with: with:
registry: ghcr.io registry: ghcr.io
username: ${{ env.GHCR_OWNER }} username: Soulter
password: ${{ secrets.GHCR_GITHUB_TOKEN }} password: ${{ secrets.GHCR_GITHUB_TOKEN }}
- name: Build and Push Release Image - name: Build and Push Docker to DockerHub and Github GHCR
uses: docker/build-push-action@v6 uses: docker/build-push-action@v6
with: with:
context: . context: .
platforms: linux/amd64,linux/arm64 platforms: linux/amd64,linux/arm64
push: true push: true
tags: | tags: |
${{ steps.release-meta.outputs.is_prerelease == 'false' && format('{0}/astrbot:latest', env.DOCKER_HUB_USERNAME) || '' }} ${{ secrets.DOCKER_HUB_USERNAME }}/astrbot:latest
${{ steps.release-meta.outputs.is_prerelease == 'false' && env.HAS_GHCR_TOKEN == 'true' && format('ghcr.io/{0}/astrbot:latest', env.GHCR_OWNER) || '' }} ${{ secrets.DOCKER_HUB_USERNAME }}/astrbot:${{ github.event_name == 'workflow_dispatch' && steps.get-latest-tag.outputs.latest_tag || github.ref_name }}
${{ format('{0}/astrbot:{1}', env.DOCKER_HUB_USERNAME, steps.release-meta.outputs.version) }} ghcr.io/soulter/astrbot:latest
${{ env.HAS_GHCR_TOKEN == 'true' && format('ghcr.io/{0}/astrbot:{1}', env.GHCR_OWNER, steps.release-meta.outputs.version) || '' }} ghcr.io/soulter/astrbot:${{ github.event_name == 'workflow_dispatch' && steps.get-latest-tag.outputs.latest_tag || github.ref_name }}
- name: Post build notifications - name: Post build notifications
run: echo "Release Docker image has been built and pushed successfully" run: echo "Docker image has been built and pushed successfully"
-58
View File
@@ -1,58 +0,0 @@
name: Smoke Test
on:
push:
branches:
- master
paths-ignore:
- 'README*.md'
- 'changelogs/**'
- 'dashboard/**'
pull_request:
workflow_dispatch:
jobs:
smoke-test:
name: Run smoke tests
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.12'
- name: Install UV package manager
run: |
pip install uv
- name: Install dependencies
run: |
uv sync
timeout-minutes: 15
- name: Run smoke tests
run: |
uv run main.py &
APP_PID=$!
echo "Waiting for application to start..."
for i in {1..60}; do
if curl -f http://localhost:6185 > /dev/null 2>&1; then
echo "Application started successfully!"
kill $APP_PID
exit 0
fi
sleep 1
done
echo "Application failed to start within 30 seconds"
kill $APP_PID 2>/dev/null || true
exit 1
timeout-minutes: 2
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
pull-requests: write pull-requests: write
steps: steps:
- uses: actions/stale@v10 - uses: actions/stale@v9
with: with:
repo-token: ${{ secrets.GITHUB_TOKEN }} repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: 'Stale issue message' stale-issue-message: 'Stale issue message'
+18 -37
View File
@@ -1,52 +1,33 @@
# Python related
__pycache__ __pycache__
.mypy_cache
.venv*
.conda/
uv.lock
.coverage
# IDE and editors
.vscode
.idea
# Logs and temporary files
botpy.log botpy.log
logs/ .vscode
temp .venv*
cookies.json .idea
# Data files
data_v2.db data_v2.db
data_v3.db data_v3.db
data
configs/session configs/session
configs/config.yaml configs/config.yaml
**/.DS_Store
temp
cmd_config.json cmd_config.json
data
# Plugins and packages cookies.json
logs/
addons/plugins addons/plugins
packages/python_interpreter/workplace .coverage
tests/astrbot_plugin_openai
# Dashboard
tests/astrbot_plugin_openai
chroma
dashboard/node_modules/ dashboard/node_modules/
dashboard/dist/ dashboard/dist/
.DS_Store
package-lock.json package-lock.json
package.json package.json
yarn.lock
# Operating System
**/.DS_Store
.DS_Store
# AstrBot specific
.astrbot
astrbot.lock
# Other
chroma
venv/* venv/*
packages/python_interpreter/workplace
.venv/*
.conda/
.idea
pytest.ini pytest.ini
AGENTS.md .astrbot
IFLOW.md
+5 -17
View File
@@ -6,20 +6,8 @@ ci:
autoupdate_schedule: weekly autoupdate_schedule: weekly
autoupdate_commit_msg: ":balloon: pre-commit autoupdate" autoupdate_commit_msg: ":balloon: pre-commit autoupdate"
repos: repos:
- repo: https://github.com/astral-sh/ruff-pre-commit - repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version. rev: v0.11.2
rev: v0.14.1 hooks:
hooks: - id: ruff
# Run the linter. - id: ruff-format
- id: ruff-check
types_or: [ python, pyi ]
args: [ --fix ]
# Run the formatter.
- id: ruff-format
types_or: [ python, pyi ]
- repo: https://github.com/asottile/pyupgrade
rev: v3.21.0
hooks:
- id: pyupgrade
args: [--py310-plus]
-65
View File
@@ -1,65 +0,0 @@
# CONTRIBUTING
## 贡献指南
首先,感谢您花时间做出贡献!❤️
所有类型的贡献都受到鼓励和重视。有关不同的帮助方式和处理方式的详细信息,请参阅[目录](#目录)。在做出贡献之前,请确保阅读相关部分。这将使我们维护人员的工作变得更加容易,并为所有参与者带来顺畅的体验。社区期待您的贡献。🎉
### 目录
- [报告问题](#报告问题)
- [提交代码更改](#提交代码更改)
### 报告问题
如果您在使用 AstrBot 时遇到任何问题,请按照以下步骤报告:
1. **检查现有问题**:在提交新问题之前,请先检查 [Issues](https://github.com/AstrBotDevs/AstrBot/issues) 中是否已经存在类似的问题。
2. **创建新问题**:如果没有类似的问题,请创建一个新问题。请确保提供以下信息:
- 问题的简要描述
- 重现问题的步骤
- 预期结果和实际结果
- 相关日志或错误消息
### 提交代码更改
#### 分支命名
我们使用 `fix/` 前缀来修复错误,使用 `feat/` 前缀来添加新功能。对于 `fix/` 分支,请使用简短的描述,或者直接使用 Issue 编号。例如:`fix/1234` 或者 `fix/1234-login-typo`。对于 `feat/` 分支,请使用简短的描述,例如:`feat/add-user-profile`
#### PR 描述
- 请使用英文描述您的 PR。
- 标题请使用 `fix: `, `feat: `, `docs: `, `style: `, `refactor: `, `test: `, `chore: ` 等语义化前缀,并简要描述更改内容。如:`fix: correct login page typo`
## Contributing Guide
First off, thanks for taking the time to contribute! ❤️
All types of contributions are encouraged and valued. See the [Table of Contents](#table-of-contents) for different ways to help and details about how this project handles them. Please make sure to read the relevant section before making your contribution. It will make it a lot easier for us maintainers and smooth out the experience for all involved. The community looks forward to your contributions. 🎉
### Table of Contents
- [Reporting Issues](#reporting-issues)
- [Pull Requests](#pull-requests)
### Reporting Issues
If you encounter any issues while using AstrBot, please follow these steps to report them:
1. **Check Existing Issues**: Before submitting a new issue, please check if a similar issue already exists in the [Issues](https://github.com/AstrBotDevs/AstrBot/issues) section of the repository.
2. **Create a New Issue**: If no similar issue exists, please create a new issue. Make sure to provide the following information:
- A brief description of the issue
- Steps to reproduce the issue
- Expected and actual results
- Relevant logs or error messages
### Pull Requests
#### Branch Naming
We use the `fix/` prefix for bug fixes and the `feat/` prefix for new features. For `fix/` branches, please use a short description or directly use the Issue number, e.g., `fix/1234` or `fix/1234-login-typo`. For `feat/` branches, please use a short description, e.g., `feat/add-user-profile`.
#### PR Description
- Please use English to describe your PR.
- Use semantic prefixes like `fix: `, `feat: `, `docs: `, `style: `, `refactor: `, `test: `, `chore: ` in the title, followed by a brief description of the changes, e.g., `fix: correct login page typo`.
+17 -14
View File
@@ -4,6 +4,8 @@ WORKDIR /AstrBot
COPY . /AstrBot/ COPY . /AstrBot/
RUN apt-get update && apt-get install -y --no-install-recommends \ RUN apt-get update && apt-get install -y --no-install-recommends \
nodejs \
npm \
gcc \ gcc \
build-essential \ build-essential \
python3-dev \ python3-dev \
@@ -11,22 +13,23 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libssl-dev \ libssl-dev \
ca-certificates \ ca-certificates \
bash \ bash \
ffmpeg \
curl \
gnupg \
git \
&& apt-get clean \ && apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* && rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y curl gnupg \ RUN python -m pip install uv
&& curl -fsSL https://deb.nodesource.com/setup_lts.x | bash - \
&& apt-get install -y nodejs
RUN python -m pip install uv \
&& echo "3.11" > .python-version
RUN uv pip install -r requirements.txt --no-cache-dir --system RUN uv pip install -r requirements.txt --no-cache-dir --system
RUN uv pip install socksio uv pilk --no-cache-dir --system RUN uv pip install socksio uv pyffmpeg pilk --no-cache-dir --system
# 释出 ffmpeg
RUN python -c "from pyffmpeg import FFmpeg; ff = FFmpeg();"
# add /root/.pyffmpeg/bin/ffmpeg to PATH, inorder to use ffmpeg
RUN echo 'export PATH=$PATH:/root/.pyffmpeg/bin' >> ~/.bashrc
EXPOSE 6185
EXPOSE 6186
CMD [ "python", "main.py" ]
EXPOSE 6185
CMD ["python", "main.py"]
+35
View File
@@ -0,0 +1,35 @@
FROM python:3.10-slim
WORKDIR /AstrBot
COPY . /AstrBot/
RUN apt-get update && apt-get install -y --no-install-recommends \
gcc \
build-essential \
python3-dev \
libffi-dev \
libssl-dev \
curl \
unzip \
ca-certificates \
bash \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Installation of Node.js
ENV NVM_DIR="/root/.nvm"
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.2/install.sh | bash && \
. "$NVM_DIR/nvm.sh" && \
nvm install 22 && \
nvm use 22
RUN /bin/bash -c ". \"$NVM_DIR/nvm.sh\" && node -v && npm -v"
RUN python -m pip install uv
RUN uv pip install -r requirements.txt --no-cache-dir --system
RUN uv pip install socksio uv pyffmpeg --no-cache-dir --system
EXPOSE 6185
EXPOSE 6186
CMD ["python", "main.py"]
+111 -131
View File
@@ -1,67 +1,48 @@
<img width="430" height="31" alt="image" src="https://github.com/user-attachments/assets/474c822c-fab7-41be-8c23-6dae252823ed" /><p align="center">
![AstrBot-Logo-Simplified](https://github.com/user-attachments/assets/ffd99b6b-3272-4682-beaa-6fe74250f7d9) ![AstrBot-Logo-Simplified](https://github.com/user-attachments/assets/ffd99b6b-3272-4682-beaa-6fe74250f7d9)
</p>
<div align="center"> <div align="center">
_✨ 易上手的多平台 LLM 聊天机器人及开发框架 ✨_
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_en.md">English</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_ja.md">日本語</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_zh-TW.md">繁體中文</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_fr.md">Français</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_ru.md">Русский</a>
<div>
<a href="https://trendshift.io/repositories/12875" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12875" alt="Soulter%2FAstrBot | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> <a href="https://trendshift.io/repositories/12875" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12875" alt="Soulter%2FAstrBot | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
<a href="https://hellogithub.com/repository/AstrBotDevs/AstrBot" target="_blank"><img src="https://api.hellogithub.com/v1/widgets/recommend.svg?rid=d127d50cd5e54c5382328acc3bb25483&claim_uid=ZO9by7qCXgSd6Lp&t=2" alt="FeaturedHelloGitHub" style="width: 250px; height: 54px;" width="250" height="54" /></a>
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/Soulter/AstrBot?style=for-the-badge&color=76bad9)](https://github.com/Soulter/AstrBot/releases/latest)
<img src="https://img.shields.io/badge/python-3.10+-blue.svg?style=for-the-badge&color=76bad9" alt="python">
<a href="https://hub.docker.com/r/soulter/astrbot"><img alt="Docker pull" src="https://img.shields.io/docker/pulls/soulter/astrbot.svg?style=for-the-badge&color=76bad9"/></a>
<a href="https://qm.qq.com/cgi-bin/qm/qr?k=wtbaNx7EioxeaqS9z7RQWVXPIxg2zYr7&jump_from=webapi&authKey=vlqnv/AV2DbJEvGIcxdlNSpfxVy+8vVqijgreRdnVKOaydpc+YSw4MctmEbr0k5"><img alt="QQ_community" src="https://img.shields.io/badge/QQ群-775869627-purple?style=for-the-badge&color=76bad9"></a>
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
[![wakatime](https://wakatime.com/badge/user/915e5316-99c6-4563-a483-ef186cf000c9/project/018e705a-a1a7-409a-a849-3013485e6c8e.svg?style=for-the-badge&color=76bad9)](https://wakatime.com/badge/user/915e5316-99c6-4563-a483-ef186cf000c9/project/018e705a-a1a7-409a-a849-3013485e6c8e)
![Dynamic JSON Badge](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.soulter.top%2Fastrbot%2Fstats&query=v&label=7日消息量&cacheSeconds=3600&style=for-the-badge&color=3b618e)
![Dynamic JSON Badge](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.soulter.top%2Fastrbot%2Fplugin-num&query=%24.result&suffix=%E4%B8%AA&style=for-the-badge&label=%E6%8F%92%E4%BB%B6%E5%B8%82%E5%9C%BA&cacheSeconds=3600)
<a href="https://github.com/Soulter/AstrBot/blob/master/README_en.md">English</a>
<a href="https://github.com/Soulter/AstrBot/blob/master/README_ja.md">日本語</a>
<a href="https://astrbot.app/">查看文档</a>
<a href="https://github.com/Soulter/AstrBot/issues">问题提交</a>
</div> </div>
<br> AstrBot 是一个开源的一站式 Agentic 聊天机器人平台及开发框架。
<div> ## ✨ 主要功能
<img src="https://img.shields.io/github/v/release/AstrBotDevs/AstrBot?color=76bad9" href="https://github.com/AstrBotDevs/AstrBot/releases/latest">
<img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="python">
<img src="https://deepwiki.com/badge.svg" href="https://deepwiki.com/AstrBotDevs/AstrBot">
<a href="https://zread.ai/AstrBotDevs/AstrBot" target="_blank"><img src="https://img.shields.io/badge/Ask_Zread-_.svg?style=flat&color=00b0aa&labelColor=000000&logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB3aWR0aD0iMTYiIGhlaWdodD0iMTYiIHZpZXdCb3g9IjAgMCAxNiAxNiIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPHBhdGggZD0iTTQuOTYxNTYgMS42MDAxSDIuMjQxNTZDMS44ODgxIDEuNjAwMSAxLjYwMTU2IDEuODg2NjQgMS42MDE1NiAyLjI0MDFWNC45NjAxQzEuNjAxNTYgNS4zMTM1NiAxLjg4ODEgNS42MDAxIDIuMjQxNTYgNS42MDAxSDQuOTYxNTZDNS4zMTUwMiA1LjYwMDEgNS42MDE1NiA1LjMxMzU2IDUuNjAxNTYgNC45NjAxVjIuMjQwMUM1LjYwMTU2IDEuODg2NjQgNS4zMTUwMiAxLjYwMDEgNC45NjE1NiAxLjYwMDFaIiBmaWxsPSIjZmZmIi8%2BCjxwYXRoIGQ9Ik00Ljk2MTU2IDEwLjM5OTlIMi4yNDE1NkMxLjg4ODEgMTAuMzk5OSAxLjYwMTU2IDEwLjY4NjQgMS42MDE1NiAxMS4wMzk5VjEzLjc1OTlDMS42MDE1NiAxNC4xMTM0IDEuODg4MSAxNC4zOTk5IDIuMjQxNTYgMTQuMzk5OUg0Ljk2MTU2QzUuMzE1MDIgMTQuMzk5OSA1LjYwMTU2IDE0LjExMzQgNS42MDE1NiAxMy43NTk5VjExLjAzOTlDNS42MDE1NiAxMC42ODY0IDUuMzE1MDIgMTAuMzk5OSA0Ljk2MTU2IDEwLjM5OTlaIiBmaWxsPSIjZmZmIi8%2BCjxwYXRoIGQ9Ik0xMy43NTg0IDEuNjAwMUgxMS4wMzg0QzEwLjY4NSAxLjYwMDEgMTAuMzk4NCAxLjg4NjY0IDEwLjM5ODQgMi4yNDAxVjQuOTYwMUMxMC4zOTg0IDUuMzEzNTYgMTAuNjg1IDUuNjAwMSAxMS4wMzg0IDUuNjAwMUgxMy43NTg0QzE0LjExMTkgNS42MDAxIDE0LjM5ODQgNS4zMTM1NiAxNC4zOTg0IDQuOTYwMVYyLjI0MDFDMTQuMzk4NCAxLjg4NjY0IDE0LjExMTkgMS42MDAxIDEzLjc1ODQgMS42MDAxWiIgZmlsbD0iI2ZmZiIvPgo8cGF0aCBkPSJNNCAxMkwxMiA0TDQgMTJaIiBmaWxsPSIjZmZmIi8%2BCjxwYXRoIGQ9Ik00IDEyTDEyIDQiIHN0cm9rZT0iI2ZmZiIgc3Ryb2tlLXdpZHRoPSIxLjUiIHN0cm9rZS1saW5lY2FwPSJyb3VuZCIvPgo8L3N2Zz4K&logoColor=ffffff" alt="zread"/></a>
<a href="https://hub.docker.com/r/soulter/astrbot"><img alt="Docker pull" src="https://img.shields.io/docker/pulls/soulter/astrbot.svg?color=76bad9"/></a>
<img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.soulter.top%2Fastrbot%2Fplugin-num&query=%24.result&suffix=%E4%B8%AA&label=%E6%8F%92%E4%BB%B6%E5%B8%82%E5%9C%BA&cacheSeconds=3600">
<img src="https://gitcode.com/Soulter/AstrBot/star/badge.svg" href="https://gitcode.com/Soulter/AstrBot">
</div>
<br> 1. **大模型对话**。支持接入多种大模型服务。支持多模态、工具调用、MCP、原生知识库、人设等功能。
2. **多消息平台支持**。支持接入 QQ、企业微信、微信公众号、飞书、Telegram、钉钉、Discord、KOOK 等平台。支持速率限制、白名单、百度内容审核。
3. **Agent**。完善适配的 Agentic 能力。支持多轮工具调用、内置沙盒代码执行器、网页搜索等功能。
4. **插件扩展**。深度优化的插件机制,支持[开发插件](https://astrbot.app/dev/plugin.html)扩展功能,社区插件生态丰富。
5. **WebUI**。可视化配置和管理机器人,功能齐全。
<a href="https://astrbot.app/">文档</a> ## ✨ 使用方式
<a href="https://blog.astrbot.app/">Blog</a>
<a href="https://astrbot.featurebase.app/roadmap">路线图</a>
<a href="https://github.com/AstrBotDevs/AstrBot/issues">问题提交</a>
</div>
AstrBot 是一个开源的一站式 Agent 聊天机器人平台,可接入主流即时通讯软件,为个人、开发者和团队打造可靠、可扩展的对话式智能基础设施。无论是个人 AI 伙伴、智能客服、自动化助手,还是企业知识库,AstrBot 都能在你的即时通讯软件平台的工作流中快速构建生产可用的 AI 应用。 #### Docker 部署
<img width="1776" height="1080" alt="image" src="https://github.com/user-attachments/assets/00782c4c-4437-4d97-aabc-605e3738da5c" />
## 主要功能
1. 💯 免费 & 开源。
1. ✨ AI 大模型对话,多模态,Agent,MCP,知识库,人格设定。
2. 🤖 支持接入 Dify、阿里云百炼、Coze 等智能体平台。
2. 🌐 多平台,支持 QQ、企业微信、飞书、钉钉、微信公众号、Telegram、Slack 以及[更多](#支持的消息平台)。
3. 📦 插件扩展,已有近 800 个插件可一键安装。
5. 💻 WebUI 支持。
6. 🌐 国际化(i18n)支持。
## 快速开始
#### Docker 部署(推荐 🥳)
推荐使用 Docker / Docker Compose 方式部署 AstrBot。 推荐使用 Docker / Docker Compose 方式部署 AstrBot。
请参阅官方文档 [使用 Docker 部署 AstrBot](https://astrbot.app/deploy/astrbot/docker.html#%E4%BD%BF%E7%94%A8-docker-%E9%83%A8%E7%BD%B2-astrbot) 。 请参阅官方文档 [使用 Docker 部署 AstrBot](https://astrbot.app/deploy/astrbot/docker.html#%E4%BD%BF%E7%94%A8-docker-%E9%83%A8%E7%BD%B2-astrbot) 。
#### uv 部署
```bash
uvx astrbot
```
#### 宝塔面板部署 #### 宝塔面板部署
AstrBot 与宝塔面板合作,已上架至宝塔面板。 AstrBot 与宝塔面板合作,已上架至宝塔面板。
@@ -84,7 +65,7 @@ AstrBot 已由雨云官方上架至云应用平台,可一键部署。
社区贡献的部署方式。 社区贡献的部署方式。
[![Run on Repl.it](https://repl.it/badge/github/AstrBotDevs/AstrBot)](https://repl.it/github/AstrBotDevs/AstrBot) [![Run on Repl.it](https://repl.it/badge/github/Soulter/AstrBot)](https://repl.it/github/Soulter/AstrBot)
#### Windows 一键安装器部署 #### Windows 一键安装器部署
@@ -98,7 +79,9 @@ AstrBot 已由雨云官方上架至云应用平台,可一键部署。
#### 手动部署 #### 手动部署
首先安装 uv > 推荐使用 `uv`。
首先,安装 uv
```bash ```bash
pip install uv pip install uv
@@ -113,73 +96,53 @@ uv run main.py
或者请参阅官方文档 [通过源码部署 AstrBot](https://astrbot.app/deploy/astrbot/cli.html) 。 或者请参阅官方文档 [通过源码部署 AstrBot](https://astrbot.app/deploy/astrbot/cli.html) 。
## 支持的消息平台 ## 消息平台支持情况
**官方维护** | 平台 | 支持性 |
| -------- | ------- |
| QQ(官方机器人接口) | ✔ |
| QQ(OneBot) | ✔ |
| Telegram | ✔ |
| 企业微信 | ✔ |
| 微信客服 | ✔ |
| 微信公众号 | ✔ |
| 飞书 | ✔ |
| 钉钉 | ✔ |
| Slack | ✔ |
| Discord | ✔ |
| [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter) | ✔ |
| [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat) | ✔ |
| 微信对话开放平台 | 🚧 |
| WhatsApp | 🚧 |
| 小爱音响 | 🚧 |
- QQ (官方平台 & OneBot) ## ⚡ 提供商支持情况
- Telegram
- 企微应用 & 企微智能机器人
- 微信客服 & 微信公众号
- 飞书
- 钉钉
- Slack
- Discord
- Satori
- Misskey
- Whatsapp (将支持)
- LINE (将支持)
**社区维护** | 名称 | 支持性 | 类型 | 备注 |
| -------- | ------- | ------- | ------- |
| OpenAI API | ✔ | 文本生成 | 也支持 DeepSeek、Gemini、Kimi、xAI 等兼容 OpenAI API 的服务 |
| Claude API | ✔ | 文本生成 | |
| Google Gemini API | ✔ | 文本生成 | |
| Dify | ✔ | LLMOps | |
| 阿里云百炼应用 | ✔ | LLMOps | |
| Ollama | ✔ | 模型加载器 | 本地部署 DeepSeek、Llama 等开源语言模型 |
| LM Studio | ✔ | 模型加载器 | 本地部署 DeepSeek、Llama 等开源语言模型 |
| LLMTuner | ✔ | 模型加载器 | 本地加载 lora 等微调模型 |
| [优云智算](https://www.compshare.cn/?ytag=GPU_YY-gh_astrbot&referral_code=FV7DcGowN4hB5UuXKgpE74) | ✔ | 模型 API 及算力服务平台 | |
| [302.AI](https://share.302.ai/rr1M3l) | ✔ | 模型 API 服务平台 | |
| 硅基流动 | ✔ | 模型 API 服务平台 | |
| PPIO 派欧云 | ✔ | 模型 API 服务平台 | |
| OneAPI | ✔ | LLM 分发系统 | |
| Whisper | ✔ | 语音转文本 | 支持 API、本地部署 |
| SenseVoice | ✔ | 语音转文本 | 本地部署 |
| OpenAI TTS API | ✔ | 文本转语音 | |
| GSVI | ✔ | 文本转语音 | GPT-Sovits-Inference |
| GPT-SoVITs | ✔ | 文本转语音 | GPT-Sovits-Inference |
| FishAudio | ✔ | 文本转语音 | GPT-Sovits 作者参与的项目 |
| Edge TTS | ✔ | 文本转语音 | Edge 浏览器的免费 TTS |
| 阿里云百炼 TTS | ✔ | 文本转语音 | |
| Azure TTS | ✔ | 文本转语音 | Microsoft Azure TTS |
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
- [Bilibili 私信](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
- [wxauto](https://github.com/luosheng520qaq/wxauto-repost-onebotv11)
## 支持的模型服务
**大模型服务**
- OpenAI 及兼容服务
- Anthropic
- Google Gemini
- Moonshot AI
- 智谱 AI
- DeepSeek
- Ollama (本地部署)
- LM Studio (本地部署)
- [优云智算](https://www.compshare.cn/?ytag=GPU_YY-gh_astrbot&referral_code=FV7DcGowN4hB5UuXKgpE74)
- [302.AI](https://share.302.ai/rr1M3l)
- [小马算力](https://www.tokenpony.cn/3YPyf)
- [硅基流动](https://docs.siliconflow.cn/cn/usercases/use-siliconcloud-in-astrbot)
- [PPIO 派欧云](https://ppio.com/user/register?invited_by=AIOONE)
- ModelScope
- OneAPI
**LLMOps 平台**
- Dify
- 阿里云百炼应用
- Coze
**语音转文本服务**
- OpenAI Whisper
- SenseVoice
**文本转语音服务**
- OpenAI TTS
- Gemini TTS
- GPT-Sovits-Inference
- GPT-Sovits
- FishAudio
- Edge TTS
- 阿里云百炼 TTS
- Azure TTS
- Minimax TTS
- 火山引擎 TTS
## ❤️ 贡献 ## ❤️ 贡献
@@ -194,29 +157,43 @@ uv run main.py
AstrBot 使用 `ruff` 进行代码格式化和检查。 AstrBot 使用 `ruff` 进行代码格式化和检查。
```bash ```bash
git clone https://github.com/AstrBotDevs/AstrBot git clone https://github.com/Soulter/AstrBot
pip install pre-commit pip install pre-commit
pre-commit install pre-commit install
``` ```
## 🌍 社区 ## 🌟 支持
### QQ 群组 - Star 这个项目!
- 在[爱发电](https://afdian.com/a/soulter)支持我!
- 1 群:322154837 ## ✨ Demo
- 3 群:630166526
- 5 群:822130018
- 6 群:753075035
- 7 群:743746109
- 开发者群:975206796
### Telegram 群组 <details><summary>👉 点击展开多张 Demo 截图 👈</summary>
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a> <div align='center'>
### Discord 群组 <img src="https://github.com/user-attachments/assets/4ee688d9-467d-45c8-99d6-368f9a8a92d8" width="600">
_✨基于 Docker 的沙箱化代码执行器(Beta 测试)✨_
<img src="https://github.com/user-attachments/assets/0378f407-6079-4f64-ae4c-e97ab20611d2" height=500>
_✨ 多模态、网页搜索、长文本转图片(可配置) ✨_
<img src="https://github.com/user-attachments/assets/e137a9e1-340a-4bf2-bb2b-771132780735" height=150>
<img src="https://github.com/user-attachments/assets/480f5e82-cf6a-4955-a869-0d73137aa6e1" height=150>
_✨ 插件系统——部分插件展示 ✨_
<img src="https://github.com/user-attachments/assets/0cdbf564-2f59-4da5-b524-ce0e7ef3d978" width=600>
_✨ WebUI ✨_
</div>
</details>
<a href="https://discord.gg/hAVk6tgV36"><img alt="Discord_community" src="https://img.shields.io/badge/Discord-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
## ❤️ Special Thanks ## ❤️ Special Thanks
@@ -226,21 +203,24 @@ pre-commit install
<img src="https://contrib.rocks/image?repo=AstrBotDevs/AstrBot" /> <img src="https://contrib.rocks/image?repo=AstrBotDevs/AstrBot" />
</a> </a>
此外,本项目的诞生离不开以下开源项目的帮助 此外,本项目的诞生离不开以下开源项目:
- [NapNeko/NapCatQQ](https://github.com/NapNeko/NapCatQQ) - 伟大的猫猫框架 - [NapNeko/NapCatQQ](https://github.com/NapNeko/NapCatQQ) - 伟大的猫猫框架
- [wechatpy/wechatpy](https://github.com/wechatpy/wechatpy)
## ⭐ Star History ## ⭐ Star History
> [!TIP] > [!TIP]
> 如果本项目对您的生活 / 工作产生了帮助,或者您关注本项目的未来发展,请给项目 Star,这是我维护这个开源项目的动力 <3 > 如果本项目对您的生活 / 工作产生了帮助,或者您关注本项目的未来发展,请给项目 Star,这是我维护这个开源项目的动力 <3
<div align="center"> <div align="center">
[![Star History Chart](https://api.star-history.com/svg?repos=astrbotdevs/astrbot&type=Date)](https://star-history.com/#astrbotdevs/astrbot&Date) [![Star History Chart](https://api.star-history.com/svg?repos=soulter/astrbot&type=Date)](https://star-history.com/#soulter/astrbot&Date)
</div> </div>
</details> ![10k-star-banner-credit-by-kevin](https://github.com/user-attachments/assets/c97fc5fb-20b9-4bc8-9998-c20b930ab097)
_私は、高性能ですから!_ _私は、高性能ですから!_
+140 -205
View File
@@ -1,247 +1,182 @@
![AstrBot-Logo-Simplified](https://github.com/user-attachments/assets/ffd99b6b-3272-4682-beaa-6fe74250f7d9) <p align="center">
![6e1279651f16d7fdf4727558b72bbaf1](https://github.com/user-attachments/assets/ead4c551-fc3c-48f7-a6f7-afbfdb820512)
</p> </p>
<div align="center"> <div align="center">
<br> _✨ Easy-to-use Multi-platform LLM Chatbot & Development Framework ✨_
<div>
<a href="https://trendshift.io/repositories/12875" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12875" alt="Soulter%2FAstrBot | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> <a href="https://trendshift.io/repositories/12875" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12875" alt="Soulter%2FAstrBot | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
<a href="https://hellogithub.com/repository/AstrBotDevs/AstrBot" target="_blank"><img src="https://api.hellogithub.com/v1/widgets/recommend.svg?rid=d127d50cd5e54c5382328acc3bb25483&claim_uid=ZO9by7qCXgSd6Lp&t=2" alt="FeaturedHelloGitHub" style="width: 250px; height: 54px;" width="250" height="54" /></a>
</div>
<br> [![GitHub release (latest by date)](https://img.shields.io/github/v/release/Soulter/AstrBot)](https://github.com/Soulter/AstrBot/releases/latest)
<img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="python">
<div> <a href="https://hub.docker.com/r/soulter/astrbot"><img alt="Docker pull" src="https://img.shields.io/docker/pulls/soulter/astrbot.svg"/></a>
<img src="https://img.shields.io/github/v/release/AstrBotDevs/AstrBot?style=for-the-badge&color=76bad9" href="https://github.com/AstrBotDevs/AstrBot/releases/latest"> <a href="https://qm.qq.com/cgi-bin/qm/qr?k=wtbaNx7EioxeaqS9z7RQWVXPIxg2zYr7&jump_from=webapi&authKey=vlqnv/AV2DbJEvGIcxdlNSpfxVy+8vVqijgreRdnVKOaydpc+YSw4MctmEbr0k5"><img alt="Static Badge" src="https://img.shields.io/badge/QQ群-630166526-purple"></a>
<img src="https://img.shields.io/badge/python-3.10+-blue.svg?style=for-the-badge&color=76bad9" alt="python"> [![wakatime](https://wakatime.com/badge/user/915e5316-99c6-4563-a483-ef186cf000c9/project/018e705a-a1a7-409a-a849-3013485e6c8e.svg)](https://wakatime.com/badge/user/915e5316-99c6-4563-a483-ef186cf000c9/project/018e705a-a1a7-409a-a849-3013485e6c8e)
<a href="https://hub.docker.com/r/soulter/astrbot"><img alt="Docker pull" src="https://img.shields.io/docker/pulls/soulter/astrbot.svg?style=for-the-badge&color=76bad9"/></a> ![Dynamic JSON Badge](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.soulter.top%2Fastrbot%2Fstats&query=v&label=7%E6%97%A5%E6%B6%88%E6%81%AF%E4%B8%8A%E8%A1%8C%E9%87%8F&cacheSeconds=3600)
<a href="https://qm.qq.com/cgi-bin/qm/qr?k=wtbaNx7EioxeaqS9z7RQWVXPIxg2zYr7&jump_from=webapi&authKey=vlqnv/AV2DbJEvGIcxdlNSpfxVy+8vVqijgreRdnVKOaydpc+YSw4MctmEbr0k5"><img alt="QQ_community" src="https://img.shields.io/badge/QQ群-775869627-purple?style=for-the-badge&color=76bad9"></a> [![codecov](https://codecov.io/gh/Soulter/AstrBot/graph/badge.svg?token=FF3P5967B8)](https://codecov.io/gh/Soulter/AstrBot)
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
<img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.soulter.top%2Fastrbot%2Fplugin-num&query=%24.result&suffix=%20plugins&style=for-the-badge&label=Marketplace&cacheSeconds=3600">
</div>
<br>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README.md">中文</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_ja.md">日本語</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_zh-TW.md">繁體中文</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_fr.md">Français</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_ru.md">Русский</a>
<a href="https://astrbot.app/">Documentation</a> <a href="https://astrbot.app/">Documentation</a>
<a href="https://blog.astrbot.app/">Blog</a> <a href="https://github.com/Soulter/AstrBot/issues">Issue Tracking</a>
<a href="https://astrbot.featurebase.app/roadmap">Roadmap</a>
<a href="https://github.com/AstrBotDevs/AstrBot/issues">Issue Tracker</a>
</div> </div>
AstrBot is an open-source all-in-one Agent chatbot platform that integrates with mainstream instant messaging apps. It provides reliable and scalable conversational AI infrastructure for individuals, developers, and teams. Whether you're building a personal AI companion, intelligent customer service, automation assistant, or enterprise knowledge base, AstrBot enables you to quickly build production-ready AI applications within your IM platform workflows. AstrBot is a loosely coupled, asynchronous chatbot and development framework that supports multi-platform deployment, featuring an easy-to-use plugin system and comprehensive Large Language Model (LLM) integration capabilities.
<img width="1776" height="1080" alt="image" src="https://github.com/user-attachments/assets/00782c4c-4437-4d97-aabc-605e3738da5c" /> ## ✨ Key Features
## Key Features 1. **LLM Conversations** - Supports various LLMs including OpenAI API, Google Gemini, Llama, Deepseek, ChatGLM, etc. Enables local model deployment via Ollama/LLMTuner. Features multi-turn dialogues, personality contexts, multimodal capabilities (image understanding), and speech-to-text (Whisper).
2. **Multi-platform Integration** - Supports QQ (OneBot), QQ Channels, WeChat (Gewechat), Feishu, and Telegram. Planned support for DingTalk, Discord, WhatsApp, and Xiaomi Smart Speakers. Includes rate limiting, whitelisting, keyword filtering, and Baidu content moderation.
3. **Agent Capabilities** - Native support for code execution, natural language TODO lists, web search. Integrates with [Dify Platform](https://dify.ai/) for easy access to Dify assistants/knowledge bases/workflows.
4. **Plugin System** - Optimized plugin mechanism with minimal development effort. Supports multiple installed plugins.
5. **Web Dashboard** - Visual configuration management, plugin controls, logging, and WebChat interface for direct LLM interaction.
6. **High Stability & Modularity** - Event bus and pipeline architecture ensures high modularization and loose coupling.
1. 💯 Free & Open Source. > [!TIP]
2. ✨ AI LLM Conversations, Multimodal, Agent, MCP, Knowledge Base, Persona Settings. > Dashboard Demo: [https://demo.astrbot.app/](https://demo.astrbot.app/)
3. 🤖 Supports integration with Dify, Alibaba Cloud Bailian, Coze and other agent platforms. > Username: `astrbot`, Password: `astrbot` (LLM not configured for chat page)
4. 🌐 Multi-Platform: QQ, WeChat Work, Feishu, DingTalk, WeChat Official Accounts, Telegram, Slack, and [more](#supported-messaging-platforms).
5. 📦 Plugin Extensions with nearly 800 plugins available for one-click installation.
6. 💻 WebUI Support.
7. 🌐 Internationalization (i18n) Support.
## Quick Start ## ✨ Deployment
#### Docker Deployment (Recommended 🥳) #### Docker Deployment
We recommend deploying AstrBot using Docker or Docker Compose. See docs: [Deploy with Docker](https://astrbot.app/deploy/astrbot/docker.html#docker-deployment)
Please refer to the official documentation: [Deploy AstrBot with Docker](https://astrbot.app/deploy/astrbot/docker.html#%E4%BD%BF%E7%94%A8-docker-%E9%83%A8%E7%BD%B2-astrbot). #### Windows Installer
#### uv Deployment Requires Python (>3.10). See docs: [Windows Installer Guide](https://astrbot.app/deploy/astrbot/windows.html)
```bash #### Replit Deployment
uvx astrbot
```
#### BT-Panel Deployment [![Run on Repl.it](https://repl.it/badge/github/Soulter/AstrBot)](https://repl.it/github/Soulter/AstrBot)
AstrBot has partnered with BT-Panel and is now available in their marketplace.
Please refer to the official documentation: [BT-Panel Deployment](https://astrbot.app/deploy/astrbot/btpanel.html).
#### 1Panel Deployment
AstrBot has been officially listed on the 1Panel marketplace.
Please refer to the official documentation: [1Panel Deployment](https://astrbot.app/deploy/astrbot/1panel.html).
#### Deploy on RainYun
AstrBot has been officially listed on RainYun's cloud application platform with one-click deployment.
[![Deploy on RainYun](https://rainyun-apps.cn-nb1.rains3.com/materials/deploy-on-rainyun-en.svg)](https://app.rainyun.com/apps/rca/store/5994?ref=NjU1ODg0)
#### Deploy on Replit
Community-contributed deployment method.
[![Run on Repl.it](https://repl.it/badge/github/AstrBotDevs/AstrBot)](https://repl.it/github/AstrBotDevs/AstrBot)
#### Windows One-Click Installer
Please refer to the official documentation: [Deploy AstrBot with Windows One-Click Installer](https://astrbot.app/deploy/astrbot/windows.html).
#### CasaOS Deployment #### CasaOS Deployment
Community-contributed deployment method. Community-contributed method.
See docs: [CasaOS Deployment](https://astrbot.app/deploy/astrbot/casaos.html)
Please refer to the official documentation: [CasaOS Deployment](https://astrbot.app/deploy/astrbot/casaos.html).
#### Manual Deployment #### Manual Deployment
First, install uv: See docs: [Source Code Deployment](https://astrbot.app/deploy/astrbot/cli.html)
```bash ## ⚡ Platform Support
pip install uv
```
Install AstrBot via Git Clone: | Platform | Status | Details | Message Types |
| -------------------------------------------------------------- | ------ | ------------------- | ------------------- |
| QQ (Official Bot) | ✔ | Private/Group chats | Text, Images |
| QQ (OneBot) | ✔ | Private/Group chats | Text, Images, Voice |
| WeChat (Personal) | ✔ | Private/Group chats | Text, Images, Voice |
| [Telegram](https://github.com/Soulter/astrbot_plugin_telegram) | ✔ | Private/Group chats | Text, Images |
| [WeChat Work](https://github.com/Soulter/astrbot_plugin_wecom) | ✔ | Private chats | Text, Images, Voice |
| Feishu | ✔ | Group chats | Text, Images |
| WeChat Open Platform | 🚧 | Planned | - |
| Discord | 🚧 | Planned | - |
| WhatsApp | 🚧 | Planned | - |
| Xiaomi Speakers | 🚧 | Planned | - |
```bash ## Provider Support Status
git clone https://github.com/AstrBotDevs/AstrBot && cd AstrBot
uv run main.py
```
Or refer to the official documentation: [Deploy AstrBot from Source](https://astrbot.app/deploy/astrbot/cli.html). | Name | Support | Type | Notes |
|---------------------------|---------|------------------------|-----------------------------------------------------------------------|
| OpenAI API | ✔ | Text Generation | Supports all OpenAI API-compatible services including DeepSeek, Google Gemini, GLM, Moonshot, Alibaba Cloud Bailian, Silicon Flow, xAI, etc. |
| Claude API | ✔ | Text Generation | |
| Google Gemini API | ✔ | Text Generation | |
| Dify | ✔ | LLMOps | |
| DashScope (Alibaba Cloud) | ✔ | LLMOps | |
| Ollama | ✔ | Model Loader | Local deployment for open-source LLMs (DeepSeek, Llama, etc.) |
| LM Studio | ✔ | Model Loader | Local deployment for open-source LLMs (DeepSeek, Llama, etc.) |
| LLMTuner | ✔ | Model Loader | Local loading of fine-tuned models (e.g. LoRA) |
| OneAPI | ✔ | LLM Distribution | |
| Whisper | ✔ | Speech-to-Text | Supports API and local deployment |
| SenseVoice | ✔ | Speech-to-Text | Local deployment |
| OpenAI TTS API | ✔ | Text-to-Speech | |
| Fishaudio | ✔ | Text-to-Speech | Project involving GPT-Sovits author |
## Supported Messaging Platforms # 🦌 Roadmap
**Officially Maintained**
- QQ (Official Platform & OneBot)
- Telegram
- WeChat Work Application & WeChat Work Intelligent Bot
- WeChat Customer Service & WeChat Official Accounts
- Feishu (Lark)
- DingTalk
- Slack
- Discord
- Satori
- Misskey
- WhatsApp (Coming Soon)
- LINE (Coming Soon)
**Community Maintained**
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
- [Bilibili Direct Messages](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
- [wxauto](https://github.com/luosheng520qaq/wxauto-repost-onebotv11)
## Supported Model Services
**LLM Services**
- OpenAI and Compatible Services
- Anthropic
- Google Gemini
- Moonshot AI
- Zhipu AI
- DeepSeek
- Ollama (Self-hosted)
- LM Studio (Self-hosted)
- [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_astrbot&referral_code=FV7DcGowN4hB5UuXKgpE74)
- [302.AI](https://share.302.ai/rr1M3l)
- [TokenPony](https://www.tokenpony.cn/3YPyf)
- [SiliconFlow](https://docs.siliconflow.cn/cn/usecases/use-siliconcloud-in-astrbot)
- [PPIO Cloud](https://ppio.com/user/register?invited_by=AIOONE)
- ModelScope
- OneAPI
**LLMOps Platforms**
- Dify
- Alibaba Cloud Bailian Applications
- Coze
**Speech-to-Text Services**
- OpenAI Whisper
- SenseVoice
**Text-to-Speech Services**
- OpenAI TTS
- Gemini TTS
- GPT-Sovits-Inference
- GPT-Sovits
- FishAudio
- Edge TTS
- Alibaba Cloud Bailian TTS
- Azure TTS
- Minimax TTS
- Volcano Engine TTS
## ❤️ Contributing
Issues and Pull Requests are always welcome! Feel free to submit your changes to this project :)
### How to Contribute
You can contribute by reviewing issues or helping with pull request reviews. Any issues or PRs are welcome to encourage community participation. Of course, these are just suggestions—you can contribute in any way you like. For adding new features, please discuss through an Issue first.
### Development Environment
AstrBot uses `ruff` for code formatting and linting.
```bash
git clone https://github.com/AstrBotDevs/AstrBot
pip install pre-commit
pre-commit install
```
## 🌍 Community
### QQ Groups
- Group 1: 322154837
- Group 3: 630166526
- Group 5: 822130018
- Group 6: 753075035
- Developer Group: 975206796
### Telegram Group
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
### Discord Server
<a href="https://discord.gg/hAVk6tgV36"><img alt="Discord_community" src="https://img.shields.io/badge/Discord-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
## ❤️ Special Thanks
Special thanks to all Contributors and plugin developers for their contributions to AstrBot ❤️
<a href="https://github.com/AstrBotDevs/AstrBot/graphs/contributors">
<img src="https://contrib.rocks/image?repo=AstrBotDevs/AstrBot" />
</a>
Additionally, the birth of this project would not have been possible without the help of the following open-source projects:
- [NapNeko/NapCatQQ](https://github.com/NapNeko/NapCatQQ) - The amazing cat framework
## ⭐ Star History
> [!TIP] > [!TIP]
> If this project has helped you in your life or work, or if you're interested in its future development, please give the project a Star. It's the driving force behind maintaining this open-source project <3 > Suggestions welcome via Issues <3
<div align="center"> - [ ] Ensure feature parity across all platform adapters
- [ ] Optimize plugin APIs
- [ ] Add default TTS services (e.g., GPT-Sovits)
- [ ] Enhance chat features with persistent memory
- [ ] i18n Planning
[![Star History Chart](https://api.star-history.com/svg?repos=astrbotdevs/astrbot&type=Date)](https://star-history.com/#astrbotdevs/astrbot&Date) ## ❤️ Contributions
All Issues/PRs welcome! Simply submit your changes to this project :)
For major features, please discuss via Issues first.
## 🌟 Support
- Star this project!
- Support via [Afdian](https://afdian.com/a/soulter)
- WeChat support: [QR Code](https://drive.soulter.top/f/pYfA/d903f4fa49a496fda3f16d2be9e023b5.png)
## ✨ Demos
> [!NOTE]
> Code executor file I/O currently tested with Napcat(QQ)/Lagrange(QQ)
<div align='center'>
<img src="https://github.com/user-attachments/assets/4ee688d9-467d-45c8-99d6-368f9a8a92d8" width="600">
_✨ Docker-based Sandboxed Code Executor (Beta) ✨_
<img src="https://github.com/user-attachments/assets/0378f407-6079-4f64-ae4c-e97ab20611d2" height=500>
_✨ Multimodal Input, Web Search, Text-to-Image ✨_
<img src="https://github.com/user-attachments/assets/8ec12797-e70f-460a-959e-48eca39ca2bb" height=100>
_✨ Natural Language TODO Lists ✨_
<img src="https://github.com/user-attachments/assets/e137a9e1-340a-4bf2-bb2b-771132780735" height=150>
<img src="https://github.com/user-attachments/assets/480f5e82-cf6a-4955-a869-0d73137aa6e1" height=150>
_✨ Plugin System Showcase ✨_
<img src="https://github.com/user-attachments/assets/592a8630-14c7-4e06-b496-9c0386e4f36c" width=600>
_✨ Web Dashboard ✨_
![webchat](https://drive.soulter.top/f/vlsA/ezgif-5-fb044b2542.gif)
_✨ Built-in Web Chat Interface ✨_
</div> </div>
</details> ## ⭐ Star History
> [!TIP]
> If this project helps you, please give it a star <3
<div align="center">
[![Star History Chart](https://api.star-history.com/svg?repos=soulter/astrbot&type=Date)](https://star-history.com/#soulter/astrbot&Date)
</div>
## Disclaimer
1. Licensed under `AGPL-v3`.
2. WeChat integration uses [Gewechat](https://github.com/Devo919/Gewechat). Use at your own risk with non-critical accounts.
3. Users must comply with local laws and regulations.
<!-- ## ✨ ATRI [Beta]
Available as plugin: [astrbot_plugin_atri](https://github.com/Soulter/astrbot_plugin_atri)
1. Qwen1.5-7B-Chat Lora model fine-tuned with ATRI character data
2. Long-term memory
3. Meme understanding & responses
4. TTS integration
-->
_私は、高性能ですから!_ _私は、高性能ですから!_
-248
View File
@@ -1,248 +0,0 @@
![AstrBot-Logo-Simplified](https://github.com/user-attachments/assets/ffd99b6b-3272-4682-beaa-6fe74250f7d9)
</p>
<div align="center">
<br>
<div>
<a href="https://trendshift.io/repositories/12875" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12875" alt="Soulter%2FAstrBot | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
<a href="https://hellogithub.com/repository/AstrBotDevs/AstrBot" target="_blank"><img src="https://api.hellogithub.com/v1/widgets/recommend.svg?rid=d127d50cd5e54c5382328acc3bb25483&claim_uid=ZO9by7qCXgSd6Lp&t=2" alt="FeaturedHelloGitHub" style="width: 250px; height: 54px;" width="250" height="54" /></a>
</div>
<br>
<div>
<img src="https://img.shields.io/github/v/release/AstrBotDevs/AstrBot?style=for-the-badge&color=76bad9" href="https://github.com/AstrBotDevs/AstrBot/releases/latest">
<img src="https://img.shields.io/badge/python-3.10+-blue.svg?style=for-the-badge&color=76bad9" alt="python">
<a href="https://hub.docker.com/r/soulter/astrbot"><img alt="Docker pull" src="https://img.shields.io/docker/pulls/soulter/astrbot.svg?style=for-the-badge&color=76bad9"/></a>
<a href="https://qm.qq.com/cgi-bin/qm/qr?k=wtbaNx7EioxeaqS9z7RQWVXPIxg2zYr7&jump_from=webapi&authKey=vlqnv/AV2DbJEvGIcxdlNSpfxVy+8vVqijgreRdnVKOaydpc+YSw4MctmEbr0k5"><img alt="QQ_community" src="https://img.shields.io/badge/QQ群-775869627-purple?style=for-the-badge&color=76bad9"></a>
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
<img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.soulter.top%2Fastrbot%2Fplugin-num&query=%24.result&suffix=%20plugins&style=for-the-badge&label=Marketplace&cacheSeconds=3600">
</div>
<br>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README.md">中文</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_en.md">English</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_ja.md">日本語</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_zh-TW.md">繁體中文</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_ru.md">Русский</a>
<a href="https://astrbot.app/">Documentation</a>
<a href="https://blog.astrbot.app/">Blog</a>
<a href="https://astrbot.featurebase.app/roadmap">Feuille de route</a>
<a href="https://github.com/AstrBotDevs/AstrBot/issues">Signaler un problème</a>
</div>
AstrBot est une plateforme de chatbot Agent tout-en-un open source qui s'intègre aux principales applications de messagerie instantanée. Elle fournit une infrastructure d'IA conversationnelle fiable et évolutive pour les particuliers, les développeurs et les équipes. Que vous construisiez un compagnon IA personnel, un service client intelligent, un assistant d'automatisation ou une base de connaissances d'entreprise, AstrBot vous permet de créer rapidement des applications d'IA prêtes pour la production dans les flux de travail de votre plateforme de messagerie.
<img width="1776" height="1080" alt="image" src="https://github.com/user-attachments/assets/00782c4c-4437-4d97-aabc-605e3738da5c" />
## Fonctionnalités principales
1. 💯 Gratuit & Open Source.
2. ✨ Conversations avec LLM IA, Multimodal, Agent, MCP, Base de connaissances, Paramètres de personnalité.
3. 🤖 Prise en charge de l'intégration avec Dify, Alibaba Cloud Bailian, Coze et autres plateformes d'agents.
4. 🌐 Multi-plateforme : QQ, WeChat Work, Feishu, DingTalk, Comptes officiels WeChat, Telegram, Slack, et [plus encore](#plateformes-de-messagerie-prises-en-charge).
5. 📦 Extensions de plugins avec près de 800 plugins disponibles pour une installation en un clic.
6. 💻 Support WebUI.
7. 🌐 Support de l'internationalisation (i18n).
## Démarrage rapide
#### Déploiement Docker (Recommandé 🥳)
Nous recommandons de déployer AstrBot en utilisant Docker ou Docker Compose.
Veuillez consulter la documentation officielle : [Déployer AstrBot avec Docker](https://astrbot.app/deploy/astrbot/docker.html#%E4%BD%BF%E7%94%A8-docker-%E9%83%A8%E7%BD%B2-astrbot).
#### Déploiement uv
```bash
uvx astrbot
```
#### Déploiement BT-Panel
AstrBot s'est associé à BT-Panel et est maintenant disponible sur leur marketplace.
Veuillez consulter la documentation officielle : [Déploiement BT-Panel](https://astrbot.app/deploy/astrbot/btpanel.html).
#### Déploiement 1Panel
AstrBot a été officiellement listé sur le marketplace 1Panel.
Veuillez consulter la documentation officielle : [Déploiement 1Panel](https://astrbot.app/deploy/astrbot/1panel.html).
#### Déployer sur RainYun
AstrBot a été officiellement listé sur la plateforme d'applications cloud de RainYun avec un déploiement en un clic.
[![Deploy on RainYun](https://rainyun-apps.cn-nb1.rains3.com/materials/deploy-on-rainyun-en.svg)](https://app.rainyun.com/apps/rca/store/5994?ref=NjU1ODg0)
#### Déployer sur Replit
Méthode de déploiement contribuée par la communauté.
[![Run on Repl.it](https://repl.it/badge/github/AstrBotDevs/AstrBot)](https://repl.it/github/AstrBotDevs/AstrBot)
#### Installateur Windows en un clic
Veuillez consulter la documentation officielle : [Déployer AstrBot avec l'installateur Windows en un clic](https://astrbot.app/deploy/astrbot/windows.html).
#### Déploiement CasaOS
Méthode de déploiement contribuée par la communauté.
Veuillez consulter la documentation officielle : [Déploiement CasaOS](https://astrbot.app/deploy/astrbot/casaos.html).
#### Déploiement manuel
Tout d'abord, installez uv :
```bash
pip install uv
```
Installez AstrBot via Git Clone :
```bash
git clone https://github.com/AstrBotDevs/AstrBot && cd AstrBot
uv run main.py
```
Ou consultez la documentation officielle : [Déployer AstrBot depuis les sources](https://astrbot.app/deploy/astrbot/cli.html).
## Plateformes de messagerie prises en charge
**Maintenues officiellement**
- QQ (Plateforme officielle & OneBot)
- Telegram
- Application WeChat Work & Bot intelligent WeChat Work
- Service client WeChat & Comptes officiels WeChat
- Feishu (Lark)
- DingTalk
- Slack
- Discord
- Satori
- Misskey
- WhatsApp (Bientôt disponible)
- LINE (Bientôt disponible)
**Maintenues par la communauté**
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
- [Messages directs Bilibili](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
- [wxauto](https://github.com/luosheng520qaq/wxauto-repost-onebotv11)
## Services de modèles pris en charge
**Services LLM**
- OpenAI et services compatibles
- Anthropic
- Google Gemini
- Moonshot AI
- Zhipu AI
- DeepSeek
- Ollama (Auto-hébergé)
- LM Studio (Auto-hébergé)
- [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_astrbot&referral_code=FV7DcGowN4hB5UuXKgpE74)
- [302.AI](https://share.302.ai/rr1M3l)
- [TokenPony](https://www.tokenpony.cn/3YPyf)
- [SiliconFlow](https://docs.siliconflow.cn/cn/usecases/use-siliconcloud-in-astrbot)
- [PPIO Cloud](https://ppio.com/user/register?invited_by=AIOONE)
- ModelScope
- OneAPI
**Plateformes LLMOps**
- Dify
- Applications Alibaba Cloud Bailian
- Coze
**Services de reconnaissance vocale**
- OpenAI Whisper
- SenseVoice
**Services de synthèse vocale**
- OpenAI TTS
- Gemini TTS
- GPT-Sovits-Inference
- GPT-Sovits
- FishAudio
- Edge TTS
- Alibaba Cloud Bailian TTS
- Azure TTS
- Minimax TTS
- Volcano Engine TTS
## ❤️ Contribuer
Les Issues et Pull Requests sont toujours les bienvenues ! N'hésitez pas à soumettre vos modifications à ce projet :)
### Comment contribuer
Vous pouvez contribuer en examinant les issues ou en aidant à la revue des pull requests. Toutes les issues ou PRs sont les bienvenues pour encourager la participation de la communauté. Bien sûr, ce ne sont que des suggestions - vous pouvez contribuer de la manière que vous souhaitez. Pour l'ajout de nouvelles fonctionnalités, veuillez d'abord en discuter via une Issue.
### Environnement de développement
AstrBot utilise `ruff` pour le formatage et le linting du code.
```bash
git clone https://github.com/AstrBotDevs/AstrBot
pip install pre-commit
pre-commit install
```
## 🌍 Communauté
### Groupes QQ
- Groupe 1 : 322154837
- Groupe 3 : 630166526
- Groupe 5 : 822130018
- Groupe 6 : 753075035
- Groupe développeurs : 975206796
### Groupe Telegram
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
### Serveur Discord
<a href="https://discord.gg/hAVk6tgV36"><img alt="Discord_community" src="https://img.shields.io/badge/Discord-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
## ❤️ Remerciements spéciaux
Un grand merci à tous les contributeurs et développeurs de plugins pour leurs contributions à AstrBot ❤️
<a href="https://github.com/AstrBotDevs/AstrBot/graphs/contributors">
<img src="https://contrib.rocks/image?repo=AstrBotDevs/AstrBot" />
</a>
De plus, la naissance de ce projet n'aurait pas été possible sans l'aide des projets open source suivants :
- [NapNeko/NapCatQQ](https://github.com/NapNeko/NapCatQQ) - L'incroyable framework chat
## ⭐ Historique des étoiles
> [!TIP]
> Si ce projet vous a aidé dans votre vie ou votre travail, ou si vous êtes intéressé par son développement futur, veuillez donner une étoile au projet. C'est la force motrice derrière la maintenance de ce projet open source <3
<div align="center">
[![Star History Chart](https://api.star-history.com/svg?repos=astrbotdevs/astrbot&type=Date)](https://star-history.com/#astrbotdevs/astrbot&Date)
</div>
</details>
_私は、高性能ですから!_
+103 -183
View File
@@ -1,247 +1,167 @@
![AstrBot-Logo-Simplified](https://github.com/user-attachments/assets/ffd99b6b-3272-4682-beaa-6fe74250f7d9) <p align="center">
![6e1279651f16d7fdf4727558b72bbaf1](https://github.com/user-attachments/assets/ead4c551-fc3c-48f7-a6f7-afbfdb820512)
</p> </p>
<div align="center"> <div align="center">
<br> _✨ 簡単に使えるマルチプラットフォーム LLM チャットボットおよび開発フレームワーク ✨_
<div>
<a href="https://trendshift.io/repositories/12875" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12875" alt="Soulter%2FAstrBot | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a> <a href="https://trendshift.io/repositories/12875" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12875" alt="Soulter%2FAstrBot | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
<a href="https://hellogithub.com/repository/AstrBotDevs/AstrBot" target="_blank"><img src="https://api.hellogithub.com/v1/widgets/recommend.svg?rid=d127d50cd5e54c5382328acc3bb25483&claim_uid=ZO9by7qCXgSd6Lp&t=2" alt="FeaturedHelloGitHub" style="width: 250px; height: 54px;" width="250" height="54" /></a>
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/Soulter/AstrBot)](https://github.com/Soulter/AstrBot/releases/latest)
<img src="https://img.shields.io/badge/python-3.10+-blue.svg" alt="python">
<a href="https://hub.docker.com/r/soulter/astrbot"><img alt="Docker pull" src="https://img.shields.io/docker/pulls/soulter/astrbot.svg"/></a>
<img alt="Static Badge" src="https://img.shields.io/badge/QQ群-630166526-purple">
[![wakatime](https://wakatime.com/badge/user/915e5316-99c6-4563-a483-ef186cf000c9/project/018e705a-a1a7-409a-a849-3013485e6c8e.svg)](https://wakatime.com/badge/user/915e5316-99c6-4563-a483-ef186cf000c9/project/018e705a-a1a7-409a-a849-3013485e6c8e)
![Dynamic JSON Badge](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.soulter.top%2Fastrbot%2Fstats&query=v&label=7%E6%97%A5%E6%B6%88%E6%81%AF%E4%B8%8A%E8%A1%8C%E9%87%8F&cacheSeconds=3600)
[![codecov](https://codecov.io/gh/Soulter/AstrBot/graph/badge.svg?token=FF3P5967B8)](https://codecov.io/gh/Soulter/AstrBot)
<a href="https://astrbot.app/">ドキュメントを見る</a>
<a href="https://github.com/Soulter/AstrBot/issues">問題を報告する</a>
</div> </div>
<br> AstrBot は、疎結合、非同期、複数のメッセージプラットフォームに対応したデプロイ、使いやすいプラグインシステム、および包括的な大規模言語モデル(LLM)接続機能を備えたチャットボットおよび開発フレームワークです。
<div> ## ✨ 主な機能
<img src="https://img.shields.io/github/v/release/AstrBotDevs/AstrBot?style=for-the-badge&color=76bad9" href="https://github.com/AstrBotDevs/AstrBot/releases/latest">
<img src="https://img.shields.io/badge/python-3.10+-blue.svg?style=for-the-badge&color=76bad9" alt="python">
<a href="https://hub.docker.com/r/soulter/astrbot"><img alt="Docker pull" src="https://img.shields.io/docker/pulls/soulter/astrbot.svg?style=for-the-badge&color=76bad9"/></a>
<a href="https://qm.qq.com/cgi-bin/qm/qr?k=wtbaNx7EioxeaqS9z7RQWVXPIxg2zYr7&jump_from=webapi&authKey=vlqnv/AV2DbJEvGIcxdlNSpfxVy+8vVqijgreRdnVKOaydpc+YSw4MctmEbr0k5"><img alt="QQ_community" src="https://img.shields.io/badge/QQ群-775869627-purple?style=for-the-badge&color=76bad9"></a>
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
<img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.soulter.top%2Fastrbot%2Fplugin-num&query=%24.result&suffix=%E5%80%8B&style=for-the-badge&label=%E3%83%97%E3%83%A9%E3%82%B0%E3%82%A4%E3%83%B3&cacheSeconds=3600">
</div>
<br> 1. **大規模言語モデルの対話**。OpenAI API、Google Gemini、Llama、Deepseek、ChatGLM など、さまざまな大規模言語モデルをサポートし、Ollama、LLMTuner を介してローカルにデプロイされた大規模モデルをサポートします。多輪対話、人格シナリオ、多モーダル機能を備え、画像理解、音声からテキストへの変換(Whisper)をサポートします。
2. **複数のメッセージプラットフォームの接続**。QQOneBot)、QQ チャンネル、Feishu、Telegram への接続をサポートします。今後、DingTalk、Discord、WhatsApp、Xiaoai 音響をサポートする予定です。レート制限、ホワイトリスト、キーワードフィルタリング、Baidu コンテンツ監査をサポートします。
3. **エージェント**。一部のエージェント機能をネイティブにサポートし、コードエグゼキューター、自然言語タスク、ウェブ検索などを提供します。[Dify プラットフォーム](https://dify.ai/)と連携し、Dify スマートアシスタント、ナレッジベース、Dify ワークフローを簡単に接続できます。
4. **プラグインの拡張**。深く最適化されたプラグインメカニズムを備え、[プラグインの開発](https://astrbot.app/dev/plugin.html)をサポートし、機能を拡張できます。複数のプラグインのインストールをサポートします。
5. **ビジュアル管理パネル**。設定の視覚的な変更、プラグイン管理、ログの表示などをサポートし、設定の難易度を低減します。WebChat を統合し、パネル上で大規模モデルと対話できます。
6. **高い安定性と高いモジュール性**。イベントバスとパイプラインに基づくアーキテクチャ設計により、高度にモジュール化され、低結合です。
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README.md">中文</a> > [!TIP]
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_en.md">English</a> > 管理パネルのオンラインデモを体験する: [https://demo.astrbot.app/](https://demo.astrbot.app/)
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_zh-TW.md">繁體中文</a> >
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_fr.md">Français</a> > ユーザー名: `astrbot`, パスワード: `astrbot`。LLM が設定されていないため、チャットページで大規模モデルを使用することはできません。(デモのログインパスワードを変更しないでください 😭)
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_ru.md">Русский</a>
<a href="https://astrbot.app/">ドキュメント</a> ## ✨ 使用方法
<a href="https://blog.astrbot.app/">Blog</a>
<a href="https://astrbot.featurebase.app/roadmap">ロードマップ</a>
<a href="https://github.com/AstrBotDevs/AstrBot/issues">Issue</a>
</div>
AstrBot は、主要なインスタントメッセージングアプリと統合できるオープンソースのオールインワン Agent チャットボットプラットフォームです。個人、開発者、チームに信頼性が高くスケーラブルな会話型 AI インフラストラクチャを提供します。パーソナル AI コンパニオン、インテリジェントカスタマーサービス、オートメーションアシスタント、エンタープライズナレッジベースなど、AstrBot を使用すると、IM プラットフォームのワークフロー内で本番環境対応の AI アプリケーションを迅速に構築できます。 #### Docker デプロイ
<img width="1776" height="1080" alt="image" src="https://github.com/user-attachments/assets/00782c4c-4437-4d97-aabc-605e3738da5c" /> 公式ドキュメント [Docker を使用して AstrBot をデプロイする](https://astrbot.app/deploy/astrbot/docker.html#%E4%BD%BF%E7%94%A8-docker-%E9%83%A8%E7%BD%B2-astrbot) を参照してください。
## 主な機能 #### Windows ワンクリックインストーラーのデプロイ
1. 💯 無料 & オープンソース コンピュータに Python(>3.10)がインストールされている必要があります。公式ドキュメント [Windows ワンクリックインストーラーを使用して AstrBot をデプロイする](https://astrbot.app/deploy/astrbot/windows.html) を参照してください
2. ✨ AI 大規模言語モデル対話、マルチモーダル、Agent、MCP、ナレッジベース、ペルソナ設定。
3. 🤖 Dify、Alibaba Cloud 百炼、Coze などの Agent プラットフォームとの統合をサポート。
4. 🌐 マルチプラットフォーム:QQ、WeChat Work、Feishu、DingTalk、WeChat 公式アカウント、Telegram、Slack、[その他](#サポートされているメッセージプラットフォーム)。
5. 📦 約800個のプラグインをワンクリックでインストール可能なプラグイン拡張機能。
6. 💻 WebUI サポート。
7. 🌐 国際化(i18n)サポート。
## クイックスタート #### Replit デプロイ
#### Docker デプロイ(推奨 🥳) [![Run on Repl.it](https://repl.it/badge/github/Soulter/AstrBot)](https://repl.it/github/Soulter/AstrBot)
Docker / Docker Compose を使用した AstrBot のデプロイを推奨します。
公式ドキュメント [Docker を使用した AstrBot のデプロイ](https://astrbot.app/deploy/astrbot/docker.html#%E4%BD%BF%E7%94%A8-docker-%E9%83%A8%E7%BD%B2-astrbot) をご参照ください。
#### uv デプロイ
```bash
uvx astrbot
```
#### 宝塔パネルデプロイ
AstrBot は宝塔パネルと提携し、宝塔パネルに公開されています。
公式ドキュメント [宝塔パネルデプロイ](https://astrbot.app/deploy/astrbot/btpanel.html) をご参照ください。
#### 1Panel デプロイ
AstrBot は 1Panel 公式により 1Panel パネルに公開されています。
公式ドキュメント [1Panel デプロイ](https://astrbot.app/deploy/astrbot/1panel.html) をご参照ください。
#### 雨云でのデプロイ
AstrBot は雨云公式によりクラウドアプリケーションプラットフォームに公開され、ワンクリックでデプロイ可能です。
[![Deploy on RainYun](https://rainyun-apps.cn-nb1.rains3.com/materials/deploy-on-rainyun-en.svg)](https://app.rainyun.com/apps/rca/store/5994?ref=NjU1ODg0)
#### Replit でのデプロイ
コミュニティ貢献によるデプロイ方法。
[![Run on Repl.it](https://repl.it/badge/github/AstrBotDevs/AstrBot)](https://repl.it/github/AstrBotDevs/AstrBot)
#### Windows ワンクリックインストーラーデプロイ
公式ドキュメント [Windows ワンクリックインストーラーを使用した AstrBot のデプロイ](https://astrbot.app/deploy/astrbot/windows.html) をご参照ください。
#### CasaOS デプロイ #### CasaOS デプロイ
コミュニティ貢献によるデプロイ方法。 コミュニティが提供するデプロイ方法です
公式ドキュメント [CasaOS デプロイ](https://astrbot.app/deploy/astrbot/casaos.html) を参照ください。 公式ドキュメント [ソースコードを使用して AstrBot をデプロイする](https://astrbot.app/deploy/astrbot/casaos.html) を参照してください。
#### 手動デプロイ #### 手動デプロイ
まず uv をインストールします: 公式ドキュメント [ソースコードを使用して AstrBot をデプロイする](https://astrbot.app/deploy/astrbot/cli.html) を参照してください。
```bash ## ⚡ メッセージプラットフォームのサポート状況
pip install uv
```
Git Clone で AstrBot をインストール: | プラットフォーム | サポート状況 | 詳細 | メッセージタイプ |
| -------- | ------- | ------- | ------ |
| QQ(公式ロボットインターフェース) | ✔ | プライベートチャット、グループチャット、QQ チャンネルプライベートチャット、グループチャット | テキスト、画像 |
| QQ(OneBot) | ✔ | プライベートチャット、グループチャット | テキスト、画像、音声 |
| WeChat(個人アカウント) | ✔ | WeChat 個人アカウントのプライベートチャット、グループチャット | テキスト、画像、音声 |
| [Telegram](https://github.com/Soulter/astrbot_plugin_telegram) | ✔ | プライベートチャット、グループチャット | テキスト、画像 |
| [WeChat(企業 WeChat)](https://github.com/Soulter/astrbot_plugin_wecom) | ✔ | プライベートチャット | テキスト、画像、音声 |
| Feishu | ✔ | グループチャット | テキスト、画像 |
| WeChat 対話オープンプラットフォーム | 🚧 | 計画中 | - |
| Discord | 🚧 | 計画中 | - |
| WhatsApp | 🚧 | 計画中 | - |
| Xiaoai 音響 | 🚧 | 計画中 | - |
```bash # 🦌 今後のロードマップ
git clone https://github.com/AstrBotDevs/AstrBot && cd AstrBot
uv run main.py
```
または、公式ドキュメント [ソースコードから AstrBot をデプロイ](https://astrbot.app/deploy/astrbot/cli.html) をご参照ください。 > [!TIP]
> Issue でさらに多くの提案を歓迎します <3
## サポートされているメッセージプラットフォーム - [ ] 現在のすべてのプラットフォームアダプターの機能の一貫性を確保し、改善する
- [ ] プラグインインターフェースの最適化
- [ ] GPT-Sovits などの TTS サービスをデフォルトでサポート
- [ ] "チャット強化" 部分を完成させ、永続的な記憶をサポート
- [ ] i18n の計画
**公式メンテナンス** ## ❤️ 貢献
- QQ (公式プラットフォーム & OneBot) Issue や Pull Request を歓迎します!このプロジェクトに変更を加えるだけです :)
- Telegram
- WeChat Work アプリケーション & WeChat Work インテリジェントボット
- WeChat カスタマーサービス & WeChat 公式アカウント
- Feishu (Lark)
- DingTalk
- Slack
- Discord
- Satori
- Misskey
- WhatsApp (近日対応予定)
- LINE (近日対応予定)
**コミュニティメンテナンス** 新機能の追加については、まず Issue で議論してください。
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter) ## 🌟 サポート
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
- [Bilibili ダイレクトメッセージ](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
- [wxauto](https://github.com/luosheng520qaq/wxauto-repost-onebotv11)
## サポートされているモデルサービス - このプロジェクトに Star を付けてください!
- [愛発電](https://afdian.com/a/soulter)で私をサポートしてください!
- [WeChat](https://drive.soulter.top/f/pYfA/d903f4fa49a496fda3f16d2be9e023b5.png)で私をサポートしてください~
**大規模言語モデルサービス** ## ✨ デモ
- OpenAI および互換サービス > [!NOTE]
- Anthropic > コードエグゼキューターのファイル入力/出力は現在 Napcat(QQ)、Lagrange(QQ) でのみテストされています
- Google Gemini
- Moonshot AI
- 智谱 AI
- DeepSeek
- Ollama (セルフホスト)
- LM Studio (セルフホスト)
- [優云智算](https://www.compshare.cn/?ytag=GPU_YY-gh_astrbot&referral_code=FV7DcGowN4hB5UuXKgpE74)
- [302.AI](https://share.302.ai/rr1M3l)
- [小馬算力](https://www.tokenpony.cn/3YPyf)
- [硅基流動](https://docs.siliconflow.cn/cn/usercases/use-siliconcloud-in-astrbot)
- [PPIO 派欧云](https://ppio.com/user/register?invited_by=AIOONE)
- ModelScope
- OneAPI
**LLMOps プラットフォーム** <div align='center'>
- Dify <img src="https://github.com/user-attachments/assets/4ee688d9-467d-45c8-99d6-368f9a8a92d8" width="600">
- Alibaba Cloud 百炼アプリケーション
- Coze
**音声認識サービス** _✨ Docker ベースのサンドボックス化されたコードエグゼキューター(ベータテスト中)✨_
- OpenAI Whisper <img src="https://github.com/user-attachments/assets/0378f407-6079-4f64-ae4c-e97ab20611d2" height=500>
- SenseVoice
**音声合成サービス** _✨ 多モーダル、ウェブ検索、長文の画像変換(設定可能)✨_
- OpenAI TTS <img src="https://github.com/user-attachments/assets/8ec12797-e70f-460a-959e-48eca39ca2bb" height=100>
- Gemini TTS
- GPT-Sovits-Inference
- GPT-Sovits
- FishAudio
- Edge TTS
- Alibaba Cloud 百炼 TTS
- Azure TTS
- Minimax TTS
- Volcano Engine TTS
## ❤️ コントリビューション _✨ 自然言語タスク ✨_
Issue や Pull Request は大歓迎です!このプロジェクトに変更を送信してください :) <img src="https://github.com/user-attachments/assets/e137a9e1-340a-4bf2-bb2b-771132780735" height=150>
<img src="https://github.com/user-attachments/assets/480f5e82-cf6a-4955-a869-0d73137aa6e1" height=150>
### コントリビュート方法 _✨ プラグインシステム - 一部のプラグインの展示 ✨_
Issue を確認したり、PR(プルリクエスト)のレビューを手伝うことで貢献できます。どんな Issue や PR への参加も歓迎され、コミュニティ貢献を促進します。もちろん、これらは提案に過ぎず、どんな方法でも貢献できます。新機能の追加については、まず Issue で議論してください。 <img src="https://github.com/user-attachments/assets/592a8630-14c7-4e06-b496-9c0386e4f36c" width="600">
### 開発環境 _✨ 管理パネル ✨_
AstrBot はコードのフォーマットとチェックに `ruff` を使用しています。 ![webchat](https://drive.soulter.top/f/vlsA/ezgif-5-fb044b2542.gif)
```bash _✨ 内蔵 Web Chat、オンラインでボットと対話 ✨_
git clone https://github.com/AstrBotDevs/AstrBot
pip install pre-commit
pre-commit install
```
## 🌍 コミュニティ </div>
### QQ グループ
- 1群: 322154837
- 3群: 630166526
- 5群: 822130018
- 6群: 753075035
- 開発者群: 975206796
### Telegram グループ
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
### Discord サーバー
<a href="https://discord.gg/hAVk6tgV36"><img alt="Discord_community" src="https://img.shields.io/badge/Discord-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
## ❤️ Special Thanks
AstrBot への貢献をしていただいたすべてのコントリビューターとプラグイン開発者に特別な感謝を ❤️
<a href="https://github.com/AstrBotDevs/AstrBot/graphs/contributors">
<img src="https://contrib.rocks/image?repo=AstrBotDevs/AstrBot" />
</a>
また、このプロジェクトの誕生は以下のオープンソースプロジェクトの助けなしには実現できませんでした:
- [NapNeko/NapCatQQ](https://github.com/NapNeko/NapCatQQ) - 素晴らしい猫猫フレームワーク
## ⭐ Star History ## ⭐ Star History
> [!TIP] > [!TIP]
> このプロジェクトがあなたの生活や仕事に役立ったり、このプロジェクトの今後の発展に関心がある場合は、プロジェクトに Star をください。これこのオープンソースプロジェクトを維持する原動力です <3 > このプロジェクトがあなたの生活や仕事に役立った場合、またはこのプロジェクトの将来の発展に関心がある場合は、プロジェクトに Star を付けてください。これこのオープンソースプロジェクトを維持するためのモチベーションです <3
<div align="center"> <div align="center">
[![Star History Chart](https://api.star-history.com/svg?repos=astrbotdevs/astrbot&type=Date)](https://star-history.com/#astrbotdevs/astrbot&Date) [![Star History Chart](https://api.star-history.com/svg?repos=soulter/astrbot&type=Date)](https://star-history.com/#soulter/astrbot&Date)
</div> </div>
</details> ## スポンサー
[<img src="https://api.gitsponsors.com/api/badge/img?id=575865240" height="20">](https://api.gitsponsors.com/api/badge/link?p=XEpbdGxlitw/RbcwiTX93UMzNK/jgDYC8NiSzamIPMoKvG2lBFmyXhSS/b0hFoWlBBMX2L5X5CxTDsUdyvcIEHTOfnkXz47UNOZvMwyt5CzbYpq0SEzsSV1OJF1cCo90qC/ZyYKYOWedal3MhZ3ikw==)
## 免責事項
1. このプロジェクトは `AGPL-v3` オープンソースライセンスの下で保護されています。
2. このプロジェクトを使用する際は、現地の法律および規制を遵守してください。
<!-- ## ✨ ATRI [ベータテスト]
この機能はプラグインとしてロードされます。プラグインリポジトリのアドレス:[astrbot_plugin_atri](https://github.com/Soulter/astrbot_plugin_atri)
1. 《ATRI ~ My Dear Moments》の主人公 ATRI のキャラクターセリフを微調整データセットとして使用した `Qwen1.5-7B-Chat Lora` 微調整モデル。
2. 長期記憶
3. ミームの理解と返信
4. TTS
-->
_私は、高性能ですから!_ _私は、高性能ですから!_
-248
View File
@@ -1,248 +0,0 @@
![AstrBot-Logo-Simplified](https://github.com/user-attachments/assets/ffd99b6b-3272-4682-beaa-6fe74250f7d9)
</p>
<div align="center">
<br>
<div>
<a href="https://trendshift.io/repositories/12875" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12875" alt="Soulter%2FAstrBot | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
<a href="https://hellogithub.com/repository/AstrBotDevs/AstrBot" target="_blank"><img src="https://api.hellogithub.com/v1/widgets/recommend.svg?rid=d127d50cd5e54c5382328acc3bb25483&claim_uid=ZO9by7qCXgSd6Lp&t=2" alt="FeaturedHelloGitHub" style="width: 250px; height: 54px;" width="250" height="54" /></a>
</div>
<br>
<div>
<img src="https://img.shields.io/github/v/release/AstrBotDevs/AstrBot?style=for-the-badge&color=76bad9" href="https://github.com/AstrBotDevs/AstrBot/releases/latest">
<img src="https://img.shields.io/badge/python-3.10+-blue.svg?style=for-the-badge&color=76bad9" alt="python">
<a href="https://hub.docker.com/r/soulter/astrbot"><img alt="Docker pull" src="https://img.shields.io/docker/pulls/soulter/astrbot.svg?style=for-the-badge&color=76bad9"/></a>
<a href="https://qm.qq.com/cgi-bin/qm/qr?k=wtbaNx7EioxeaqS9z7RQWVXPIxg2zYr7&jump_from=webapi&authKey=vlqnv/AV2DbJEvGIcxdlNSpfxVy+8vVqijgreRdnVKOaydpc+YSw4MctmEbr0k5"><img alt="QQ_community" src="https://img.shields.io/badge/QQ群-775869627-purple?style=for-the-badge&color=76bad9"></a>
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
<img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.soulter.top%2Fastrbot%2Fplugin-num&query=%24.result&suffix=%20%D0%BF%D0%BB%D0%B0%D0%B3%D0%B8%D0%BD%D0%BE%D0%B2&style=for-the-badge&label=%D0%9C%D0%B0%D0%B3%D0%B0%D0%B7%D0%B8%D0%BD&cacheSeconds=3600">
</div>
<br>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README.md">中文</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_en.md">English</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_ja.md">日本語</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_zh-TW.md">繁體中文</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_fr.md">Français</a>
<a href="https://astrbot.app/">Документация</a>
<a href="https://blog.astrbot.app/">Блог</a>
<a href="https://astrbot.featurebase.app/roadmap">Дорожная карта</a>
<a href="https://github.com/AstrBotDevs/AstrBot/issues">Сообщить о проблеме</a>
</div>
AstrBot — это универсальная платформа Agent-чатботов с открытым исходным кодом, которая интегрируется с основными приложениями для обмена мгновенными сообщениями. Она предоставляет надёжную и масштабируемую инфраструктуру разговорного ИИ для частных лиц, разработчиков и команд. Будь то персональный ИИ-компаньон, интеллектуальная служба поддержки, автоматизированный помощник или корпоративная база знаний — AstrBot позволяет быстро создавать готовые к использованию ИИ-приложения в рабочих процессах вашей платформы обмена сообщениями.
<img width="1776" height="1080" alt="image" src="https://github.com/user-attachments/assets/00782c4c-4437-4d97-aabc-605e3738da5c" />
## Основные возможности
1. 💯 Бесплатно и с открытым исходным кодом.
2. ✨ ИИ-диалоги с LLM, мультимодальность, Agent, MCP, база знаний, настройки личности.
3. 🤖 Поддержка интеграции с Dify, Alibaba Cloud Bailian, Coze и другими платформами агентов.
4. 🌐 Мультиплатформенность: QQ, WeChat Work, Feishu, DingTalk, официальные аккаунты WeChat, Telegram, Slack и [другие](#поддерживаемые-платформы-обмена-сообщениями).
5. 📦 Расширения плагинов с почти 800 плагинами, доступными для установки в один клик.
6. 💻 Поддержка WebUI.
7. 🌐 Поддержка интернационализации (i18n).
## Быстрый старт
#### Развёртывание Docker (Рекомендуется 🥳)
Мы рекомендуем развёртывать AstrBot с помощью Docker или Docker Compose.
См. официальную документацию: [Развёртывание AstrBot с Docker](https://astrbot.app/deploy/astrbot/docker.html#%E4%BD%BF%E7%94%A8-docker-%E9%83%A8%E7%BD%B2-astrbot).
#### Развёртывание uv
```bash
uvx astrbot
```
#### Развёртывание BT-Panel
AstrBot в партнёрстве с BT-Panel теперь доступен на их маркетплейсе.
См. официальную документацию: [Развёртывание BT-Panel](https://astrbot.app/deploy/astrbot/btpanel.html).
#### Развёртывание 1Panel
AstrBot официально размещён на маркетплейсе 1Panel.
См. официальную документацию: [Развёртывание 1Panel](https://astrbot.app/deploy/astrbot/1panel.html).
#### Развёртывание на RainYun
AstrBot официально размещён на облачной платформе приложений RainYun с развёртыванием в один клик.
[![Deploy on RainYun](https://rainyun-apps.cn-nb1.rains3.com/materials/deploy-on-rainyun-en.svg)](https://app.rainyun.com/apps/rca/store/5994?ref=NjU1ODg0)
#### Развёртывание на Replit
Метод развёртывания от сообщества.
[![Run on Repl.it](https://repl.it/badge/github/AstrBotDevs/AstrBot)](https://repl.it/github/AstrBotDevs/AstrBot)
#### Установщик Windows в один клик
См. официальную документацию: [Развёртывание AstrBot с установщиком Windows в один клик](https://astrbot.app/deploy/astrbot/windows.html).
#### Развёртывание CasaOS
Метод развёртывания от сообщества.
См. официальную документацию: [Развёртывание CasaOS](https://astrbot.app/deploy/astrbot/casaos.html).
#### Ручное развёртывание
Сначала установите uv:
```bash
pip install uv
```
Установите AstrBot через Git Clone:
```bash
git clone https://github.com/AstrBotDevs/AstrBot && cd AstrBot
uv run main.py
```
Или см. официальную документацию: [Развёртывание AstrBot из исходного кода](https://astrbot.app/deploy/astrbot/cli.html).
## Поддерживаемые платформы обмена сообщениями
**Официально поддерживаемые**
- QQ (Официальная платформа и OneBot)
- Telegram
- Приложение WeChat Work и интеллектуальный бот WeChat Work
- Служба поддержки WeChat и официальные аккаунты WeChat
- Feishu (Lark)
- DingTalk
- Slack
- Discord
- Satori
- Misskey
- WhatsApp (Скоро)
- LINE (Скоро)
**Поддерживаемые сообществом**
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
- [Личные сообщения Bilibili](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
- [wxauto](https://github.com/luosheng520qaq/wxauto-repost-onebotv11)
## Поддерживаемые сервисы моделей
**Сервисы LLM**
- OpenAI и совместимые сервисы
- Anthropic
- Google Gemini
- Moonshot AI
- Zhipu AI
- DeepSeek
- Ollama (Самостоятельное размещение)
- LM Studio (Самостоятельное размещение)
- [CompShare](https://www.compshare.cn/?ytag=GPU_YY-gh_astrbot&referral_code=FV7DcGowN4hB5UuXKgpE74)
- [302.AI](https://share.302.ai/rr1M3l)
- [TokenPony](https://www.tokenpony.cn/3YPyf)
- [SiliconFlow](https://docs.siliconflow.cn/cn/usecases/use-siliconcloud-in-astrbot)
- [PPIO Cloud](https://ppio.com/user/register?invited_by=AIOONE)
- ModelScope
- OneAPI
**Платформы LLMOps**
- Dify
- Приложения Alibaba Cloud Bailian
- Coze
**Сервисы распознавания речи**
- OpenAI Whisper
- SenseVoice
**Сервисы синтеза речи**
- OpenAI TTS
- Gemini TTS
- GPT-Sovits-Inference
- GPT-Sovits
- FishAudio
- Edge TTS
- Alibaba Cloud Bailian TTS
- Azure TTS
- Minimax TTS
- Volcano Engine TTS
## ❤️ Вклад в проект
Issues и Pull Request всегда приветствуются! Не стесняйтесь отправлять свои изменения в этот проект :)
### Как внести вклад
Вы можете внести вклад, просматривая issues или помогая с ревью pull request. Любые issues или PR приветствуются для поощрения участия сообщества. Конечно, это лишь предложения — вы можете вносить вклад любым удобным для вас способом. Для добавления новых функций сначала обсудите это через Issue.
### Среда разработки
AstrBot использует `ruff` для форматирования и линтинга кода.
```bash
git clone https://github.com/AstrBotDevs/AstrBot
pip install pre-commit
pre-commit install
```
## 🌍 Сообщество
### Группы QQ
- Группа 1: 322154837
- Группа 3: 630166526
- Группа 5: 822130018
- Группа 6: 753075035
- Группа разработчиков: 975206796
### Группа Telegram
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
### Сервер Discord
<a href="https://discord.gg/hAVk6tgV36"><img alt="Discord_community" src="https://img.shields.io/badge/Discord-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
## ❤️ Особая благодарность
Особая благодарность всем контрибьюторам и разработчикам плагинов за их вклад в AstrBot ❤️
<a href="https://github.com/AstrBotDevs/AstrBot/graphs/contributors">
<img src="https://contrib.rocks/image?repo=AstrBotDevs/AstrBot" />
</a>
Кроме того, рождение этого проекта было бы невозможно без помощи следующих проектов с открытым исходным кодом:
- [NapNeko/NapCatQQ](https://github.com/NapNeko/NapCatQQ) - Замечательный кошачий фреймворк
## ⭐ История звёзд
> [!TIP]
> Если этот проект помог вам в жизни или работе, или если вас интересует его будущее развитие, пожалуйста, поставьте проекту звезду. Это движущая сила поддержки этого проекта с открытым исходным кодом <3
<div align="center">
[![Star History Chart](https://api.star-history.com/svg?repos=astrbotdevs/astrbot&type=Date)](https://star-history.com/#astrbotdevs/astrbot&Date)
</div>
</details>
_私は、高性能ですから!_
-248
View File
@@ -1,248 +0,0 @@
![AstrBot-Logo-Simplified](https://github.com/user-attachments/assets/ffd99b6b-3272-4682-beaa-6fe74250f7d9)
</p>
<div align="center">
<br>
<div>
<a href="https://trendshift.io/repositories/12875" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12875" alt="Soulter%2FAstrBot | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
<a href="https://hellogithub.com/repository/AstrBotDevs/AstrBot" target="_blank"><img src="https://api.hellogithub.com/v1/widgets/recommend.svg?rid=d127d50cd5e54c5382328acc3bb25483&claim_uid=ZO9by7qCXgSd6Lp&t=2" alt="FeaturedHelloGitHub" style="width: 250px; height: 54px;" width="250" height="54" /></a>
</div>
<br>
<div>
<img src="https://img.shields.io/github/v/release/AstrBotDevs/AstrBot?style=for-the-badge&color=76bad9" href="https://github.com/AstrBotDevs/AstrBot/releases/latest">
<img src="https://img.shields.io/badge/python-3.10+-blue.svg?style=for-the-badge&color=76bad9" alt="python">
<a href="https://hub.docker.com/r/soulter/astrbot"><img alt="Docker pull" src="https://img.shields.io/docker/pulls/soulter/astrbot.svg?style=for-the-badge&color=76bad9"/></a>
<a href="https://qm.qq.com/cgi-bin/qm/qr?k=wtbaNx7EioxeaqS9z7RQWVXPIxg2zYr7&jump_from=webapi&authKey=vlqnv/AV2DbJEvGIcxdlNSpfxVy+8vVqijgreRdnVKOaydpc+YSw4MctmEbr0k5"><img alt="QQ_community" src="https://img.shields.io/badge/QQ群-775869627-purple?style=for-the-badge&color=76bad9"></a>
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
<img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.soulter.top%2Fastrbot%2Fplugin-num&query=%24.result&suffix=%E5%80%8B&style=for-the-badge&label=%E6%8F%92%E4%BB%B6%E5%B8%82%E5%A0%B4&cacheSeconds=3600">
</div>
<br>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README.md">简体中文</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_en.md">English</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_ja.md">日本語</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_fr.md">Français</a>
<a href="https://github.com/AstrBotDevs/AstrBot/blob/master/README_ru.md">Русский</a>
<a href="https://astrbot.app/">文件</a>
<a href="https://blog.astrbot.app/">Blog</a>
<a href="https://astrbot.featurebase.app/roadmap">路線圖</a>
<a href="https://github.com/AstrBotDevs/AstrBot/issues">問題回報</a>
</div>
AstrBot 是一個開源的一站式 Agent 聊天機器人平台,可接入主流即時通訊軟體,為個人、開發者和團隊打造可靠、可擴展的對話式智慧基礎設施。無論是個人 AI 夥伴、智慧客服、自動化助手,還是企業知識庫,AstrBot 都能在您的即時通訊軟體平台的工作流程中快速構建生產可用的 AI 應用程式。
<img width="1776" height="1080" alt="image" src="https://github.com/user-attachments/assets/00782c4c-4437-4d97-aabc-605e3738da5c" />
## 主要功能
1. 💯 免費 & 開源。
2. ✨ AI 大型模型對話,多模態,Agent,MCP,知識庫,人格設定。
3. 🤖 支援接入 Dify、阿里雲百煉、Coze 等智慧體平台。
4. 🌐 多平台:QQ、企業微信、飛書、釘釘、微信公眾號、Telegram、Slack 以及[更多](#支援的訊息平台)。
5. 📦 外掛擴充,已有近 800 個外掛可一鍵安裝。
6. 💻 WebUI 支援。
7. 🌐 國際化(i18n)支援。
## 快速開始
#### Docker 部署(推薦 🥳)
推薦使用 Docker / Docker Compose 方式部署 AstrBot。
請參閱官方文件 [使用 Docker 部署 AstrBot](https://astrbot.app/deploy/astrbot/docker.html#%E4%BD%BF%E7%94%A8-docker-%E9%83%A8%E7%BD%B2-astrbot)。
#### uv 部署
```bash
uvx astrbot
```
#### 寶塔面板部署
AstrBot 與寶塔面板合作,已上架至寶塔面板。
請參閱官方文件 [寶塔面板部署](https://astrbot.app/deploy/astrbot/btpanel.html)。
#### 1Panel 部署
AstrBot 已由 1Panel 官方上架至 1Panel 面板。
請參閱官方文件 [1Panel 部署](https://astrbot.app/deploy/astrbot/1panel.html)。
#### 在雨雲上部署
AstrBot 已由雨雲官方上架至雲端應用程式平台,可一鍵部署。
[![Deploy on RainYun](https://rainyun-apps.cn-nb1.rains3.com/materials/deploy-on-rainyun-en.svg)](https://app.rainyun.com/apps/rca/store/5994?ref=NjU1ODg0)
#### 在 Replit 上部署
社群貢獻的部署方式。
[![Run on Repl.it](https://repl.it/badge/github/AstrBotDevs/AstrBot)](https://repl.it/github/AstrBotDevs/AstrBot)
#### Windows 一鍵安裝器部署
請參閱官方文件 [使用 Windows 一鍵安裝器部署 AstrBot](https://astrbot.app/deploy/astrbot/windows.html)。
#### CasaOS 部署
社群貢獻的部署方式。
請參閱官方文件 [CasaOS 部署](https://astrbot.app/deploy/astrbot/casaos.html)。
#### 手動部署
首先安裝 uv
```bash
pip install uv
```
透過 Git Clone 安裝 AstrBot
```bash
git clone https://github.com/AstrBotDevs/AstrBot && cd AstrBot
uv run main.py
```
或者請參閱官方文件 [透過原始碼部署 AstrBot](https://astrbot.app/deploy/astrbot/cli.html)。
## 支援的訊息平台
**官方維護**
- QQ(官方平台 & OneBot
- Telegram
- 企微應用 & 企微智慧機器人
- 微信客服 & 微信公眾號
- 飛書
- 釘釘
- Slack
- Discord
- Satori
- Misskey
- Whatsapp(即將支援)
- LINE(即將支援)
**社群維護**
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
- [Bilibili 私訊](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
- [wxauto](https://github.com/luosheng520qaq/wxauto-repost-onebotv11)
## 支援的模型服務
**大型模型服務**
- OpenAI 及相容服務
- Anthropic
- Google Gemini
- Moonshot AI
- 智譜 AI
- DeepSeek
- Ollama(本機部署)
- LM Studio(本機部署)
- [優雲智算](https://www.compshare.cn/?ytag=GPU_YY-gh_astrbot&referral_code=FV7DcGowN4hB5UuXKgpE74)
- [302.AI](https://share.302.ai/rr1M3l)
- [小馬算力](https://www.tokenpony.cn/3YPyf)
- [矽基流動](https://docs.siliconflow.cn/cn/usercases/use-siliconcloud-in-astrbot)
- [PPIO 派歐雲](https://ppio.com/user/register?invited_by=AIOONE)
- ModelScope
- OneAPI
**LLMOps 平台**
- Dify
- 阿里雲百煉應用
- Coze
**語音轉文字服務**
- OpenAI Whisper
- SenseVoice
**文字轉語音服務**
- OpenAI TTS
- Gemini TTS
- GPT-Sovits-Inference
- GPT-Sovits
- FishAudio
- Edge TTS
- 阿里雲百煉 TTS
- Azure TTS
- Minimax TTS
- 火山引擎 TTS
## ❤️ 貢獻
歡迎任何 Issues/Pull Requests!只需要將您的變更提交到此專案 :)
### 如何貢獻
您可以透過檢視問題或協助審核 PR(拉取請求)來貢獻。任何問題或 PR 都歡迎參與,以促進社群貢獻。當然,這些只是建議,您可以以任何方式進行貢獻。對於新功能的新增,請先透過 Issue 討論。
### 開發環境
AstrBot 使用 `ruff` 進行程式碼格式化和檢查。
```bash
git clone https://github.com/AstrBotDevs/AstrBot
pip install pre-commit
pre-commit install
```
## 🌍 社群
### QQ 群組
- 1 群:322154837
- 3 群:630166526
- 5 群:822130018
- 6 群:753075035
- 開發者群:975206796
### Telegram 群組
<a href="https://t.me/+hAsD2Ebl5as3NmY1"><img alt="Telegram_community" src="https://img.shields.io/badge/Telegram-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
### Discord 群組
<a href="https://discord.gg/hAVk6tgV36"><img alt="Discord_community" src="https://img.shields.io/badge/Discord-AstrBot-purple?style=for-the-badge&color=76bad9"></a>
## ❤️ Special Thanks
特別感謝所有 Contributors 和外掛開發者對 AstrBot 的貢獻 ❤️
<a href="https://github.com/AstrBotDevs/AstrBot/graphs/contributors">
<img src="https://contrib.rocks/image?repo=AstrBotDevs/AstrBot" />
</a>
此外,本專案的誕生離不開以下開源專案的幫助:
- [NapNeko/NapCatQQ](https://github.com/NapNeko/NapCatQQ) - 偉大的貓貓框架
## ⭐ Star History
> [!TIP]
> 如果本專案對您的生活 / 工作產生了幫助,或者您關注本專案的未來發展,請給專案 Star,這是我們維護這個開源專案的動力 <3
<div align="center">
[![Star History Chart](https://api.star-history.com/svg?repos=astrbotdevs/astrbot&type=Date)](https://star-history.com/#astrbotdevs/astrbot&Date)
</div>
</details>
_私は、高性能ですから!_
+11 -10
View File
@@ -1,19 +1,20 @@
from astrbot import logger
from astrbot.core import html_renderer, sp
from astrbot.core.agent.tool import FunctionTool, ToolSet
from astrbot.core.agent.tool_executor import BaseFunctionToolExecutor
from astrbot.core.config.astrbot_config import AstrBotConfig from astrbot.core.config.astrbot_config import AstrBotConfig
from astrbot.core.star.register import register_agent as agent from astrbot import logger
from astrbot.core import html_renderer
from astrbot.core import sp
from astrbot.core.star.register import register_llm_tool as llm_tool from astrbot.core.star.register import register_llm_tool as llm_tool
from astrbot.core.star.register import register_agent as agent
from astrbot.core.agent.tool import ToolSet, FunctionTool
from astrbot.core.agent.tool_executor import BaseFunctionToolExecutor
__all__ = [ __all__ = [
"AstrBotConfig", "AstrBotConfig",
"BaseFunctionToolExecutor", "logger",
"FunctionTool",
"ToolSet",
"agent",
"html_renderer", "html_renderer",
"llm_tool", "llm_tool",
"logger", "agent",
"sp", "sp",
"ToolSet",
"FunctionTool",
"BaseFunctionToolExecutor",
] ]
+1 -2
View File
@@ -36,8 +36,7 @@ from astrbot.core.star.config import *
# provider # provider
from astrbot.core.provider import Provider, ProviderMetaData from astrbot.core.provider import Provider, Personality, ProviderMetaData
from astrbot.core.db.po import Personality
# platform # platform
from astrbot.core.platform import ( from astrbot.core.platform import (
+6 -5
View File
@@ -1,17 +1,18 @@
from astrbot.core.message.message_event_result import ( from astrbot.core.message.message_event_result import (
MessageEventResult,
MessageChain,
CommandResult, CommandResult,
EventResultType, EventResultType,
MessageChain,
MessageEventResult,
ResultContentType, ResultContentType,
) )
from astrbot.core.platform import AstrMessageEvent from astrbot.core.platform import AstrMessageEvent
__all__ = [ __all__ = [
"AstrMessageEvent", "MessageEventResult",
"MessageChain",
"CommandResult", "CommandResult",
"EventResultType", "EventResultType",
"MessageChain", "AstrMessageEvent",
"MessageEventResult",
"ResultContentType", "ResultContentType",
] ]
+39 -42
View File
@@ -1,52 +1,49 @@
from astrbot.core.star.filter.custom_filter import CustomFilter
from astrbot.core.star.filter.event_message_type import (
EventMessageType,
EventMessageTypeFilter,
)
from astrbot.core.star.filter.permission import PermissionType, PermissionTypeFilter
from astrbot.core.star.filter.platform_adapter_type import (
PlatformAdapterType,
PlatformAdapterTypeFilter,
)
from astrbot.core.star.register import register_after_message_sent as after_message_sent
from astrbot.core.star.register import register_command as command
from astrbot.core.star.register import register_command_group as command_group
from astrbot.core.star.register import register_custom_filter as custom_filter
from astrbot.core.star.register import register_event_message_type as event_message_type
from astrbot.core.star.register import register_llm_tool as llm_tool
from astrbot.core.star.register import register_on_astrbot_loaded as on_astrbot_loaded
from astrbot.core.star.register import (
register_on_decorating_result as on_decorating_result,
)
from astrbot.core.star.register import register_on_llm_request as on_llm_request
from astrbot.core.star.register import register_on_llm_response as on_llm_response
from astrbot.core.star.register import register_on_platform_loaded as on_platform_loaded
from astrbot.core.star.register import register_permission_type as permission_type
from astrbot.core.star.register import ( from astrbot.core.star.register import (
register_command as command,
register_command_group as command_group,
register_event_message_type as event_message_type,
register_regex as regex,
register_platform_adapter_type as platform_adapter_type, register_platform_adapter_type as platform_adapter_type,
register_permission_type as permission_type,
register_custom_filter as custom_filter,
register_on_astrbot_loaded as on_astrbot_loaded,
register_on_llm_request as on_llm_request,
register_on_llm_response as on_llm_response,
register_llm_tool as llm_tool,
register_on_decorating_result as on_decorating_result,
register_after_message_sent as after_message_sent,
) )
from astrbot.core.star.register import register_regex as regex
from astrbot.core.star.filter.event_message_type import (
EventMessageTypeFilter,
EventMessageType,
)
from astrbot.core.star.filter.platform_adapter_type import (
PlatformAdapterTypeFilter,
PlatformAdapterType,
)
from astrbot.core.star.filter.permission import PermissionTypeFilter, PermissionType
from astrbot.core.star.filter.custom_filter import CustomFilter
__all__ = [ __all__ = [
"CustomFilter",
"EventMessageType",
"EventMessageTypeFilter",
"PermissionType",
"PermissionTypeFilter",
"PlatformAdapterType",
"PlatformAdapterTypeFilter",
"after_message_sent",
"command", "command",
"command_group", "command_group",
"custom_filter",
"event_message_type", "event_message_type",
"llm_tool",
"on_astrbot_loaded",
"on_decorating_result",
"on_llm_request",
"on_llm_response",
"on_platform_loaded",
"permission_type",
"platform_adapter_type",
"regex", "regex",
"platform_adapter_type",
"permission_type",
"EventMessageTypeFilter",
"EventMessageType",
"PlatformAdapterTypeFilter",
"PlatformAdapterType",
"PermissionTypeFilter",
"CustomFilter",
"custom_filter",
"PermissionType",
"on_astrbot_loaded",
"on_llm_request",
"llm_tool",
"on_decorating_result",
"after_message_sent",
"on_llm_response",
] ]
+8 -7
View File
@@ -1,22 +1,23 @@
from astrbot.core.message.components import *
from astrbot.core.platform import ( from astrbot.core.platform import (
AstrBotMessage,
AstrMessageEvent, AstrMessageEvent,
Group, Platform,
AstrBotMessage,
MessageMember, MessageMember,
MessageType, MessageType,
Platform,
PlatformMetadata, PlatformMetadata,
Group,
) )
from astrbot.core.platform.register import register_platform_adapter from astrbot.core.platform.register import register_platform_adapter
from astrbot.core.message.components import *
__all__ = [ __all__ = [
"AstrBotMessage",
"AstrMessageEvent", "AstrMessageEvent",
"Group", "Platform",
"AstrBotMessage",
"MessageMember", "MessageMember",
"MessageType", "MessageType",
"Platform",
"PlatformMetadata", "PlatformMetadata",
"register_platform_adapter", "register_platform_adapter",
"Group",
] ]
+7 -8
View File
@@ -1,18 +1,17 @@
from astrbot.core.db.po import Personality from astrbot.core.provider import Provider, STTProvider, Personality
from astrbot.core.provider import Provider, STTProvider
from astrbot.core.provider.entities import ( from astrbot.core.provider.entities import (
LLMResponse,
ProviderMetaData,
ProviderRequest, ProviderRequest,
ProviderType, ProviderType,
ProviderMetaData,
LLMResponse,
) )
__all__ = [ __all__ = [
"LLMResponse",
"Personality",
"Provider", "Provider",
"ProviderMetaData", "STTProvider",
"Personality",
"ProviderRequest", "ProviderRequest",
"ProviderType", "ProviderType",
"STTProvider", "ProviderMetaData",
"LLMResponse",
] ]
+4 -3
View File
@@ -1,7 +1,8 @@
from astrbot.core.star import Context, Star, StarTools
from astrbot.core.star.config import *
from astrbot.core.star.register import ( from astrbot.core.star.register import (
register_star as register, # 注册插件(Star register_star as register, # 注册插件(Star
) )
__all__ = ["Context", "Star", "StarTools", "register"] from astrbot.core.star import Context, Star, StarTools
from astrbot.core.star.config import *
__all__ = ["register", "Context", "Star", "StarTools"]
+2 -2
View File
@@ -1,7 +1,7 @@
from astrbot.core.utils.session_waiter import ( from astrbot.core.utils.session_waiter import (
SessionController,
SessionWaiter, SessionWaiter,
SessionController,
session_waiter, session_waiter,
) )
__all__ = ["SessionController", "SessionWaiter", "session_waiter"] __all__ = ["SessionWaiter", "SessionController", "session_waiter"]
+1 -1
View File
@@ -1 +1 @@
__version__ = "4.8.0" __version__ = "3.5.23"
+5 -5
View File
@@ -1,11 +1,11 @@
"""AstrBot CLI入口""" """
AstrBot CLI入口
import sys """
import click import click
import sys
from . import __version__ from . import __version__
from .commands import conf, init, plug, run from .commands import init, run, plug, conf
logo_tmpl = r""" logo_tmpl = r"""
___ _______.___________..______ .______ ______ .___________. ___ _______.___________..______ .______ ______ .___________.
+3 -3
View File
@@ -1,6 +1,6 @@
from .cmd_conf import conf
from .cmd_init import init from .cmd_init import init
from .cmd_plug import plug
from .cmd_run import run from .cmd_run import run
from .cmd_plug import plug
from .cmd_conf import conf
__all__ = ["conf", "init", "plug", "run"] __all__ = ["init", "run", "plug", "conf"]
+16 -19
View File
@@ -1,12 +1,9 @@
import hashlib
import json import json
import zoneinfo
from collections.abc import Callable
from typing import Any
import click import click
import hashlib
from ..utils import check_astrbot_root, get_astrbot_root import zoneinfo
from typing import Any, Callable
from ..utils import get_astrbot_root, check_astrbot_root
def _validate_log_level(value: str) -> str: def _validate_log_level(value: str) -> str:
@@ -14,7 +11,7 @@ def _validate_log_level(value: str) -> str:
value = value.upper() value = value.upper()
if value not in ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]: if value not in ["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"]:
raise click.ClickException( raise click.ClickException(
"日志级别必须是 DEBUG/INFO/WARNING/ERROR/CRITICAL 之一", "日志级别必须是 DEBUG/INFO/WARNING/ERROR/CRITICAL 之一"
) )
return value return value
@@ -76,7 +73,7 @@ def _load_config() -> dict[str, Any]:
root = get_astrbot_root() root = get_astrbot_root()
if not check_astrbot_root(root): if not check_astrbot_root(root):
raise click.ClickException( raise click.ClickException(
f"{root}不是有效的 AstrBot 根目录,如需初始化请使用 astrbot init", f"{root}不是有效的 AstrBot 根目录,如需初始化请使用 astrbot init"
) )
config_path = root / "data" / "cmd_config.json" config_path = root / "data" / "cmd_config.json"
@@ -91,7 +88,7 @@ def _load_config() -> dict[str, Any]:
try: try:
return json.loads(config_path.read_text(encoding="utf-8-sig")) return json.loads(config_path.read_text(encoding="utf-8-sig"))
except json.JSONDecodeError as e: except json.JSONDecodeError as e:
raise click.ClickException(f"配置文件解析失败: {e!s}") raise click.ClickException(f"配置文件解析失败: {str(e)}")
def _save_config(config: dict[str, Any]) -> None: def _save_config(config: dict[str, Any]) -> None:
@@ -99,8 +96,7 @@ def _save_config(config: dict[str, Any]) -> None:
config_path = get_astrbot_root() / "data" / "cmd_config.json" config_path = get_astrbot_root() / "data" / "cmd_config.json"
config_path.write_text( config_path.write_text(
json.dumps(config, ensure_ascii=False, indent=2), json.dumps(config, ensure_ascii=False, indent=2), encoding="utf-8-sig"
encoding="utf-8-sig",
) )
@@ -112,7 +108,7 @@ def _set_nested_item(obj: dict[str, Any], path: str, value: Any) -> None:
obj[part] = {} obj[part] = {}
elif not isinstance(obj[part], dict): elif not isinstance(obj[part], dict):
raise click.ClickException( raise click.ClickException(
f"配置路径冲突: {'.'.join(parts[: parts.index(part) + 1])} 不是字典", f"配置路径冲突: {'.'.join(parts[: parts.index(part) + 1])} 不是字典"
) )
obj = obj[part] obj = obj[part]
obj[parts[-1]] = value obj[parts[-1]] = value
@@ -144,6 +140,7 @@ def conf():
- callback_api_base: 回调接口基址 - callback_api_base: 回调接口基址
""" """
pass
@conf.command(name="set") @conf.command(name="set")
@@ -151,7 +148,7 @@ def conf():
@click.argument("value") @click.argument("value")
def set_config(key: str, value: str): def set_config(key: str, value: str):
"""设置配置项的值""" """设置配置项的值"""
if key not in CONFIG_VALIDATORS: if key not in CONFIG_VALIDATORS.keys():
raise click.ClickException(f"不支持的配置项: {key}") raise click.ClickException(f"不支持的配置项: {key}")
config = _load_config() config = _load_config()
@@ -173,17 +170,17 @@ def set_config(key: str, value: str):
except KeyError: except KeyError:
raise click.ClickException(f"未知的配置项: {key}") raise click.ClickException(f"未知的配置项: {key}")
except Exception as e: except Exception as e:
raise click.UsageError(f"设置配置失败: {e!s}") raise click.UsageError(f"设置配置失败: {str(e)}")
@conf.command(name="get") @conf.command(name="get")
@click.argument("key", required=False) @click.argument("key", required=False)
def get_config(key: str | None = None): def get_config(key: str = None):
"""获取配置项的值,不提供key则显示所有可配置项""" """获取配置项的值,不提供key则显示所有可配置项"""
config = _load_config() config = _load_config()
if key: if key:
if key not in CONFIG_VALIDATORS: if key not in CONFIG_VALIDATORS.keys():
raise click.ClickException(f"不支持的配置项: {key}") raise click.ClickException(f"不支持的配置项: {key}")
try: try:
@@ -194,10 +191,10 @@ def get_config(key: str | None = None):
except KeyError: except KeyError:
raise click.ClickException(f"未知的配置项: {key}") raise click.ClickException(f"未知的配置项: {key}")
except Exception as e: except Exception as e:
raise click.UsageError(f"获取配置失败: {e!s}") raise click.UsageError(f"获取配置失败: {str(e)}")
else: else:
click.echo("当前配置:") click.echo("当前配置:")
for key in CONFIG_VALIDATORS: for key in CONFIG_VALIDATORS.keys():
try: try:
value = ( value = (
"********" "********"
+2 -3
View File
@@ -1,5 +1,4 @@
import asyncio import asyncio
from pathlib import Path
import click import click
from filelock import FileLock, Timeout from filelock import FileLock, Timeout
@@ -7,14 +6,14 @@ from filelock import FileLock, Timeout
from ..utils import check_dashboard, get_astrbot_root from ..utils import check_dashboard, get_astrbot_root
async def initialize_astrbot(astrbot_root: Path) -> None: async def initialize_astrbot(astrbot_root) -> None:
"""执行 AstrBot 初始化逻辑""" """执行 AstrBot 初始化逻辑"""
dot_astrbot = astrbot_root / ".astrbot" dot_astrbot = astrbot_root / ".astrbot"
if not dot_astrbot.exists(): if not dot_astrbot.exists():
click.echo(f"Current Directory: {astrbot_root}") click.echo(f"Current Directory: {astrbot_root}")
click.echo( click.echo(
"如果你确认这是 Astrbot root directory, 你需要在当前目录下创建一个 .astrbot 文件标记该目录为 AstrBot 的数据目录。", "如果你确认这是 Astrbot root directory, 你需要在当前目录下创建一个 .astrbot 文件标记该目录为 AstrBot 的数据目录。"
) )
if click.confirm( if click.confirm(
f"请检查当前目录是否正确,确认正确请回车: {astrbot_root}", f"请检查当前目录是否正确,确认正确请回车: {astrbot_root}",
+10 -8
View File
@@ -1,29 +1,31 @@
import re import re
import shutil
from pathlib import Path from pathlib import Path
import click import click
import shutil
from ..utils import ( from ..utils import (
PluginStatus, get_git_repo,
build_plug_list, build_plug_list,
manage_plugin,
PluginStatus,
check_astrbot_root, check_astrbot_root,
get_astrbot_root, get_astrbot_root,
get_git_repo,
manage_plugin,
) )
@click.group() @click.group()
def plug(): def plug():
"""插件管理""" """插件管理"""
pass
def _get_data_path() -> Path: def _get_data_path() -> Path:
base = get_astrbot_root() base = get_astrbot_root()
if not check_astrbot_root(base): if not check_astrbot_root(base):
raise click.ClickException( raise click.ClickException(
f"{base}不是有效的 AstrBot 根目录,如需初始化请使用 astrbot init", f"{base}不是有效的 AstrBot 根目录,如需初始化请使用 astrbot init"
) )
return (base / "data").resolve() return (base / "data").resolve()
@@ -39,7 +41,7 @@ def display_plugins(plugins, title=None, color=None):
desc = p["desc"][:30] + ("..." if len(p["desc"]) > 30 else "") desc = p["desc"][:30] + ("..." if len(p["desc"]) > 30 else "")
click.echo( click.echo(
f"{p['name']:<20} {p['version']:<10} {p['status']:<10} " f"{p['name']:<20} {p['version']:<10} {p['status']:<10} "
f"{p['author']:<15} {desc:<30}", f"{p['author']:<15} {desc:<30}"
) )
@@ -76,7 +78,7 @@ def new(name: str):
f"desc: {desc}\n" f"desc: {desc}\n"
f"version: {version}\n" f"version: {version}\n"
f"author: {author}\n" f"author: {author}\n"
f"repo: {repo}\n", f"repo: {repo}\n"
) )
# 重写 README.md # 重写 README.md
@@ -84,7 +86,7 @@ def new(name: str):
f.write(f"# {name}\n\n{desc}\n\n# 支持\n\n[帮助文档](https://astrbot.app)\n") f.write(f"# {name}\n\n{desc}\n\n# 支持\n\n[帮助文档](https://astrbot.app)\n")
# 重写 main.py # 重写 main.py
with open(plug_path / "main.py", encoding="utf-8") as f: with open(plug_path / "main.py", "r", encoding="utf-8") as f:
content = f.read() content = f.read()
new_content = content.replace( new_content = content.replace(
+6 -5
View File
@@ -1,18 +1,19 @@
import asyncio
import os import os
import sys import sys
import traceback
from pathlib import Path from pathlib import Path
import click import click
import asyncio
import traceback
from filelock import FileLock, Timeout from filelock import FileLock, Timeout
from ..utils import check_astrbot_root, check_dashboard, get_astrbot_root from ..utils import check_dashboard, check_astrbot_root, get_astrbot_root
async def run_astrbot(astrbot_root: Path): async def run_astrbot(astrbot_root: Path):
"""运行 AstrBot""" """运行 AstrBot"""
from astrbot.core import LogBroker, LogManager, db_helper, logger from astrbot.core import logger, LogManager, LogBroker, db_helper
from astrbot.core.initial_loader import InitialLoader from astrbot.core.initial_loader import InitialLoader
await check_dashboard(astrbot_root / "data") await check_dashboard(astrbot_root / "data")
@@ -37,7 +38,7 @@ def run(reload: bool, port: str) -> None:
if not check_astrbot_root(astrbot_root): if not check_astrbot_root(astrbot_root):
raise click.ClickException( raise click.ClickException(
f"{astrbot_root}不是有效的 AstrBot 根目录,如需初始化请使用 astrbot init", f"{astrbot_root}不是有效的 AstrBot 根目录,如需初始化请使用 astrbot init"
) )
os.environ["ASTRBOT_ROOT"] = str(astrbot_root) os.environ["ASTRBOT_ROOT"] = str(astrbot_root)
+6 -6
View File
@@ -1,18 +1,18 @@
from .basic import ( from .basic import (
get_astrbot_root,
check_astrbot_root, check_astrbot_root,
check_dashboard, check_dashboard,
get_astrbot_root,
) )
from .plugin import PluginStatus, build_plug_list, get_git_repo, manage_plugin from .plugin import get_git_repo, manage_plugin, build_plug_list, PluginStatus
from .version_comparator import VersionComparator from .version_comparator import VersionComparator
__all__ = [ __all__ = [
"PluginStatus", "get_astrbot_root",
"VersionComparator",
"build_plug_list",
"check_astrbot_root", "check_astrbot_root",
"check_dashboard", "check_dashboard",
"get_astrbot_root",
"get_git_repo", "get_git_repo",
"manage_plugin", "manage_plugin",
"build_plug_list",
"VersionComparator",
"PluginStatus",
] ]
+13 -22
View File
@@ -21,9 +21,8 @@ def get_astrbot_root() -> Path:
async def check_dashboard(astrbot_root: Path) -> None: async def check_dashboard(astrbot_root: Path) -> None:
"""检查是否安装了dashboard""" """检查是否安装了dashboard"""
from astrbot.core.utils.io import get_dashboard_version, download_dashboard
from astrbot.core.config.default import VERSION from astrbot.core.config.default import VERSION
from astrbot.core.utils.io import download_dashboard, get_dashboard_version
from .version_comparator import VersionComparator from .version_comparator import VersionComparator
try: try:
@@ -38,10 +37,7 @@ async def check_dashboard(astrbot_root: Path) -> None:
): ):
click.echo("正在安装管理面板...") click.echo("正在安装管理面板...")
await download_dashboard( await download_dashboard(
path="data/dashboard.zip", path="data/dashboard.zip", extract_path=str(astrbot_root)
extract_path=str(astrbot_root),
version=f"v{VERSION}",
latest=False,
) )
click.echo("管理面板安装完成") click.echo("管理面板安装完成")
@@ -49,26 +45,21 @@ async def check_dashboard(astrbot_root: Path) -> None:
if VersionComparator.compare_version(VERSION, dashboard_version) <= 0: if VersionComparator.compare_version(VERSION, dashboard_version) <= 0:
click.echo("管理面板已是最新版本") click.echo("管理面板已是最新版本")
return return
try: else:
version = dashboard_version.split("v")[1] try:
click.echo(f"管理面板版本: {version}") version = dashboard_version.split("v")[1]
await download_dashboard( click.echo(f"管理面板版本: {version}")
path="data/dashboard.zip", await download_dashboard(
extract_path=str(astrbot_root), path="data/dashboard.zip", extract_path=str(astrbot_root)
version=f"v{VERSION}", )
latest=False, except Exception as e:
) click.echo(f"下载管理面板失败: {e}")
except Exception as e: return
click.echo(f"下载管理面板失败: {e}")
return
except FileNotFoundError: except FileNotFoundError:
click.echo("初始化管理面板目录...") click.echo("初始化管理面板目录...")
try: try:
await download_dashboard( await download_dashboard(
path=str(astrbot_root / "dashboard.zip"), path=str(astrbot_root / "dashboard.zip"), extract_path=str(astrbot_root)
extract_path=str(astrbot_root),
version=f"v{VERSION}",
latest=False,
) )
click.echo("管理面板初始化完成") click.echo("管理面板初始化完成")
except Exception as e: except Exception as e:
+31 -44
View File
@@ -1,14 +1,14 @@
import shutil import shutil
import tempfile import tempfile
import httpx
import yaml
from enum import Enum from enum import Enum
from io import BytesIO from io import BytesIO
from pathlib import Path from pathlib import Path
from zipfile import ZipFile from zipfile import ZipFile
import click import click
import httpx
import yaml
from .version_comparator import VersionComparator from .version_comparator import VersionComparator
@@ -32,8 +32,7 @@ def get_git_repo(url: str, target_path: Path, proxy: str | None = None):
release_url = f"https://api.github.com/repos/{author}/{repo}/releases" release_url = f"https://api.github.com/repos/{author}/{repo}/releases"
try: try:
with httpx.Client( with httpx.Client(
proxy=proxy if proxy else None, proxy=proxy if proxy else None, follow_redirects=True
follow_redirects=True,
) as client: ) as client:
resp = client.get(release_url) resp = client.get(release_url)
resp.raise_for_status() resp.raise_for_status()
@@ -56,8 +55,7 @@ def get_git_repo(url: str, target_path: Path, proxy: str | None = None):
# 下载并解压 # 下载并解压
with httpx.Client( with httpx.Client(
proxy=proxy if proxy else None, proxy=proxy if proxy else None, follow_redirects=True
follow_redirects=True,
) as client: ) as client:
resp = client.get(download_url) resp = client.get(download_url)
if ( if (
@@ -91,7 +89,6 @@ def load_yaml_metadata(plugin_dir: Path) -> dict:
Returns: Returns:
dict: 包含元数据的字典,如果读取失败则返回空字典 dict: 包含元数据的字典,如果读取失败则返回空字典
""" """
yaml_path = plugin_dir / "metadata.yaml" yaml_path = plugin_dir / "metadata.yaml"
if yaml_path.exists(): if yaml_path.exists():
@@ -110,7 +107,6 @@ def build_plug_list(plugins_dir: Path) -> list:
Returns: Returns:
list: 包含插件信息的字典列表 list: 包含插件信息的字典列表
""" """
# 获取本地插件信息 # 获取本地插件信息
result = [] result = []
@@ -128,17 +124,15 @@ def build_plug_list(plugins_dir: Path) -> list:
if metadata and all( if metadata and all(
k in metadata for k in ["name", "desc", "version", "author", "repo"] k in metadata for k in ["name", "desc", "version", "author", "repo"]
): ):
result.append( result.append({
{ "name": str(metadata.get("name", "")),
"name": str(metadata.get("name", "")), "desc": str(metadata.get("desc", "")),
"desc": str(metadata.get("desc", "")), "version": str(metadata.get("version", "")),
"version": str(metadata.get("version", "")), "author": str(metadata.get("author", "")),
"author": str(metadata.get("author", "")), "repo": str(metadata.get("repo", "")),
"repo": str(metadata.get("repo", "")), "status": PluginStatus.INSTALLED,
"status": PluginStatus.INSTALLED, "local_path": str(plugin_dir),
"local_path": str(plugin_dir), })
},
)
# 获取在线插件列表 # 获取在线插件列表
online_plugins = [] online_plugins = []
@@ -148,17 +142,15 @@ def build_plug_list(plugins_dir: Path) -> list:
resp.raise_for_status() resp.raise_for_status()
data = resp.json() data = resp.json()
for plugin_id, plugin_info in data.items(): for plugin_id, plugin_info in data.items():
online_plugins.append( online_plugins.append({
{ "name": str(plugin_id),
"name": str(plugin_id), "desc": str(plugin_info.get("desc", "")),
"desc": str(plugin_info.get("desc", "")), "version": str(plugin_info.get("version", "")),
"version": str(plugin_info.get("version", "")), "author": str(plugin_info.get("author", "")),
"author": str(plugin_info.get("author", "")), "repo": str(plugin_info.get("repo", "")),
"repo": str(plugin_info.get("repo", "")), "status": PluginStatus.NOT_INSTALLED,
"status": PluginStatus.NOT_INSTALLED, "local_path": None,
"local_path": None, })
},
)
except Exception as e: except Exception as e:
click.echo(f"获取在线插件列表失败: {e}", err=True) click.echo(f"获取在线插件列表失败: {e}", err=True)
@@ -172,8 +164,7 @@ def build_plug_list(plugins_dir: Path) -> list:
) )
if ( if (
VersionComparator.compare_version( VersionComparator.compare_version(
local_plugin["version"], local_plugin["version"], online_plugin["version"]
online_plugin["version"],
) )
< 0 < 0
): ):
@@ -191,10 +182,7 @@ def build_plug_list(plugins_dir: Path) -> list:
def manage_plugin( def manage_plugin(
plugin: dict, plugin: dict, plugins_dir: Path, is_update: bool = False, proxy: str | None = None
plugins_dir: Path,
is_update: bool = False,
proxy: str | None = None,
) -> None: ) -> None:
"""安装或更新插件 """安装或更新插件
@@ -203,7 +191,6 @@ def manage_plugin(
plugins_dir (Path): 插件目录 plugins_dir (Path): 插件目录
is_update (bool, optional): 是否为更新操作. 默认为 False is_update (bool, optional): 是否为更新操作. 默认为 False
proxy (str, optional): 代理服务器地址 proxy (str, optional): 代理服务器地址
""" """
plugin_name = plugin["name"] plugin_name = plugin["name"]
repo_url = plugin["repo"] repo_url = plugin["repo"]
@@ -221,26 +208,26 @@ def manage_plugin(
raise click.ClickException(f"插件 {plugin_name} 未安装,无法更新") raise click.ClickException(f"插件 {plugin_name} 未安装,无法更新")
# 备份现有插件 # 备份现有插件
if is_update and backup_path is not None and backup_path.exists(): if is_update and backup_path.exists():
shutil.rmtree(backup_path) shutil.rmtree(backup_path)
if is_update and backup_path is not None: if is_update:
shutil.copytree(target_path, backup_path) shutil.copytree(target_path, backup_path)
try: try:
click.echo( click.echo(
f"正在从 {repo_url} {'更新' if is_update else '下载'}插件 {plugin_name}...", f"正在从 {repo_url} {'更新' if is_update else '下载'}插件 {plugin_name}..."
) )
get_git_repo(repo_url, target_path, proxy) get_git_repo(repo_url, target_path, proxy)
# 更新成功,删除备份 # 更新成功,删除备份
if is_update and backup_path is not None and backup_path.exists(): if is_update and backup_path.exists():
shutil.rmtree(backup_path) shutil.rmtree(backup_path)
click.echo(f"插件 {plugin_name} {'更新' if is_update else '安装'}成功") click.echo(f"插件 {plugin_name} {'更新' if is_update else '安装'}成功")
except Exception as e: except Exception as e:
if target_path.exists(): if target_path.exists():
shutil.rmtree(target_path, ignore_errors=True) shutil.rmtree(target_path, ignore_errors=True)
if is_update and backup_path is not None and backup_path.exists(): if is_update and backup_path.exists():
shutil.move(backup_path, target_path) shutil.move(backup_path, target_path)
raise click.ClickException( raise click.ClickException(
f"{'更新' if is_update else '安装'}插件 {plugin_name} 时出错: {e}", f"{'更新' if is_update else '安装'}插件 {plugin_name} 时出错: {e}"
) )
+12 -10
View File
@@ -1,4 +1,6 @@
"""拷贝自 astrbot.core.utils.version_comparator""" """
拷贝自 astrbot.core.utils.version_comparator
"""
import re import re
@@ -40,15 +42,15 @@ class VersionComparator:
for i in range(length): for i in range(length):
if v1_parts[i] > v2_parts[i]: if v1_parts[i] > v2_parts[i]:
return 1 return 1
if v1_parts[i] < v2_parts[i]: elif v1_parts[i] < v2_parts[i]:
return -1 return -1
# 比较预发布标签 # 比较预发布标签
if v1_prerelease is None and v2_prerelease is not None: if v1_prerelease is None and v2_prerelease is not None:
return 1 # 没有预发布标签的版本高于有预发布标签的版本 return 1 # 没有预发布标签的版本高于有预发布标签的版本
if v1_prerelease is not None and v2_prerelease is None: elif v1_prerelease is not None and v2_prerelease is None:
return -1 # 有预发布标签的版本低于没有预发布标签的版本 return -1 # 有预发布标签的版本低于没有预发布标签的版本
if v1_prerelease is not None and v2_prerelease is not None: elif v1_prerelease is not None and v2_prerelease is not None:
len_pre = max(len(v1_prerelease), len(v2_prerelease)) len_pre = max(len(v1_prerelease), len(v2_prerelease))
for i in range(len_pre): for i in range(len_pre):
p1 = v1_prerelease[i] if i < len(v1_prerelease) else None p1 = v1_prerelease[i] if i < len(v1_prerelease) else None
@@ -56,21 +58,21 @@ class VersionComparator:
if p1 is None and p2 is not None: if p1 is None and p2 is not None:
return -1 return -1
if p1 is not None and p2 is None: elif p1 is not None and p2 is None:
return 1 return 1
if isinstance(p1, int) and isinstance(p2, str): elif isinstance(p1, int) and isinstance(p2, str):
return -1 return -1
if isinstance(p1, str) and isinstance(p2, int): elif isinstance(p1, str) and isinstance(p2, int):
return 1 return 1
if isinstance(p1, int) and isinstance(p2, int): elif isinstance(p1, int) and isinstance(p2, int):
if p1 > p2: if p1 > p2:
return 1 return 1
if p1 < p2: elif p1 < p2:
return -1 return -1
elif isinstance(p1, str) and isinstance(p2, str): elif isinstance(p1, str) and isinstance(p2, str):
if p1 > p2: if p1 > p2:
return 1 return 1
if p1 < p2: elif p1 < p2:
return -1 return -1
return 0 # 预发布标签完全相同 return 0 # 预发布标签完全相同
+7 -9
View File
@@ -1,14 +1,12 @@
import os import os
from .log import LogManager, LogBroker # noqa
from astrbot.core.config import AstrBotConfig
from astrbot.core.config.default import DB_PATH
from astrbot.core.db.sqlite import SQLiteDatabase
from astrbot.core.file_token_service import FileTokenService
from astrbot.core.utils.pip_installer import PipInstaller
from astrbot.core.utils.shared_preferences import SharedPreferences
from astrbot.core.utils.t2i.renderer import HtmlRenderer from astrbot.core.utils.t2i.renderer import HtmlRenderer
from astrbot.core.utils.shared_preferences import SharedPreferences
from .log import LogBroker, LogManager # noqa from astrbot.core.utils.pip_installer import PipInstaller
from astrbot.core.db.sqlite import SQLiteDatabase
from astrbot.core.config.default import DB_PATH
from astrbot.core.config import AstrBotConfig
from astrbot.core.file_token_service import FileTokenService
from .utils.astrbot_path import get_astrbot_data_path from .utils.astrbot_path import get_astrbot_data_path
# 初始化数据存储文件夹 # 初始化数据存储文件夹
+4 -5
View File
@@ -1,14 +1,13 @@
from dataclasses import dataclass from dataclasses import dataclass
from typing import Generic
from .hooks import BaseAgentRunHooks
from .run_context import TContext
from .tool import FunctionTool from .tool import FunctionTool
from typing import Generic
from .run_context import TContext
from .hooks import BaseAgentRunHooks
@dataclass @dataclass
class Agent(Generic[TContext]): class Agent(Generic[TContext]):
name: str name: str
instructions: str | None = None instructions: str | None = None
tools: list[str | FunctionTool] | None = None tools: list[str, FunctionTool] | None = None
run_hooks: BaseAgentRunHooks[TContext] | None = None run_hooks: BaseAgentRunHooks[TContext] | None = None
+2 -6
View File
@@ -1,18 +1,14 @@
from typing import Generic from typing import Generic
from .tool import FunctionTool
from .agent import Agent from .agent import Agent
from .run_context import TContext from .run_context import TContext
from .tool import FunctionTool
class HandoffTool(FunctionTool, Generic[TContext]): class HandoffTool(FunctionTool, Generic[TContext]):
"""Handoff tool for delegating tasks to another agent.""" """Handoff tool for delegating tasks to another agent."""
def __init__( def __init__(
self, self, agent: Agent[TContext], parameters: dict | None = None, **kwargs
agent: Agent[TContext],
parameters: dict | None = None,
**kwargs,
): ):
self.agent = agent self.agent = agent
super().__init__( super().__init__(
+6 -9
View File
@@ -1,13 +1,12 @@
from typing import Generic
import mcp import mcp
from dataclasses import dataclass
from astrbot.core.agent.tool import FunctionTool
from astrbot.core.provider.entities import LLMResponse
from .run_context import ContextWrapper, TContext from .run_context import ContextWrapper, TContext
from typing import Generic
from astrbot.core.provider.entities import LLMResponse
from astrbot.core.agent.tool import FunctionTool
@dataclass
class BaseAgentRunHooks(Generic[TContext]): class BaseAgentRunHooks(Generic[TContext]):
async def on_agent_begin(self, run_context: ContextWrapper[TContext]): ... async def on_agent_begin(self, run_context: ContextWrapper[TContext]): ...
async def on_tool_start( async def on_tool_start(
@@ -24,7 +23,5 @@ class BaseAgentRunHooks(Generic[TContext]):
tool_result: mcp.types.CallToolResult | None, tool_result: mcp.types.CallToolResult | None,
): ... ): ...
async def on_agent_done( async def on_agent_done(
self, self, run_context: ContextWrapper[TContext], llm_response: LLMResponse
run_context: ContextWrapper[TContext],
llm_response: LLMResponse,
): ... ): ...
+34 -211
View File
@@ -1,44 +1,28 @@
import asyncio import asyncio
import logging import logging
from contextlib import AsyncExitStack
from datetime import timedelta from datetime import timedelta
from typing import Generic from typing import Optional
from contextlib import AsyncExitStack
from tenacity import (
before_sleep_log,
retry,
retry_if_exception_type,
stop_after_attempt,
wait_exponential,
)
from astrbot import logger from astrbot import logger
from astrbot.core.agent.run_context import ContextWrapper
from astrbot.core.utils.log_pipe import LogPipe from astrbot.core.utils.log_pipe import LogPipe
from .run_context import TContext
from .tool import FunctionTool
try: try:
import anyio
import mcp import mcp
from mcp.client.sse import sse_client from mcp.client.sse import sse_client
except (ModuleNotFoundError, ImportError): except (ModuleNotFoundError, ImportError):
logger.warning( logger.warning("警告: 缺少依赖库 'mcp',将无法使用 MCP 服务。")
"Warning: Missing 'mcp' dependency, MCP services will be unavailable."
)
try: try:
from mcp.client.streamable_http import streamablehttp_client from mcp.client.streamable_http import streamablehttp_client
except (ModuleNotFoundError, ImportError): except (ModuleNotFoundError, ImportError):
logger.warning( logger.warning(
"Warning: Missing 'mcp' dependency or MCP library version too old, Streamable HTTP connection unavailable.", "警告: 缺少依赖库 'mcp' 或者 mcp 库版本过低,无法使用 Streamable HTTP 连接方式。"
) )
def _prepare_config(config: dict) -> dict: def _prepare_config(config: dict) -> dict:
"""Prepare configuration, handle nested format""" """准备配置,处理嵌套格式"""
if config.get("mcpServers"): if "mcpServers" in config and config["mcpServers"]:
first_key = next(iter(config["mcpServers"])) first_key = next(iter(config["mcpServers"]))
config = config["mcpServers"][first_key] config = config["mcpServers"][first_key]
config.pop("active", None) config.pop("active", None)
@@ -46,7 +30,7 @@ def _prepare_config(config: dict) -> dict:
async def _quick_test_mcp_connection(config: dict) -> tuple[bool, str]: async def _quick_test_mcp_connection(config: dict) -> tuple[bool, str]:
"""Quick test MCP server connectivity""" """快速测试 MCP 服务器可达性"""
import aiohttp import aiohttp
cfg = _prepare_config(config.copy()) cfg = _prepare_config(config.copy())
@@ -56,15 +40,8 @@ async def _quick_test_mcp_connection(config: dict) -> tuple[bool, str]:
timeout = cfg.get("timeout", 10) timeout = cfg.get("timeout", 10)
try: try:
if "transport" in cfg:
transport_type = cfg["transport"]
elif "type" in cfg:
transport_type = cfg["type"]
else:
raise Exception("MCP connection config missing transport or type field")
async with aiohttp.ClientSession() as session: async with aiohttp.ClientSession() as session:
if transport_type == "streamable_http": if cfg.get("transport") == "streamable_http":
test_payload = { test_payload = {
"jsonrpc": "2.0", "jsonrpc": "2.0",
"method": "initialize", "method": "initialize",
@@ -87,7 +64,8 @@ async def _quick_test_mcp_connection(config: dict) -> tuple[bool, str]:
) as response: ) as response:
if response.status == 200: if response.status == 200:
return True, "" return True, ""
return False, f"HTTP {response.status}: {response.reason}" else:
return False, f"HTTP {response.status}: {response.reason}"
else: else:
async with session.get( async with session.get(
url, url,
@@ -99,10 +77,11 @@ async def _quick_test_mcp_connection(config: dict) -> tuple[bool, str]:
) as response: ) as response:
if response.status == 200: if response.status == 200:
return True, "" return True, ""
return False, f"HTTP {response.status}: {response.reason}" else:
return False, f"HTTP {response.status}: {response.reason}"
except asyncio.TimeoutError: except asyncio.TimeoutError:
return False, f"Connection timeout: {timeout} seconds" return False, f"连接超时: {timeout}"
except Exception as e: except Exception as e:
return False, f"{e!s}" return False, f"{e!s}"
@@ -110,42 +89,30 @@ async def _quick_test_mcp_connection(config: dict) -> tuple[bool, str]:
class MCPClient: class MCPClient:
def __init__(self): def __init__(self):
# Initialize session and client objects # Initialize session and client objects
self.session: mcp.ClientSession | None = None self.session: Optional[mcp.ClientSession] = None
self.exit_stack = AsyncExitStack() self.exit_stack = AsyncExitStack()
self._old_exit_stacks: list[AsyncExitStack] = [] # Track old stacks for cleanup
self.name: str | None = None self.name = None
self.active: bool = True self.active: bool = True
self.tools: list[mcp.Tool] = [] self.tools: list[mcp.Tool] = []
self.server_errlogs: list[str] = [] self.server_errlogs: list[str] = []
self.running_event = asyncio.Event() self.running_event = asyncio.Event()
# Store connection config for reconnection
self._mcp_server_config: dict | None = None
self._server_name: str | None = None
self._reconnect_lock = asyncio.Lock() # Lock for thread-safe reconnection
self._reconnecting: bool = False # For logging and debugging
async def connect_to_server(self, mcp_server_config: dict, name: str): async def connect_to_server(self, mcp_server_config: dict, name: str):
"""Connect to MCP server """连接到 MCP 服务器
If `url` parameter exists: 如果 `url` 参数存在:
1. When transport is specified as `streamable_http`, use Streamable HTTP connection. 1. transport 指定为 `streamable_http` 时,使用 Streamable HTTP 连接方式。
2. When transport is specified as `sse`, use SSE connection. 1. transport 指定为 `sse` 时,使用 SSE 连接方式。
3. If not specified, default to SSE connection to MCP service. 2. 如果没有指定,默认使用 SSE 的方式连接到 MCP 服务。
Args: Args:
mcp_server_config (dict): Configuration for the MCP server. See https://modelcontextprotocol.io/quickstart/server mcp_server_config (dict): Configuration for the MCP server. See https://modelcontextprotocol.io/quickstart/server
""" """
# Store config for reconnection
self._mcp_server_config = mcp_server_config
self._server_name = name
cfg = _prepare_config(mcp_server_config.copy()) cfg = _prepare_config(mcp_server_config.copy())
def logging_callback(msg: str): def logging_callback(msg: str):
# Handle MCP service error logs # 处理 MCP 服务的错误日志
print(f"MCP Server {name} Error: {msg}") print(f"MCP Server {name} Error: {msg}")
self.server_errlogs.append(msg) self.server_errlogs.append(msg)
@@ -154,14 +121,7 @@ class MCPClient:
if not success: if not success:
raise Exception(error_msg) raise Exception(error_msg)
if "transport" in cfg: if cfg.get("transport") != "streamable_http":
transport_type = cfg["transport"]
elif "type" in cfg:
transport_type = cfg["type"]
else:
raise Exception("MCP connection config missing transport or type field")
if transport_type != "streamable_http":
# SSE transport method # SSE transport method
self._streams_context = sse_client( self._streams_context = sse_client(
url=cfg["url"], url=cfg["url"],
@@ -170,22 +130,22 @@ class MCPClient:
sse_read_timeout=cfg.get("sse_read_timeout", 60 * 5), sse_read_timeout=cfg.get("sse_read_timeout", 60 * 5),
) )
streams = await self.exit_stack.enter_async_context( streams = await self.exit_stack.enter_async_context(
self._streams_context, self._streams_context
) )
# Create a new client session # Create a new client session
read_timeout = timedelta(seconds=cfg.get("session_read_timeout", 60)) read_timeout = timedelta(seconds=cfg.get("session_read_timeout", 20))
self.session = await self.exit_stack.enter_async_context( self.session = await self.exit_stack.enter_async_context(
mcp.ClientSession( mcp.ClientSession(
*streams, *streams,
read_timeout_seconds=read_timeout, read_timeout_seconds=read_timeout,
logging_callback=logging_callback, # type: ignore logging_callback=logging_callback, # type: ignore
), )
) )
else: else:
timeout = timedelta(seconds=cfg.get("timeout", 30)) timeout = timedelta(seconds=cfg.get("timeout", 30))
sse_read_timeout = timedelta( sse_read_timeout = timedelta(
seconds=cfg.get("sse_read_timeout", 60 * 5), seconds=cfg.get("sse_read_timeout", 60 * 5)
) )
self._streams_context = streamablehttp_client( self._streams_context = streamablehttp_client(
url=cfg["url"], url=cfg["url"],
@@ -195,18 +155,18 @@ class MCPClient:
terminate_on_close=cfg.get("terminate_on_close", True), terminate_on_close=cfg.get("terminate_on_close", True),
) )
read_s, write_s, _ = await self.exit_stack.enter_async_context( read_s, write_s, _ = await self.exit_stack.enter_async_context(
self._streams_context, self._streams_context
) )
# Create a new client session # Create a new client session
read_timeout = timedelta(seconds=cfg.get("session_read_timeout", 60)) read_timeout = timedelta(seconds=cfg.get("session_read_timeout", 20))
self.session = await self.exit_stack.enter_async_context( self.session = await self.exit_stack.enter_async_context(
mcp.ClientSession( mcp.ClientSession(
read_stream=read_s, read_stream=read_s,
write_stream=write_s, write_stream=write_s,
read_timeout_seconds=read_timeout, read_timeout_seconds=read_timeout,
logging_callback=logging_callback, # type: ignore logging_callback=logging_callback, # type: ignore
), )
) )
else: else:
@@ -215,7 +175,7 @@ class MCPClient:
) )
def callback(msg: str): def callback(msg: str):
# Handle MCP service error logs # 处理 MCP 服务的错误日志
self.server_errlogs.append(msg) self.server_errlogs.append(msg)
stdio_transport = await self.exit_stack.enter_async_context( stdio_transport = await self.exit_stack.enter_async_context(
@@ -232,154 +192,17 @@ class MCPClient:
# Create a new client session # Create a new client session
self.session = await self.exit_stack.enter_async_context( self.session = await self.exit_stack.enter_async_context(
mcp.ClientSession(*stdio_transport), mcp.ClientSession(*stdio_transport)
) )
await self.session.initialize() await self.session.initialize()
async def list_tools_and_save(self) -> mcp.ListToolsResult: async def list_tools_and_save(self) -> mcp.ListToolsResult:
"""List all tools from the server and save them to self.tools""" """List all tools from the server and save them to self.tools"""
if not self.session:
raise Exception("MCP Client is not initialized")
response = await self.session.list_tools() response = await self.session.list_tools()
self.tools = response.tools self.tools = response.tools
return response return response
async def _reconnect(self) -> None:
"""Reconnect to the MCP server using the stored configuration.
Uses asyncio.Lock to ensure thread-safe reconnection in concurrent environments.
Raises:
Exception: raised when reconnection fails
"""
async with self._reconnect_lock:
# Check if already reconnecting (useful for logging)
if self._reconnecting:
logger.debug(
f"MCP Client {self._server_name} is already reconnecting, skipping"
)
return
if not self._mcp_server_config or not self._server_name:
raise Exception("Cannot reconnect: missing connection configuration")
self._reconnecting = True
try:
logger.info(
f"Attempting to reconnect to MCP server {self._server_name}..."
)
# Save old exit_stack for later cleanup (don't close it now to avoid cancel scope issues)
if self.exit_stack:
self._old_exit_stacks.append(self.exit_stack)
# Mark old session as invalid
self.session = None
# Create new exit stack for new connection
self.exit_stack = AsyncExitStack()
# Reconnect using stored config
await self.connect_to_server(self._mcp_server_config, self._server_name)
await self.list_tools_and_save()
logger.info(
f"Successfully reconnected to MCP server {self._server_name}"
)
except Exception as e:
logger.error(
f"Failed to reconnect to MCP server {self._server_name}: {e}"
)
raise
finally:
self._reconnecting = False
async def call_tool_with_reconnect(
self,
tool_name: str,
arguments: dict,
read_timeout_seconds: timedelta,
) -> mcp.types.CallToolResult:
"""Call MCP tool with automatic reconnection on failure, max 2 retries.
Args:
tool_name: tool name
arguments: tool arguments
read_timeout_seconds: read timeout
Returns:
MCP tool call result
Raises:
ValueError: MCP session is not available
anyio.ClosedResourceError: raised after reconnection failure
"""
@retry(
retry=retry_if_exception_type(anyio.ClosedResourceError),
stop=stop_after_attempt(2),
wait=wait_exponential(multiplier=1, min=1, max=3),
before_sleep=before_sleep_log(logger, logging.WARNING),
reraise=True,
)
async def _call_with_retry():
if not self.session:
raise ValueError("MCP session is not available for MCP function tools.")
try:
return await self.session.call_tool(
name=tool_name,
arguments=arguments,
read_timeout_seconds=read_timeout_seconds,
)
except anyio.ClosedResourceError:
logger.warning(
f"MCP tool {tool_name} call failed (ClosedResourceError), attempting to reconnect..."
)
# Attempt to reconnect
await self._reconnect()
# Reraise the exception to trigger tenacity retry
raise
return await _call_with_retry()
async def cleanup(self): async def cleanup(self):
"""Clean up resources including old exit stacks from reconnections""" """Clean up resources"""
# Close current exit stack await self.exit_stack.aclose()
try: self.running_event.set() # Set the running event to indicate cleanup is done
await self.exit_stack.aclose()
except Exception as e:
logger.debug(f"Error closing current exit stack: {e}")
# Don't close old exit stacks as they may be in different task contexts
# They will be garbage collected naturally
# Just clear the list to release references
self._old_exit_stacks.clear()
# Set running_event first to unblock any waiting tasks
self.running_event.set()
class MCPTool(FunctionTool, Generic[TContext]):
"""A function tool that calls an MCP service."""
def __init__(
self, mcp_tool: mcp.Tool, mcp_client: MCPClient, mcp_server_name: str, **kwargs
):
super().__init__(
name=mcp_tool.name,
description=mcp_tool.description or "",
parameters=mcp_tool.inputSchema,
)
self.mcp_tool = mcp_tool
self.mcp_client = mcp_client
self.mcp_server_name = mcp_server_name
async def call(
self, context: ContextWrapper[TContext], **kwargs
) -> mcp.types.CallToolResult:
return await self.mcp_client.call_tool_with_reconnect(
tool_name=self.mcp_tool.name,
arguments=kwargs,
read_timeout_seconds=timedelta(seconds=context.tool_call_timeout),
)
-192
View File
@@ -1,192 +0,0 @@
# Inspired by MoonshotAI/kosong, credits to MoonshotAI/kosong authors for the original implementation.
# License: Apache License 2.0
from typing import Any, ClassVar, Literal, cast
from pydantic import BaseModel, GetCoreSchemaHandler, model_validator
from pydantic_core import core_schema
class ContentPart(BaseModel):
"""A part of the content in a message."""
__content_part_registry: ClassVar[dict[str, type["ContentPart"]]] = {}
type: str
def __init_subclass__(cls, **kwargs: Any) -> None:
super().__init_subclass__(**kwargs)
invalid_subclass_error_msg = f"ContentPart subclass {cls.__name__} must have a `type` field of type `str`"
type_value = getattr(cls, "type", None)
if type_value is None or not isinstance(type_value, str):
raise ValueError(invalid_subclass_error_msg)
cls.__content_part_registry[type_value] = cls
@classmethod
def __get_pydantic_core_schema__(
cls, source_type: Any, handler: GetCoreSchemaHandler
) -> core_schema.CoreSchema:
# If we're dealing with the base ContentPart class, use custom validation
if cls.__name__ == "ContentPart":
def validate_content_part(value: Any) -> Any:
# if it's already an instance of a ContentPart subclass, return it
if hasattr(value, "__class__") and issubclass(value.__class__, cls):
return value
# if it's a dict with a type field, dispatch to the appropriate subclass
if isinstance(value, dict) and "type" in value:
type_value: Any | None = cast(dict[str, Any], value).get("type")
if not isinstance(type_value, str):
raise ValueError(f"Cannot validate {value} as ContentPart")
target_class = cls.__content_part_registry[type_value]
return target_class.model_validate(value)
raise ValueError(f"Cannot validate {value} as ContentPart")
return core_schema.no_info_plain_validator_function(validate_content_part)
# for subclasses, use the default schema
return handler(source_type)
class TextPart(ContentPart):
"""
>>> TextPart(text="Hello, world!").model_dump()
{'type': 'text', 'text': 'Hello, world!'}
"""
type: str = "text"
text: str
class ImageURLPart(ContentPart):
"""
>>> ImageURLPart(image_url="http://example.com/image.jpg").model_dump()
{'type': 'image_url', 'image_url': 'http://example.com/image.jpg'}
"""
class ImageURL(BaseModel):
url: str
"""The URL of the image, can be data URI scheme like `data:image/png;base64,...`."""
id: str | None = None
"""The ID of the image, to allow LLMs to distinguish different images."""
type: str = "image_url"
image_url: ImageURL
class AudioURLPart(ContentPart):
"""
>>> AudioURLPart(audio_url=AudioURLPart.AudioURL(url="https://example.com/audio.mp3")).model_dump()
{'type': 'audio_url', 'audio_url': {'url': 'https://example.com/audio.mp3', 'id': None}}
"""
class AudioURL(BaseModel):
url: str
"""The URL of the audio, can be data URI scheme like `data:audio/aac;base64,...`."""
id: str | None = None
"""The ID of the audio, to allow LLMs to distinguish different audios."""
type: str = "audio_url"
audio_url: AudioURL
class ToolCall(BaseModel):
"""
A tool call requested by the assistant.
>>> ToolCall(
... id="123",
... function=ToolCall.FunctionBody(
... name="function",
... arguments="{}"
... ),
... ).model_dump()
{'type': 'function', 'id': '123', 'function': {'name': 'function', 'arguments': '{}'}}
"""
class FunctionBody(BaseModel):
name: str
arguments: str | None
type: Literal["function"] = "function"
id: str
"""The ID of the tool call."""
function: FunctionBody
"""The function body of the tool call."""
extra_content: dict[str, Any] | None = None
"""Extra metadata for the tool call."""
def model_dump(self, **kwargs: Any) -> dict[str, Any]:
if self.extra_content is None:
kwargs.setdefault("exclude", set()).add("extra_content")
return super().model_dump(**kwargs)
class ToolCallPart(BaseModel):
"""A part of the tool call."""
arguments_part: str | None = None
"""A part of the arguments of the tool call."""
class Message(BaseModel):
"""A message in a conversation."""
role: Literal[
"system",
"user",
"assistant",
"tool",
]
content: str | list[ContentPart] | None = None
"""The content of the message."""
tool_calls: list[ToolCall] | list[dict] | None = None
"""The tool calls of the message."""
tool_call_id: str | None = None
"""The ID of the tool call."""
@model_validator(mode="after")
def check_content_required(self):
# assistant + tool_calls is not None: allow content to be None
if self.role == "assistant" and self.tool_calls is not None:
return self
# other all cases: content is required
if self.content is None:
raise ValueError(
"content is required unless role='assistant' and tool_calls is not None"
)
return self
class AssistantMessageSegment(Message):
"""A message segment from the assistant."""
role: Literal["assistant"] = "assistant"
class ToolCallMessageSegment(Message):
"""A message segment representing a tool call."""
role: Literal["tool"] = "tool"
class UserMessageSegment(Message):
"""A message segment from the user."""
role: Literal["user"] = "user"
class SystemMessageSegment(Message):
"""A message segment from the system."""
role: Literal["system"] = "system"
+1 -3
View File
@@ -1,9 +1,7 @@
import typing as T
from dataclasses import dataclass from dataclasses import dataclass
import typing as T
from astrbot.core.message.message_event_result import MessageChain from astrbot.core.message.message_event_result import MessageChain
class AgentResponseData(T.TypedDict): class AgentResponseData(T.TypedDict):
chain: MessageChain chain: MessageChain
+4 -9
View File
@@ -1,22 +1,17 @@
from dataclasses import dataclass
from typing import Any, Generic from typing import Any, Generic
from pydantic import Field
from pydantic.dataclasses import dataclass
from typing_extensions import TypeVar from typing_extensions import TypeVar
from .message import Message from astrbot.core.platform.astr_message_event import AstrMessageEvent
TContext = TypeVar("TContext", default=Any) TContext = TypeVar("TContext", default=Any)
@dataclass(config={"arbitrary_types_allowed": True}) @dataclass
class ContextWrapper(Generic[TContext]): class ContextWrapper(Generic[TContext]):
"""A context for running an agent, which can be used to pass additional data or state.""" """A context for running an agent, which can be used to pass additional data or state."""
context: TContext context: TContext
messages: list[Message] = Field(default_factory=list) event: AstrMessageEvent
"""This field stores the llm message context for the agent run, agent runners will maintain this field automatically."""
tool_call_timeout: int = 60 # Default tool call timeout in seconds
NoContext = ContextWrapper[None] NoContext = ContextWrapper[None]
+16 -23
View File
@@ -1,13 +1,12 @@
import abc import abc
import typing as T import typing as T
from enum import Enum, auto from enum import Enum, auto
from astrbot import logger
from astrbot.core.provider.entities import LLMResponse
from ..hooks import BaseAgentRunHooks
from ..response import AgentResponse
from ..run_context import ContextWrapper, TContext from ..run_context import ContextWrapper, TContext
from ..response import AgentResponse
from ..hooks import BaseAgentRunHooks
from ..tool_executor import BaseFunctionToolExecutor
from astrbot.core.provider import Provider
from astrbot.core.provider.entities import LLMResponse
class AgentState(Enum): class AgentState(Enum):
@@ -23,43 +22,37 @@ class BaseAgentRunner(T.Generic[TContext]):
@abc.abstractmethod @abc.abstractmethod
async def reset( async def reset(
self, self,
provider: Provider,
run_context: ContextWrapper[TContext], run_context: ContextWrapper[TContext],
tool_executor: BaseFunctionToolExecutor[TContext],
agent_hooks: BaseAgentRunHooks[TContext], agent_hooks: BaseAgentRunHooks[TContext],
**kwargs: T.Any, **kwargs: T.Any,
) -> None: ) -> None:
"""Reset the agent to its initial state. """
Reset the agent to its initial state.
This method should be called before starting a new run. This method should be called before starting a new run.
""" """
... ...
@abc.abstractmethod @abc.abstractmethod
async def step(self) -> T.AsyncGenerator[AgentResponse, None]: async def step(self) -> T.AsyncGenerator[AgentResponse, None]:
"""Process a single step of the agent.""" """
... Process a single step of the agent.
"""
@abc.abstractmethod
async def step_until_done(
self, max_step: int
) -> T.AsyncGenerator[AgentResponse, None]:
"""Process steps until the agent is done."""
... ...
@abc.abstractmethod @abc.abstractmethod
def done(self) -> bool: def done(self) -> bool:
"""Check if the agent has completed its task. """
Check if the agent has completed its task.
Returns True if the agent is done, False otherwise. Returns True if the agent is done, False otherwise.
""" """
... ...
@abc.abstractmethod @abc.abstractmethod
def get_final_llm_resp(self) -> LLMResponse | None: def get_final_llm_resp(self) -> LLMResponse | None:
"""Get the final observation from the agent. """
Get the final observation from the agent.
This method should be called after the agent is done. This method should be called after the agent is done.
""" """
... ...
def _transition_state(self, new_state: AgentState) -> None:
"""Transition the agent state."""
if self._state != new_state:
logger.debug(f"Agent state transition: {self._state} -> {new_state}")
self._state = new_state
@@ -1,367 +0,0 @@
import base64
import json
import sys
import typing as T
import astrbot.core.message.components as Comp
from astrbot import logger
from astrbot.core import sp
from astrbot.core.message.message_event_result import MessageChain
from astrbot.core.provider.entities import (
LLMResponse,
ProviderRequest,
)
from ...hooks import BaseAgentRunHooks
from ...response import AgentResponseData
from ...run_context import ContextWrapper, TContext
from ..base import AgentResponse, AgentState, BaseAgentRunner
from .coze_api_client import CozeAPIClient
if sys.version_info >= (3, 12):
from typing import override
else:
from typing_extensions import override
class CozeAgentRunner(BaseAgentRunner[TContext]):
"""Coze Agent Runner"""
@override
async def reset(
self,
request: ProviderRequest,
run_context: ContextWrapper[TContext],
agent_hooks: BaseAgentRunHooks[TContext],
provider_config: dict,
**kwargs: T.Any,
) -> None:
self.req = request
self.streaming = kwargs.get("streaming", False)
self.final_llm_resp = None
self._state = AgentState.IDLE
self.agent_hooks = agent_hooks
self.run_context = run_context
self.api_key = provider_config.get("coze_api_key", "")
if not self.api_key:
raise Exception("Coze API Key 不能为空。")
self.bot_id = provider_config.get("bot_id", "")
if not self.bot_id:
raise Exception("Coze Bot ID 不能为空。")
self.api_base: str = provider_config.get("coze_api_base", "https://api.coze.cn")
if not isinstance(self.api_base, str) or not self.api_base.startswith(
("http://", "https://"),
):
raise Exception(
"Coze API Base URL 格式不正确,必须以 http:// 或 https:// 开头。",
)
self.timeout = provider_config.get("timeout", 120)
if isinstance(self.timeout, str):
self.timeout = int(self.timeout)
self.auto_save_history = provider_config.get("auto_save_history", True)
# 创建 API 客户端
self.api_client = CozeAPIClient(api_key=self.api_key, api_base=self.api_base)
# 会话相关缓存
self.file_id_cache: dict[str, dict[str, str]] = {}
@override
async def step(self):
"""
执行 Coze Agent 的一个步骤
"""
if not self.req:
raise ValueError("Request is not set. Please call reset() first.")
if self._state == AgentState.IDLE:
try:
await self.agent_hooks.on_agent_begin(self.run_context)
except Exception as e:
logger.error(f"Error in on_agent_begin hook: {e}", exc_info=True)
# 开始处理,转换到运行状态
self._transition_state(AgentState.RUNNING)
try:
# 执行 Coze 请求并处理结果
async for response in self._execute_coze_request():
yield response
except Exception as e:
logger.error(f"Coze 请求失败:{str(e)}")
self._transition_state(AgentState.ERROR)
self.final_llm_resp = LLMResponse(
role="err", completion_text=f"Coze 请求失败:{str(e)}"
)
yield AgentResponse(
type="err",
data=AgentResponseData(
chain=MessageChain().message(f"Coze 请求失败:{str(e)}")
),
)
finally:
await self.api_client.close()
@override
async def step_until_done(
self, max_step: int = 30
) -> T.AsyncGenerator[AgentResponse, None]:
while not self.done():
async for resp in self.step():
yield resp
async def _execute_coze_request(self):
"""执行 Coze 请求的核心逻辑"""
prompt = self.req.prompt or ""
session_id = self.req.session_id or "unknown"
image_urls = self.req.image_urls or []
contexts = self.req.contexts or []
system_prompt = self.req.system_prompt
# 用户ID参数
user_id = session_id
# 获取或创建会话ID
conversation_id = await sp.get_async(
scope="umo",
scope_id=user_id,
key="coze_conversation_id",
default="",
)
# 构建消息
additional_messages = []
if system_prompt:
if not self.auto_save_history or not conversation_id:
additional_messages.append(
{
"role": "system",
"content": system_prompt,
"content_type": "text",
},
)
# 处理历史上下文
if not self.auto_save_history and contexts:
for ctx in contexts:
if isinstance(ctx, dict) and "role" in ctx and "content" in ctx:
# 处理上下文中的图片
content = ctx["content"]
if isinstance(content, list):
# 多模态内容,需要处理图片
processed_content = []
for item in content:
if isinstance(item, dict):
if item.get("type") == "text":
processed_content.append(item)
elif item.get("type") == "image_url":
# 处理图片上传
try:
image_data = item.get("image_url", {})
url = image_data.get("url", "")
if url:
file_id = (
await self._download_and_upload_image(
url, session_id
)
)
processed_content.append(
{
"type": "file",
"file_id": file_id,
"file_url": url,
}
)
except Exception as e:
logger.warning(f"处理上下文图片失败: {e}")
continue
if processed_content:
additional_messages.append(
{
"role": ctx["role"],
"content": processed_content,
"content_type": "object_string",
}
)
else:
# 纯文本内容
additional_messages.append(
{
"role": ctx["role"],
"content": content,
"content_type": "text",
}
)
# 构建当前消息
if prompt or image_urls:
if image_urls:
# 多模态
object_string_content = []
if prompt:
object_string_content.append({"type": "text", "text": prompt})
for url in image_urls:
# the url is a base64 string
try:
image_data = base64.b64decode(url)
file_id = await self.api_client.upload_file(image_data)
object_string_content.append(
{
"type": "image",
"file_id": file_id,
}
)
except Exception as e:
logger.warning(f"处理图片失败 {url}: {e}")
continue
if object_string_content:
content = json.dumps(object_string_content, ensure_ascii=False)
additional_messages.append(
{
"role": "user",
"content": content,
"content_type": "object_string",
}
)
elif prompt:
# 纯文本
additional_messages.append(
{
"role": "user",
"content": prompt,
"content_type": "text",
},
)
# 执行 Coze API 请求
accumulated_content = ""
message_started = False
async for chunk in self.api_client.chat_messages(
bot_id=self.bot_id,
user_id=user_id,
additional_messages=additional_messages,
conversation_id=conversation_id,
auto_save_history=self.auto_save_history,
stream=True,
timeout=self.timeout,
):
event_type = chunk.get("event")
data = chunk.get("data", {})
if event_type == "conversation.chat.created":
if isinstance(data, dict) and "conversation_id" in data:
await sp.put_async(
scope="umo",
scope_id=user_id,
key="coze_conversation_id",
value=data["conversation_id"],
)
if event_type == "conversation.message.delta":
# 增量消息
content = data.get("content", "")
if not content and "delta" in data:
content = data["delta"].get("content", "")
if not content and "text" in data:
content = data.get("text", "")
if content:
accumulated_content += content
message_started = True
# 如果是流式响应,发送增量数据
if self.streaming:
yield AgentResponse(
type="streaming_delta",
data=AgentResponseData(
chain=MessageChain().message(content)
),
)
elif event_type == "conversation.message.completed":
# 消息完成
logger.debug("Coze message completed")
message_started = True
elif event_type == "conversation.chat.completed":
# 对话完成
logger.debug("Coze chat completed")
break
elif event_type == "error":
# 错误处理
error_msg = data.get("msg", "未知错误")
error_code = data.get("code", "UNKNOWN")
logger.error(f"Coze 出现错误: {error_code} - {error_msg}")
raise Exception(f"Coze 出现错误: {error_code} - {error_msg}")
if not message_started and not accumulated_content:
logger.warning("Coze 未返回任何内容")
accumulated_content = ""
# 创建最终响应
chain = MessageChain(chain=[Comp.Plain(accumulated_content)])
self.final_llm_resp = LLMResponse(role="assistant", result_chain=chain)
self._transition_state(AgentState.DONE)
try:
await self.agent_hooks.on_agent_done(self.run_context, self.final_llm_resp)
except Exception as e:
logger.error(f"Error in on_agent_done hook: {e}", exc_info=True)
# 返回最终结果
yield AgentResponse(
type="llm_result",
data=AgentResponseData(chain=chain),
)
async def _download_and_upload_image(
self,
image_url: str,
session_id: str | None = None,
) -> str:
"""下载图片并上传到 Coze,返回 file_id"""
import hashlib
# 计算哈希实现缓存
cache_key = hashlib.md5(image_url.encode("utf-8")).hexdigest()
if session_id:
if session_id not in self.file_id_cache:
self.file_id_cache[session_id] = {}
if cache_key in self.file_id_cache[session_id]:
file_id = self.file_id_cache[session_id][cache_key]
logger.debug(f"[Coze] 使用缓存的 file_id: {file_id}")
return file_id
try:
image_data = await self.api_client.download_image(image_url)
file_id = await self.api_client.upload_file(image_data)
if session_id:
self.file_id_cache[session_id][cache_key] = file_id
logger.debug(f"[Coze] 图片上传成功并缓存,file_id: {file_id}")
return file_id
except Exception as e:
logger.error(f"处理图片失败 {image_url}: {e!s}")
raise Exception(f"处理图片失败: {e!s}")
@override
def done(self) -> bool:
"""检查 Agent 是否已完成工作"""
return self._state in (AgentState.DONE, AgentState.ERROR)
@override
def get_final_llm_resp(self) -> LLMResponse | None:
return self.final_llm_resp
@@ -1,324 +0,0 @@
import asyncio
import io
import json
from collections.abc import AsyncGenerator
from typing import Any
import aiohttp
from astrbot.core import logger
class CozeAPIClient:
def __init__(self, api_key: str, api_base: str = "https://api.coze.cn"):
self.api_key = api_key
self.api_base = api_base
self.session = None
async def _ensure_session(self):
"""确保HTTP session存在"""
if self.session is None:
connector = aiohttp.TCPConnector(
ssl=False if self.api_base.startswith("http://") else True,
limit=100,
limit_per_host=30,
keepalive_timeout=30,
enable_cleanup_closed=True,
)
timeout = aiohttp.ClientTimeout(
total=120, # 默认超时时间
connect=30,
sock_read=120,
)
headers = {
"Authorization": f"Bearer {self.api_key}",
"Accept": "text/event-stream",
}
self.session = aiohttp.ClientSession(
headers=headers,
timeout=timeout,
connector=connector,
)
return self.session
async def upload_file(
self,
file_data: bytes,
) -> str:
"""上传文件到 Coze 并返回 file_id
Args:
file_data (bytes): 文件的二进制数据
Returns:
str: 上传成功后返回的 file_id
"""
session = await self._ensure_session()
url = f"{self.api_base}/v1/files/upload"
try:
file_io = io.BytesIO(file_data)
async with session.post(
url,
data={
"file": file_io,
},
timeout=aiohttp.ClientTimeout(total=60),
) as response:
if response.status == 401:
raise Exception("Coze API 认证失败,请检查 API Key 是否正确")
response_text = await response.text()
logger.debug(
f"文件上传响应状态: {response.status}, 内容: {response_text}",
)
if response.status != 200:
raise Exception(
f"文件上传失败,状态码: {response.status}, 响应: {response_text}",
)
try:
result = await response.json()
except json.JSONDecodeError:
raise Exception(f"文件上传响应解析失败: {response_text}")
if result.get("code") != 0:
raise Exception(f"文件上传失败: {result.get('msg', '未知错误')}")
file_id = result["data"]["id"]
logger.debug(f"[Coze] 图片上传成功,file_id: {file_id}")
return file_id
except asyncio.TimeoutError:
logger.error("文件上传超时")
raise Exception("文件上传超时")
except Exception as e:
logger.error(f"文件上传失败: {e!s}")
raise Exception(f"文件上传失败: {e!s}")
async def download_image(self, image_url: str) -> bytes:
"""下载图片并返回字节数据
Args:
image_url (str): 图片的URL
Returns:
bytes: 图片的二进制数据
"""
session = await self._ensure_session()
try:
async with session.get(image_url) as response:
if response.status != 200:
raise Exception(f"下载图片失败,状态码: {response.status}")
image_data = await response.read()
return image_data
except Exception as e:
logger.error(f"下载图片失败 {image_url}: {e!s}")
raise Exception(f"下载图片失败: {e!s}")
async def chat_messages(
self,
bot_id: str,
user_id: str,
additional_messages: list[dict] | None = None,
conversation_id: str | None = None,
auto_save_history: bool = True,
stream: bool = True,
timeout: float = 120,
) -> AsyncGenerator[dict[str, Any], None]:
"""发送聊天消息并返回流式响应
Args:
bot_id: Bot ID
user_id: 用户ID
additional_messages: 额外消息列表
conversation_id: 会话ID
auto_save_history: 是否自动保存历史
stream: 是否流式响应
timeout: 超时时间
"""
session = await self._ensure_session()
url = f"{self.api_base}/v3/chat"
payload = {
"bot_id": bot_id,
"user_id": user_id,
"stream": stream,
"auto_save_history": auto_save_history,
}
if additional_messages:
payload["additional_messages"] = additional_messages
params = {}
if conversation_id:
params["conversation_id"] = conversation_id
logger.debug(f"Coze chat_messages payload: {payload}, params: {params}")
try:
async with session.post(
url,
json=payload,
params=params,
timeout=aiohttp.ClientTimeout(total=timeout),
) as response:
if response.status == 401:
raise Exception("Coze API 认证失败,请检查 API Key 是否正确")
if response.status != 200:
raise Exception(f"Coze API 流式请求失败,状态码: {response.status}")
# SSE
buffer = ""
event_type = None
event_data = None
async for chunk in response.content:
if chunk:
buffer += chunk.decode("utf-8", errors="ignore")
lines = buffer.split("\n")
buffer = lines[-1]
for line in lines[:-1]:
line = line.strip()
if not line:
if event_type and event_data:
yield {"event": event_type, "data": event_data}
event_type = None
event_data = None
elif line.startswith("event:"):
event_type = line[6:].strip()
elif line.startswith("data:"):
data_str = line[5:].strip()
if data_str and data_str != "[DONE]":
try:
event_data = json.loads(data_str)
except json.JSONDecodeError:
event_data = {"content": data_str}
except asyncio.TimeoutError:
raise Exception(f"Coze API 流式请求超时 ({timeout}秒)")
except Exception as e:
raise Exception(f"Coze API 流式请求失败: {e!s}")
async def clear_context(self, conversation_id: str):
"""清空会话上下文
Args:
conversation_id: 会话ID
Returns:
dict: API响应结果
"""
session = await self._ensure_session()
url = f"{self.api_base}/v3/conversation/message/clear_context"
payload = {"conversation_id": conversation_id}
try:
async with session.post(url, json=payload) as response:
response_text = await response.text()
if response.status == 401:
raise Exception("Coze API 认证失败,请检查 API Key 是否正确")
if response.status != 200:
raise Exception(f"Coze API 请求失败,状态码: {response.status}")
try:
return json.loads(response_text)
except json.JSONDecodeError:
raise Exception("Coze API 返回非JSON格式")
except asyncio.TimeoutError:
raise Exception("Coze API 请求超时")
except aiohttp.ClientError as e:
raise Exception(f"Coze API 请求失败: {e!s}")
async def get_message_list(
self,
conversation_id: str,
order: str = "desc",
limit: int = 10,
offset: int = 0,
):
"""获取消息列表
Args:
conversation_id: 会话ID
order: 排序方式 (asc/desc)
limit: 限制数量
offset: 偏移量
Returns:
dict: API响应结果
"""
session = await self._ensure_session()
url = f"{self.api_base}/v3/conversation/message/list"
params = {
"conversation_id": conversation_id,
"order": order,
"limit": limit,
"offset": offset,
}
try:
async with session.get(url, params=params) as response:
response.raise_for_status()
return await response.json()
except Exception as e:
logger.error(f"获取Coze消息列表失败: {e!s}")
raise Exception(f"获取Coze消息列表失败: {e!s}")
async def close(self):
"""关闭会话"""
if self.session:
await self.session.close()
self.session = None
if __name__ == "__main__":
import asyncio
import os
async def test_coze_api_client():
api_key = os.getenv("COZE_API_KEY", "")
bot_id = os.getenv("COZE_BOT_ID", "")
client = CozeAPIClient(api_key=api_key)
try:
with open("README.md", "rb") as f:
file_data = f.read()
file_id = await client.upload_file(file_data)
print(f"Uploaded file_id: {file_id}")
async for event in client.chat_messages(
bot_id=bot_id,
user_id="test_user",
additional_messages=[
{
"role": "user",
"content": json.dumps(
[
{"type": "text", "text": "这是什么"},
{"type": "file", "file_id": file_id},
],
ensure_ascii=False,
),
"content_type": "object_string",
},
],
stream=True,
):
print(f"Event: {event}")
finally:
await client.close()
asyncio.run(test_coze_api_client())
@@ -1,403 +0,0 @@
import asyncio
import functools
import queue
import re
import sys
import threading
import typing as T
from dashscope import Application
from dashscope.app.application_response import ApplicationResponse
import astrbot.core.message.components as Comp
from astrbot.core import logger, sp
from astrbot.core.message.message_event_result import MessageChain
from astrbot.core.provider.entities import (
LLMResponse,
ProviderRequest,
)
from ...hooks import BaseAgentRunHooks
from ...response import AgentResponseData
from ...run_context import ContextWrapper, TContext
from ..base import AgentResponse, AgentState, BaseAgentRunner
if sys.version_info >= (3, 12):
from typing import override
else:
from typing_extensions import override
class DashscopeAgentRunner(BaseAgentRunner[TContext]):
"""Dashscope Agent Runner"""
@override
async def reset(
self,
request: ProviderRequest,
run_context: ContextWrapper[TContext],
agent_hooks: BaseAgentRunHooks[TContext],
provider_config: dict,
**kwargs: T.Any,
) -> None:
self.req = request
self.streaming = kwargs.get("streaming", False)
self.final_llm_resp = None
self._state = AgentState.IDLE
self.agent_hooks = agent_hooks
self.run_context = run_context
self.api_key = provider_config.get("dashscope_api_key", "")
if not self.api_key:
raise Exception("阿里云百炼 API Key 不能为空。")
self.app_id = provider_config.get("dashscope_app_id", "")
if not self.app_id:
raise Exception("阿里云百炼 APP ID 不能为空。")
self.dashscope_app_type = provider_config.get("dashscope_app_type", "")
if not self.dashscope_app_type:
raise Exception("阿里云百炼 APP 类型不能为空。")
self.variables: dict = provider_config.get("variables", {}) or {}
self.rag_options: dict = provider_config.get("rag_options", {})
self.output_reference = self.rag_options.get("output_reference", False)
self.rag_options = self.rag_options.copy()
self.rag_options.pop("output_reference", None)
self.timeout = provider_config.get("timeout", 120)
if isinstance(self.timeout, str):
self.timeout = int(self.timeout)
def has_rag_options(self):
"""判断是否有 RAG 选项
Returns:
bool: 是否有 RAG 选项
"""
if self.rag_options and (
len(self.rag_options.get("pipeline_ids", [])) > 0
or len(self.rag_options.get("file_ids", [])) > 0
):
return True
return False
@override
async def step(self):
"""
执行 Dashscope Agent 的一个步骤
"""
if not self.req:
raise ValueError("Request is not set. Please call reset() first.")
if self._state == AgentState.IDLE:
try:
await self.agent_hooks.on_agent_begin(self.run_context)
except Exception as e:
logger.error(f"Error in on_agent_begin hook: {e}", exc_info=True)
# 开始处理,转换到运行状态
self._transition_state(AgentState.RUNNING)
try:
# 执行 Dashscope 请求并处理结果
async for response in self._execute_dashscope_request():
yield response
except Exception as e:
logger.error(f"阿里云百炼请求失败:{str(e)}")
self._transition_state(AgentState.ERROR)
self.final_llm_resp = LLMResponse(
role="err", completion_text=f"阿里云百炼请求失败:{str(e)}"
)
yield AgentResponse(
type="err",
data=AgentResponseData(
chain=MessageChain().message(f"阿里云百炼请求失败:{str(e)}")
),
)
@override
async def step_until_done(
self, max_step: int = 30
) -> T.AsyncGenerator[AgentResponse, None]:
while not self.done():
async for resp in self.step():
yield resp
def _consume_sync_generator(
self, response: T.Any, response_queue: queue.Queue
) -> None:
"""在线程中消费同步generator,将结果放入队列
Args:
response: 同步generator对象
response_queue: 用于传递数据的队列
"""
try:
if self.streaming:
for chunk in response:
response_queue.put(("data", chunk))
else:
response_queue.put(("data", response))
except Exception as e:
response_queue.put(("error", e))
finally:
response_queue.put(("done", None))
async def _process_stream_chunk(
self, chunk: ApplicationResponse, output_text: str
) -> tuple[str, list | None, AgentResponse | None]:
"""处理流式响应的单个chunk
Args:
chunk: Dashscope响应chunk
output_text: 当前累积的输出文本
Returns:
(更新后的output_text, doc_references, AgentResponse或None)
"""
logger.debug(f"dashscope stream chunk: {chunk}")
if chunk.status_code != 200:
logger.error(
f"阿里云百炼请求失败: request_id={chunk.request_id}, code={chunk.status_code}, message={chunk.message}, 请参考文档:https://help.aliyun.com/zh/model-studio/developer-reference/error-code",
)
self._transition_state(AgentState.ERROR)
error_msg = (
f"阿里云百炼请求失败: message={chunk.message} code={chunk.status_code}"
)
self.final_llm_resp = LLMResponse(
role="err",
result_chain=MessageChain().message(error_msg),
)
return (
output_text,
None,
AgentResponse(
type="err",
data=AgentResponseData(chain=MessageChain().message(error_msg)),
),
)
chunk_text = chunk.output.get("text", "") or ""
# RAG 引用脚标格式化
chunk_text = re.sub(r"<ref>\[(\d+)\]</ref>", r"[\1]", chunk_text)
response = None
if chunk_text:
output_text += chunk_text
response = AgentResponse(
type="streaming_delta",
data=AgentResponseData(chain=MessageChain().message(chunk_text)),
)
# 获取文档引用
doc_references = chunk.output.get("doc_references", None)
return output_text, doc_references, response
def _format_doc_references(self, doc_references: list) -> str:
"""格式化文档引用为文本
Args:
doc_references: 文档引用列表
Returns:
格式化后的引用文本
"""
ref_parts = []
for ref in doc_references:
ref_title = (
ref.get("title", "") if ref.get("title") else ref.get("doc_name", "")
)
ref_parts.append(f"{ref['index_id']}. {ref_title}\n")
ref_str = "".join(ref_parts)
return f"\n\n回答来源:\n{ref_str}"
async def _build_request_payload(
self, prompt: str, session_id: str, contexts: list, system_prompt: str
) -> dict:
"""构建请求payload
Args:
prompt: 用户输入
session_id: 会话ID
contexts: 上下文列表
system_prompt: 系统提示词
Returns:
请求payload字典
"""
conversation_id = await sp.get_async(
scope="umo",
scope_id=session_id,
key="dashscope_conversation_id",
default="",
)
# 获得会话变量
payload_vars = self.variables.copy()
session_var = await sp.get_async(
scope="umo",
scope_id=session_id,
key="session_variables",
default={},
)
payload_vars.update(session_var)
if (
self.dashscope_app_type in ["agent", "dialog-workflow"]
and not self.has_rag_options()
):
# 支持多轮对话的
p = {
"app_id": self.app_id,
"api_key": self.api_key,
"prompt": prompt,
"biz_params": payload_vars or None,
"stream": self.streaming,
"incremental_output": True,
}
if conversation_id:
p["session_id"] = conversation_id
return p
else:
# 不支持多轮对话的
payload = {
"app_id": self.app_id,
"prompt": prompt,
"api_key": self.api_key,
"biz_params": payload_vars or None,
"stream": self.streaming,
"incremental_output": True,
}
if self.rag_options:
payload["rag_options"] = self.rag_options
return payload
async def _handle_streaming_response(
self, response: T.Any, session_id: str
) -> T.AsyncGenerator[AgentResponse, None]:
"""处理流式响应
Args:
response: Dashscope 流式响应 generator
Yields:
AgentResponse 对象
"""
response_queue = queue.Queue()
consumer_thread = threading.Thread(
target=self._consume_sync_generator,
args=(response, response_queue),
daemon=True,
)
consumer_thread.start()
output_text = ""
doc_references = None
while True:
try:
item_type, item_data = await asyncio.get_event_loop().run_in_executor(
None, response_queue.get, True, 1
)
except queue.Empty:
continue
if item_type == "done":
break
elif item_type == "error":
raise item_data
elif item_type == "data":
chunk = item_data
assert isinstance(chunk, ApplicationResponse)
(
output_text,
chunk_doc_refs,
response,
) = await self._process_stream_chunk(chunk, output_text)
if response:
if response.type == "err":
yield response
return
yield response
if chunk_doc_refs:
doc_references = chunk_doc_refs
if chunk.output.session_id:
await sp.put_async(
scope="umo",
scope_id=session_id,
key="dashscope_conversation_id",
value=chunk.output.session_id,
)
# 添加 RAG 引用
if self.output_reference and doc_references:
ref_text = self._format_doc_references(doc_references)
output_text += ref_text
if self.streaming:
yield AgentResponse(
type="streaming_delta",
data=AgentResponseData(chain=MessageChain().message(ref_text)),
)
# 创建最终响应
chain = MessageChain(chain=[Comp.Plain(output_text)])
self.final_llm_resp = LLMResponse(role="assistant", result_chain=chain)
self._transition_state(AgentState.DONE)
try:
await self.agent_hooks.on_agent_done(self.run_context, self.final_llm_resp)
except Exception as e:
logger.error(f"Error in on_agent_done hook: {e}", exc_info=True)
# 返回最终结果
yield AgentResponse(
type="llm_result",
data=AgentResponseData(chain=chain),
)
async def _execute_dashscope_request(self):
"""执行 Dashscope 请求的核心逻辑"""
prompt = self.req.prompt or ""
session_id = self.req.session_id or "unknown"
image_urls = self.req.image_urls or []
contexts = self.req.contexts or []
system_prompt = self.req.system_prompt
# 检查图片输入
if image_urls:
logger.warning("阿里云百炼暂不支持图片输入,将自动忽略图片内容。")
# 构建请求payload
payload = await self._build_request_payload(
prompt, session_id, contexts, system_prompt
)
if not self.streaming:
payload["incremental_output"] = False
# 发起请求
partial = functools.partial(Application.call, **payload)
response = await asyncio.get_event_loop().run_in_executor(None, partial)
async for resp in self._handle_streaming_response(response, session_id):
yield resp
@override
def done(self) -> bool:
"""检查 Agent 是否已完成工作"""
return self._state in (AgentState.DONE, AgentState.ERROR)
@override
def get_final_llm_resp(self) -> LLMResponse | None:
return self.final_llm_resp
@@ -1,336 +0,0 @@
import base64
import os
import sys
import typing as T
import astrbot.core.message.components as Comp
from astrbot.core import logger, sp
from astrbot.core.message.message_event_result import MessageChain
from astrbot.core.provider.entities import (
LLMResponse,
ProviderRequest,
)
from astrbot.core.utils.astrbot_path import get_astrbot_data_path
from astrbot.core.utils.io import download_file
from ...hooks import BaseAgentRunHooks
from ...response import AgentResponseData
from ...run_context import ContextWrapper, TContext
from ..base import AgentResponse, AgentState, BaseAgentRunner
from .dify_api_client import DifyAPIClient
if sys.version_info >= (3, 12):
from typing import override
else:
from typing_extensions import override
class DifyAgentRunner(BaseAgentRunner[TContext]):
"""Dify Agent Runner"""
@override
async def reset(
self,
request: ProviderRequest,
run_context: ContextWrapper[TContext],
agent_hooks: BaseAgentRunHooks[TContext],
provider_config: dict,
**kwargs: T.Any,
) -> None:
self.req = request
self.streaming = kwargs.get("streaming", False)
self.final_llm_resp = None
self._state = AgentState.IDLE
self.agent_hooks = agent_hooks
self.run_context = run_context
self.api_key = provider_config.get("dify_api_key", "")
self.api_base = provider_config.get("dify_api_base", "https://api.dify.ai/v1")
self.api_type = provider_config.get("dify_api_type", "chat")
self.workflow_output_key = provider_config.get(
"dify_workflow_output_key",
"astrbot_wf_output",
)
self.dify_query_input_key = provider_config.get(
"dify_query_input_key",
"astrbot_text_query",
)
self.variables: dict = provider_config.get("variables", {}) or {}
self.timeout = provider_config.get("timeout", 60)
if isinstance(self.timeout, str):
self.timeout = int(self.timeout)
self.api_client = DifyAPIClient(self.api_key, self.api_base)
@override
async def step(self):
"""
执行 Dify Agent 的一个步骤
"""
if not self.req:
raise ValueError("Request is not set. Please call reset() first.")
if self._state == AgentState.IDLE:
try:
await self.agent_hooks.on_agent_begin(self.run_context)
except Exception as e:
logger.error(f"Error in on_agent_begin hook: {e}", exc_info=True)
# 开始处理,转换到运行状态
self._transition_state(AgentState.RUNNING)
try:
# 执行 Dify 请求并处理结果
async for response in self._execute_dify_request():
yield response
except Exception as e:
logger.error(f"Dify 请求失败:{str(e)}")
self._transition_state(AgentState.ERROR)
self.final_llm_resp = LLMResponse(
role="err", completion_text=f"Dify 请求失败:{str(e)}"
)
yield AgentResponse(
type="err",
data=AgentResponseData(
chain=MessageChain().message(f"Dify 请求失败:{str(e)}")
),
)
finally:
await self.api_client.close()
@override
async def step_until_done(
self, max_step: int = 30
) -> T.AsyncGenerator[AgentResponse, None]:
while not self.done():
async for resp in self.step():
yield resp
async def _execute_dify_request(self):
"""执行 Dify 请求的核心逻辑"""
prompt = self.req.prompt or ""
session_id = self.req.session_id or "unknown"
image_urls = self.req.image_urls or []
system_prompt = self.req.system_prompt
conversation_id = await sp.get_async(
scope="umo",
scope_id=session_id,
key="dify_conversation_id",
default="",
)
result = ""
# 处理图片上传
files_payload = []
for image_url in image_urls:
# image_url is a base64 string
try:
image_data = base64.b64decode(image_url)
file_response = await self.api_client.file_upload(
file_data=image_data,
user=session_id,
mime_type="image/png",
file_name="image.png",
)
logger.debug(f"Dify 上传图片响应:{file_response}")
if "id" not in file_response:
logger.warning(
f"上传图片后得到未知的 Dify 响应:{file_response},图片将忽略。"
)
continue
files_payload.append(
{
"type": "image",
"transfer_method": "local_file",
"upload_file_id": file_response["id"],
}
)
except Exception as e:
logger.warning(f"上传图片失败:{e}")
continue
# 获得会话变量
payload_vars = self.variables.copy()
# 动态变量
session_var = await sp.get_async(
scope="umo",
scope_id=session_id,
key="session_variables",
default={},
)
payload_vars.update(session_var)
payload_vars["system_prompt"] = system_prompt
# 处理不同的 API 类型
match self.api_type:
case "chat" | "agent" | "chatflow":
if not prompt:
prompt = "请描述这张图片。"
async for chunk in self.api_client.chat_messages(
inputs={
**payload_vars,
},
query=prompt,
user=session_id,
conversation_id=conversation_id,
files=files_payload,
timeout=self.timeout,
):
logger.debug(f"dify resp chunk: {chunk}")
if chunk["event"] == "message" or chunk["event"] == "agent_message":
result += chunk["answer"]
if not conversation_id:
await sp.put_async(
scope="umo",
scope_id=session_id,
key="dify_conversation_id",
value=chunk["conversation_id"],
)
conversation_id = chunk["conversation_id"]
# 如果是流式响应,发送增量数据
if self.streaming and chunk["answer"]:
yield AgentResponse(
type="streaming_delta",
data=AgentResponseData(
chain=MessageChain().message(chunk["answer"])
),
)
elif chunk["event"] == "message_end":
logger.debug("Dify message end")
break
elif chunk["event"] == "error":
logger.error(f"Dify 出现错误:{chunk}")
raise Exception(
f"Dify 出现错误 status: {chunk['status']} message: {chunk['message']}"
)
case "workflow":
async for chunk in self.api_client.workflow_run(
inputs={
self.dify_query_input_key: prompt,
"astrbot_session_id": session_id,
**payload_vars,
},
user=session_id,
files=files_payload,
timeout=self.timeout,
):
logger.debug(f"dify workflow resp chunk: {chunk}")
match chunk["event"]:
case "workflow_started":
logger.info(
f"Dify 工作流(ID: {chunk['workflow_run_id']})开始运行。"
)
case "node_finished":
logger.debug(
f"Dify 工作流节点(ID: {chunk['data']['node_id']} Title: {chunk['data'].get('title', '')})运行结束。"
)
case "text_chunk":
if self.streaming and chunk["data"]["text"]:
yield AgentResponse(
type="streaming_delta",
data=AgentResponseData(
chain=MessageChain().message(
chunk["data"]["text"]
)
),
)
case "workflow_finished":
logger.info(
f"Dify 工作流(ID: {chunk['workflow_run_id']})运行结束"
)
logger.debug(f"Dify 工作流结果:{chunk}")
if chunk["data"]["error"]:
logger.error(
f"Dify 工作流出现错误:{chunk['data']['error']}"
)
raise Exception(
f"Dify 工作流出现错误:{chunk['data']['error']}"
)
if self.workflow_output_key not in chunk["data"]["outputs"]:
raise Exception(
f"Dify 工作流的输出不包含指定的键名:{self.workflow_output_key}"
)
result = chunk
case _:
raise Exception(f"未知的 Dify API 类型:{self.api_type}")
if not result:
logger.warning("Dify 请求结果为空,请查看 Debug 日志。")
# 解析结果
chain = await self.parse_dify_result(result)
# 创建最终响应
self.final_llm_resp = LLMResponse(role="assistant", result_chain=chain)
self._transition_state(AgentState.DONE)
try:
await self.agent_hooks.on_agent_done(self.run_context, self.final_llm_resp)
except Exception as e:
logger.error(f"Error in on_agent_done hook: {e}", exc_info=True)
# 返回最终结果
yield AgentResponse(
type="llm_result",
data=AgentResponseData(chain=chain),
)
async def parse_dify_result(self, chunk: dict | str) -> MessageChain:
"""解析 Dify 的响应结果"""
if isinstance(chunk, str):
# Chat
return MessageChain(chain=[Comp.Plain(chunk)])
async def parse_file(item: dict):
match item["type"]:
case "image":
return Comp.Image(file=item["url"], url=item["url"])
case "audio":
# 仅支持 wav
temp_dir = os.path.join(get_astrbot_data_path(), "temp")
path = os.path.join(temp_dir, f"{item['filename']}.wav")
await download_file(item["url"], path)
return Comp.Image(file=item["url"], url=item["url"])
case "video":
return Comp.Video(file=item["url"])
case _:
return Comp.File(name=item["filename"], file=item["url"])
output = chunk["data"]["outputs"][self.workflow_output_key]
chains = []
if isinstance(output, str):
# 纯文本输出
chains.append(Comp.Plain(output))
elif isinstance(output, list):
# 主要适配 Dify 的 HTTP 请求结点的多模态输出
for item in output:
# handle Array[File]
if (
not isinstance(item, dict)
or item.get("dify_model_identity", "") != "__dify__file__"
):
chains.append(Comp.Plain(str(output)))
break
else:
chains.append(Comp.Plain(str(output)))
# scan file
files = chunk["data"].get("files", [])
for item in files:
comp = await parse_file(item)
chains.append(comp)
return MessageChain(chain=chains)
@override
def done(self) -> bool:
"""检查 Agent 是否已完成工作"""
return self._state in (AgentState.DONE, AgentState.ERROR)
@override
def get_final_llm_resp(self) -> LLMResponse | None:
return self.final_llm_resp
@@ -1,195 +0,0 @@
import codecs
import json
from collections.abc import AsyncGenerator
from typing import Any
from aiohttp import ClientResponse, ClientSession, FormData
from astrbot.core import logger
async def _stream_sse(resp: ClientResponse) -> AsyncGenerator[dict, None]:
decoder = codecs.getincrementaldecoder("utf-8")()
buffer = ""
async for chunk in resp.content.iter_chunked(8192):
buffer += decoder.decode(chunk)
while "\n\n" in buffer:
block, buffer = buffer.split("\n\n", 1)
if block.strip().startswith("data:"):
try:
yield json.loads(block[5:])
except json.JSONDecodeError:
logger.warning(f"Drop invalid dify json data: {block[5:]}")
continue
# flush any remaining text
buffer += decoder.decode(b"", final=True)
if buffer.strip().startswith("data:"):
try:
yield json.loads(buffer[5:])
except json.JSONDecodeError:
logger.warning(f"Drop invalid dify json data: {buffer[5:]}")
class DifyAPIClient:
def __init__(self, api_key: str, api_base: str = "https://api.dify.ai/v1"):
self.api_key = api_key
self.api_base = api_base
self.session = ClientSession(trust_env=True)
self.headers = {
"Authorization": f"Bearer {self.api_key}",
}
async def chat_messages(
self,
inputs: dict,
query: str,
user: str,
response_mode: str = "streaming",
conversation_id: str = "",
files: list[dict[str, Any]] | None = None,
timeout: float = 60,
) -> AsyncGenerator[dict[str, Any], None]:
if files is None:
files = []
url = f"{self.api_base}/chat-messages"
payload = locals()
payload.pop("self")
payload.pop("timeout")
logger.info(f"chat_messages payload: {payload}")
async with self.session.post(
url,
json=payload,
headers=self.headers,
timeout=timeout,
) as resp:
if resp.status != 200:
text = await resp.text()
raise Exception(
f"Dify /chat-messages 接口请求失败:{resp.status}. {text}",
)
async for event in _stream_sse(resp):
yield event
async def workflow_run(
self,
inputs: dict,
user: str,
response_mode: str = "streaming",
files: list[dict[str, Any]] | None = None,
timeout: float = 60,
):
if files is None:
files = []
url = f"{self.api_base}/workflows/run"
payload = locals()
payload.pop("self")
payload.pop("timeout")
logger.info(f"workflow_run payload: {payload}")
async with self.session.post(
url,
json=payload,
headers=self.headers,
timeout=timeout,
) as resp:
if resp.status != 200:
text = await resp.text()
raise Exception(
f"Dify /workflows/run 接口请求失败:{resp.status}. {text}",
)
async for event in _stream_sse(resp):
yield event
async def file_upload(
self,
user: str,
file_path: str | None = None,
file_data: bytes | None = None,
file_name: str | None = None,
mime_type: str | None = None,
) -> dict[str, Any]:
"""Upload a file to Dify. Must provide either file_path or file_data.
Args:
user: The user ID.
file_path: The path to the file to upload.
file_data: The file data in bytes.
file_name: Optional file name when using file_data.
Returns:
A dictionary containing the uploaded file information.
"""
url = f"{self.api_base}/files/upload"
form = FormData()
form.add_field("user", user)
if file_data is not None:
# 使用 bytes 数据
form.add_field(
"file",
file_data,
filename=file_name or "uploaded_file",
content_type=mime_type or "application/octet-stream",
)
elif file_path is not None:
# 使用文件路径
import os
with open(file_path, "rb") as f:
file_content = f.read()
form.add_field(
"file",
file_content,
filename=os.path.basename(file_path),
content_type=mime_type or "application/octet-stream",
)
else:
raise ValueError("file_path 和 file_data 不能同时为 None")
async with self.session.post(
url,
data=form,
headers=self.headers, # 不包含 Content-Type,让 aiohttp 自动设置
) as resp:
if resp.status != 200 and resp.status != 201:
text = await resp.text()
raise Exception(f"Dify 文件上传失败:{resp.status}. {text}")
return await resp.json() # {"id": "xxx", ...}
async def close(self):
await self.session.close()
async def get_chat_convs(self, user: str, limit: int = 20):
# conversations. GET
url = f"{self.api_base}/conversations"
payload = {
"user": user,
"limit": limit,
}
async with self.session.get(url, params=payload, headers=self.headers) as resp:
return await resp.json()
async def delete_chat_conv(self, user: str, conversation_id: str):
# conversation. DELETE
url = f"{self.api_base}/conversations/{conversation_id}"
payload = {
"user": user,
}
async with self.session.delete(url, json=payload, headers=self.headers) as resp:
return await resp.json()
async def rename(
self,
conversation_id: str,
name: str,
user: str,
auto_generate: bool = False,
):
# /conversations/:conversation_id/name
url = f"{self.api_base}/conversations/{conversation_id}/name"
payload = {
"user": user,
"name": name,
"auto_generate": auto_generate,
}
async with self.session.post(url, json=payload, headers=self.headers) as resp:
return await resp.json()
@@ -1,33 +1,31 @@
import sys import sys
import traceback import traceback
import typing as T import typing as T
from .base import BaseAgentRunner, AgentResponse, AgentState
from mcp.types import ( from ..hooks import BaseAgentRunHooks
BlobResourceContents, from ..tool_executor import BaseFunctionToolExecutor
CallToolResult, from ..run_context import ContextWrapper, TContext
EmbeddedResource, from ..response import AgentResponseData
ImageContent, from astrbot.core.provider.provider import Provider
TextContent,
TextResourceContents,
)
from astrbot import logger
from astrbot.core.message.message_event_result import ( from astrbot.core.message.message_event_result import (
MessageChain, MessageChain,
) )
from astrbot.core.provider.entities import ( from astrbot.core.provider.entities import (
LLMResponse,
ProviderRequest, ProviderRequest,
LLMResponse,
ToolCallMessageSegment,
AssistantMessageSegment,
ToolCallsResult, ToolCallsResult,
) )
from astrbot.core.provider.provider import Provider from mcp.types import (
TextContent,
from ..hooks import BaseAgentRunHooks ImageContent,
from ..message import AssistantMessageSegment, Message, ToolCallMessageSegment EmbeddedResource,
from ..response import AgentResponseData TextResourceContents,
from ..run_context import ContextWrapper, TContext BlobResourceContents,
from ..tool_executor import BaseFunctionToolExecutor CallToolResult,
from .base import AgentResponse, AgentState, BaseAgentRunner )
from astrbot import logger
if sys.version_info >= (3, 12): if sys.version_info >= (3, 12):
from typing import override from typing import override
@@ -55,19 +53,11 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
self.agent_hooks = agent_hooks self.agent_hooks = agent_hooks
self.run_context = run_context self.run_context = run_context
messages = [] def _transition_state(self, new_state: AgentState) -> None:
# append existing messages in the run context """转换 Agent 状态"""
for msg in request.contexts: if self._state != new_state:
messages.append(Message.model_validate(msg)) logger.debug(f"Agent state transition: {self._state} -> {new_state}")
if request.prompt is not None: self._state = new_state
m = await request.assemble_context()
messages.append(Message.model_validate(m))
if request.system_prompt:
messages.insert(
0,
Message(role="system", content=request.system_prompt),
)
self.run_context.messages = messages
async def _iter_llm_responses(self) -> T.AsyncGenerator[LLMResponse, None]: async def _iter_llm_responses(self) -> T.AsyncGenerator[LLMResponse, None]:
"""Yields chunks *and* a final LLMResponse.""" """Yields chunks *and* a final LLMResponse."""
@@ -80,7 +70,8 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
@override @override
async def step(self): async def step(self):
"""Process a single step of the agent. """
Process a single step of the agent.
This method should return the result of the step. This method should return the result of the step.
""" """
if not self.req: if not self.req:
@@ -97,26 +88,18 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
llm_resp_result = None llm_resp_result = None
async for llm_response in self._iter_llm_responses(): async for llm_response in self._iter_llm_responses():
assert isinstance(llm_response, LLMResponse)
if llm_response.is_chunk: if llm_response.is_chunk:
if llm_response.result_chain: if llm_response.result_chain:
yield AgentResponse( yield AgentResponse(
type="streaming_delta", type="streaming_delta",
data=AgentResponseData(chain=llm_response.result_chain), data=AgentResponseData(chain=llm_response.result_chain),
) )
elif llm_response.completion_text: else:
yield AgentResponse( yield AgentResponse(
type="streaming_delta", type="streaming_delta",
data=AgentResponseData( data=AgentResponseData(
chain=MessageChain().message(llm_response.completion_text), chain=MessageChain().message(llm_response.completion_text)
),
)
elif llm_response.reasoning_content:
yield AgentResponse(
type="streaming_delta",
data=AgentResponseData(
chain=MessageChain(type="reasoning").message(
llm_response.reasoning_content,
),
), ),
) )
continue continue
@@ -137,8 +120,8 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
type="err", type="err",
data=AgentResponseData( data=AgentResponseData(
chain=MessageChain().message( chain=MessageChain().message(
f"LLM 响应错误: {llm_resp.completion_text or '未知错误'}", f"LLM 响应错误: {llm_resp.completion_text or '未知错误'}"
), )
), ),
) )
@@ -146,13 +129,6 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
# 如果没有工具调用,转换到完成状态 # 如果没有工具调用,转换到完成状态
self.final_llm_resp = llm_resp self.final_llm_resp = llm_resp
self._transition_state(AgentState.DONE) self._transition_state(AgentState.DONE)
# record the final assistant message
self.run_context.messages.append(
Message(
role="assistant",
content=llm_resp.completion_text or "",
),
)
try: try:
await self.agent_hooks.on_agent_done(self.run_context, llm_resp) await self.agent_hooks.on_agent_done(self.run_context, llm_resp)
except Exception as e: except Exception as e:
@@ -168,7 +144,7 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
yield AgentResponse( yield AgentResponse(
type="llm_result", type="llm_result",
data=AgentResponseData( data=AgentResponseData(
chain=MessageChain().message(llm_resp.completion_text), chain=MessageChain().message(llm_resp.completion_text)
), ),
) )
@@ -179,16 +155,13 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
yield AgentResponse( yield AgentResponse(
type="tool_call", type="tool_call",
data=AgentResponseData( data=AgentResponseData(
chain=MessageChain(type="tool_call").message( chain=MessageChain().message(f"🔨 调用工具: {tool_call_name}")
f"🔨 调用工具: {tool_call_name}"
),
), ),
) )
async for result in self._handle_function_tools(self.req, llm_resp): async for result in self._handle_function_tools(self.req, llm_resp):
if isinstance(result, list): if isinstance(result, list):
tool_call_result_blocks = result tool_call_result_blocks = result
elif isinstance(result, MessageChain): elif isinstance(result, MessageChain):
result.type = "tool_call_result"
yield AgentResponse( yield AgentResponse(
type="tool_call_result", type="tool_call_result",
data=AgentResponseData(chain=result), data=AgentResponseData(chain=result),
@@ -196,28 +169,14 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
# 将结果添加到上下文中 # 将结果添加到上下文中
tool_calls_result = ToolCallsResult( tool_calls_result = ToolCallsResult(
tool_calls_info=AssistantMessageSegment( tool_calls_info=AssistantMessageSegment(
tool_calls=llm_resp.to_openai_to_calls_model(), role="assistant",
tool_calls=llm_resp.to_openai_tool_calls(),
content=llm_resp.completion_text, content=llm_resp.completion_text,
), ),
tool_calls_result=tool_call_result_blocks, tool_calls_result=tool_call_result_blocks,
) )
# record the assistant message with tool calls
self.run_context.messages.extend(
tool_calls_result.to_openai_messages_model()
)
self.req.append_tool_calls_result(tool_calls_result) self.req.append_tool_calls_result(tool_calls_result)
async def step_until_done(
self, max_step: int
) -> T.AsyncGenerator[AgentResponse, None]:
"""Process steps until the agent is done."""
step_count = 0
while not self.done() and step_count < max_step:
step_count += 1
async for resp in self.step():
yield resp
async def _handle_function_tools( async def _handle_function_tools(
self, self,
req: ProviderRequest, req: ProviderRequest,
@@ -239,50 +198,9 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
func_tool = req.func_tool.get_func(func_tool_name) func_tool = req.func_tool.get_func(func_tool_name)
logger.info(f"使用工具:{func_tool_name},参数:{func_tool_args}") logger.info(f"使用工具:{func_tool_name},参数:{func_tool_args}")
if not func_tool:
logger.warning(f"未找到指定的工具: {func_tool_name},将跳过。")
tool_call_result_blocks.append(
ToolCallMessageSegment(
role="tool",
tool_call_id=func_tool_id,
content=f"error: 未找到工具 {func_tool_name}",
),
)
continue
valid_params = {} # 参数过滤:只传递函数实际需要的参数
# 获取实际的 handler 函数
if func_tool.handler:
logger.debug(
f"工具 {func_tool_name} 期望的参数: {func_tool.parameters}",
)
if func_tool.parameters and func_tool.parameters.get("properties"):
expected_params = set(func_tool.parameters["properties"].keys())
valid_params = {
k: v
for k, v in func_tool_args.items()
if k in expected_params
}
# 记录被忽略的参数
ignored_params = set(func_tool_args.keys()) - set(
valid_params.keys(),
)
if ignored_params:
logger.warning(
f"工具 {func_tool_name} 忽略非期望参数: {ignored_params}",
)
else:
# 如果没有 handler(如 MCP 工具),使用所有参数
valid_params = func_tool_args
try: try:
await self.agent_hooks.on_tool_start( await self.agent_hooks.on_tool_start(
self.run_context, self.run_context, func_tool, func_tool_args
func_tool,
valid_params,
) )
except Exception as e: except Exception as e:
logger.error(f"Error in on_tool_start hook: {e}", exc_info=True) logger.error(f"Error in on_tool_start hook: {e}", exc_info=True)
@@ -290,21 +208,18 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
executor = self.tool_executor.execute( executor = self.tool_executor.execute(
tool=func_tool, tool=func_tool,
run_context=self.run_context, run_context=self.run_context,
**valid_params, # 只传递有效的参数 **func_tool_args,
) )
async for resp in executor:
_final_resp: CallToolResult | None = None
async for resp in executor: # type: ignore
if isinstance(resp, CallToolResult): if isinstance(resp, CallToolResult):
res = resp res = resp
_final_resp = resp
if isinstance(res.content[0], TextContent): if isinstance(res.content[0], TextContent):
tool_call_result_blocks.append( tool_call_result_blocks.append(
ToolCallMessageSegment( ToolCallMessageSegment(
role="tool", role="tool",
tool_call_id=func_tool_id, tool_call_id=func_tool_id,
content=res.content[0].text, content=res.content[0].text,
), )
) )
yield MessageChain().message(res.content[0].text) yield MessageChain().message(res.content[0].text)
elif isinstance(res.content[0], ImageContent): elif isinstance(res.content[0], ImageContent):
@@ -313,10 +228,10 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
role="tool", role="tool",
tool_call_id=func_tool_id, tool_call_id=func_tool_id,
content="返回了图片(已直接发送给用户)", content="返回了图片(已直接发送给用户)",
), )
) )
yield MessageChain(type="tool_direct_result").base64_image( yield MessageChain(type="tool_direct_result").base64_image(
res.content[0].data, res.content[0].data
) )
elif isinstance(res.content[0], EmbeddedResource): elif isinstance(res.content[0], EmbeddedResource):
resource = res.content[0].resource resource = res.content[0].resource
@@ -326,7 +241,7 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
role="tool", role="tool",
tool_call_id=func_tool_id, tool_call_id=func_tool_id,
content=resource.text, content=resource.text,
), )
) )
yield MessageChain().message(resource.text) yield MessageChain().message(resource.text)
elif ( elif (
@@ -339,52 +254,72 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
role="tool", role="tool",
tool_call_id=func_tool_id, tool_call_id=func_tool_id,
content="返回了图片(已直接发送给用户)", content="返回了图片(已直接发送给用户)",
), )
) )
yield MessageChain( yield MessageChain(
type="tool_direct_result", type="tool_direct_result"
).base64_image(resource.blob) ).base64_image(res.content[0].data)
else: else:
tool_call_result_blocks.append( tool_call_result_blocks.append(
ToolCallMessageSegment( ToolCallMessageSegment(
role="tool", role="tool",
tool_call_id=func_tool_id, tool_call_id=func_tool_id,
content="返回的数据类型不受支持", content="返回的数据类型不受支持",
), )
) )
yield MessageChain().message("返回的数据类型不受支持。") yield MessageChain().message("返回的数据类型不受支持。")
try:
await self.agent_hooks.on_tool_end(
self.run_context,
func_tool_name,
func_tool_args,
resp,
)
except Exception as e:
logger.error(
f"Error in on_tool_end hook: {e}", exc_info=True
)
elif resp is None: elif resp is None:
# Tool 直接请求发送消息给用户 # Tool 直接请求发送消息给用户
# 这里我们将直接结束 Agent Loop。 # 这里我们将直接结束 Agent Loop。
# 发送消息逻辑在 ToolExecutor 中处理了。
logger.warning(
f"{func_tool_name} 没有没有返回值或者将结果直接发送给用户,此工具调用不会被记录到历史中。"
)
self._transition_state(AgentState.DONE) self._transition_state(AgentState.DONE)
if res := self.run_context.event.get_result():
if res.chain:
yield MessageChain(
chain=res.chain, type="tool_direct_result"
)
try:
await self.agent_hooks.on_tool_end(
self.run_context, func_tool_name, func_tool_args, None
)
except Exception as e:
logger.error(
f"Error in on_tool_end hook: {e}", exc_info=True
)
else: else:
# 不应该出现其他类型
logger.warning( logger.warning(
f"Tool 返回了不支持的类型: {type(resp)},将忽略。", f"Tool 返回了不支持的类型: {type(resp)},将忽略。"
) )
try: try:
await self.agent_hooks.on_tool_end( await self.agent_hooks.on_tool_end(
self.run_context, self.run_context, func_tool_name, func_tool_args, None
func_tool, )
func_tool_args, except Exception as e:
_final_resp, logger.error(
) f"Error in on_tool_end hook: {e}", exc_info=True
except Exception as e: )
logger.error(f"Error in on_tool_end hook: {e}", exc_info=True)
self.run_context.event.clear_result()
except Exception as e: except Exception as e:
logger.warning(traceback.format_exc()) logger.warning(traceback.format_exc())
tool_call_result_blocks.append( tool_call_result_blocks.append(
ToolCallMessageSegment( ToolCallMessageSegment(
role="tool", role="tool",
tool_call_id=func_tool_id, tool_call_id=func_tool_id,
content=f"error: {e!s}", content=f"error: {str(e)}",
), )
) )
# 处理函数调用响应 # 处理函数调用响应
+52 -86
View File
@@ -1,82 +1,58 @@
from collections.abc import AsyncGenerator, Awaitable, Callable from dataclasses import dataclass
from typing import Any, Generic
import jsonschema
import mcp
from deprecated import deprecated from deprecated import deprecated
from pydantic import Field, model_validator from typing import Awaitable, Literal, Any, Optional
from pydantic.dataclasses import dataclass from .mcp_client import MCPClient
from astrbot.core.message.message_event_result import MessageEventResult
from .run_context import ContextWrapper, TContext
ParametersType = dict[str, Any]
ToolExecResult = str | mcp.types.CallToolResult
@dataclass @dataclass
class ToolSchema: class FunctionTool:
"""A class representing the schema of a tool for function calling.""" """A class representing a function tool that can be used in function calling."""
name: str
"""The name of the tool."""
description: str
"""The description of the tool."""
parameters: ParametersType
"""The parameters of the tool, in JSON Schema format."""
@model_validator(mode="after")
def validate_parameters(self) -> "ToolSchema":
jsonschema.validate(
self.parameters, jsonschema.Draft202012Validator.META_SCHEMA
)
return self
@dataclass
class FunctionTool(ToolSchema, Generic[TContext]):
"""A callable tool, for function calling."""
handler: (
Callable[..., Awaitable[str | None] | AsyncGenerator[MessageEventResult, None]]
| None
) = None
"""a callable that implements the tool's functionality. It should be an async function."""
name: str | None = None
parameters: dict | None = None
description: str | None = None
handler: Awaitable | None = None
"""处理函数, 当 origin 为 mcp 时,这个为空"""
handler_module_path: str | None = None handler_module_path: str | None = None
""" """处理函数的模块路径,当 origin 为 mcp 时,这个为空
The module path of the handler function. This is empty when the origin is mcp.
This field must be retained, as the handler will be wrapped in functools.partial during initialization, 必须要保留这个字段, handler 在初始化会被 functools.partial 包装,导致 handler 的 __module__ 为 functools
causing the handler's __module__ to be functools
""" """
active: bool = True active: bool = True
""" """是否激活"""
Whether the tool is active. This field is a special field for AstrBot.
You can ignore it when integrating with other frameworks. origin: Literal["local", "mcp"] = "local"
""" """函数工具的来源, local 为本地函数工具, mcp 为 MCP 服务"""
# MCP 相关字段
mcp_server_name: str | None = None
"""MCP 服务名称,当 origin 为 mcp 时有效"""
mcp_client: MCPClient | None = None
"""MCP 客户端,当 origin 为 mcp 时有效"""
def __repr__(self): def __repr__(self):
return f"FuncTool(name={self.name}, parameters={self.parameters}, description={self.description})" return f"FuncTool(name={self.name}, parameters={self.parameters}, description={self.description}, active={self.active}, origin={self.origin})"
async def call(self, context: ContextWrapper[TContext], **kwargs) -> ToolExecResult: def __dict__(self) -> dict[str, Any]:
"""Run the tool with the given arguments. The handler field has priority.""" """将 FunctionTool 转换为字典格式"""
raise NotImplementedError( return {
"FunctionTool.call() must be implemented by subclasses or set a handler." "name": self.name,
) "parameters": self.parameters,
"description": self.description,
"active": self.active,
"origin": self.origin,
"mcp_server_name": self.mcp_server_name,
}
@dataclass
class ToolSet: class ToolSet:
"""A set of function tools that can be used in function calling. """A set of function tools that can be used in function calling.
This class provides methods to add, remove, and retrieve tools, as well as This class provides methods to add, remove, and retrieve tools, as well as
convert the tools to different API formats (OpenAI, Anthropic, Google GenAI). convert the tools to different API formats (OpenAI, Anthropic, Google GenAI)."""
"""
tools: list[FunctionTool] = Field(default_factory=list) def __init__(self, tools: list[FunctionTool] = None):
self.tools: list[FunctionTool] = tools or []
def empty(self) -> bool: def empty(self) -> bool:
"""Check if the tool set is empty.""" """Check if the tool set is empty."""
@@ -95,7 +71,7 @@ class ToolSet:
"""Remove a tool by its name.""" """Remove a tool by its name."""
self.tools = [tool for tool in self.tools if tool.name != name] self.tools = [tool for tool in self.tools if tool.name != name]
def get_tool(self, name: str) -> FunctionTool | None: def get_tool(self, name: str) -> Optional[FunctionTool]:
"""Get a tool by its name.""" """Get a tool by its name."""
for tool in self.tools: for tool in self.tools:
if tool.name == name: if tool.name == name:
@@ -103,13 +79,7 @@ class ToolSet:
return None return None
@deprecated(reason="Use add_tool() instead", version="4.0.0") @deprecated(reason="Use add_tool() instead", version="4.0.0")
def add_func( def add_func(self, name: str, func_args: list, desc: str, handler: Awaitable):
self,
name: str,
func_args: list,
desc: str,
handler: Callable[..., Awaitable[Any]],
):
"""Add a function tool to the set.""" """Add a function tool to the set."""
params = { params = {
"type": "object", # hard-coded here "type": "object", # hard-coded here
@@ -134,7 +104,7 @@ class ToolSet:
self.remove_tool(name) self.remove_tool(name)
@deprecated(reason="Use get_tool() instead", version="4.0.0") @deprecated(reason="Use get_tool() instead", version="4.0.0")
def get_func(self, name: str) -> FunctionTool | None: def get_func(self, name: str) -> list[FunctionTool]:
"""Get all function tools.""" """Get all function tools."""
return self.get_tool(name) return self.get_tool(name)
@@ -155,9 +125,7 @@ class ToolSet:
}, },
} }
if ( if tool.parameters.get("properties") or not omit_empty_parameter_field:
tool.parameters and tool.parameters.get("properties")
) or not omit_empty_parameter_field:
func_def["function"]["parameters"] = tool.parameters func_def["function"]["parameters"] = tool.parameters
result.append(func_def) result.append(func_def)
@@ -167,14 +135,14 @@ class ToolSet:
"""Convert tools to Anthropic API format.""" """Convert tools to Anthropic API format."""
result = [] result = []
for tool in self.tools: for tool in self.tools:
input_schema = {"type": "object"}
if tool.parameters:
input_schema["properties"] = tool.parameters.get("properties", {})
input_schema["required"] = tool.parameters.get("required", [])
tool_def = { tool_def = {
"name": tool.name, "name": tool.name,
"description": tool.description, "description": tool.description,
"input_schema": input_schema, "input_schema": {
"type": "object",
"properties": tool.parameters.get("properties", {}),
"required": tool.parameters.get("required", []),
},
} }
result.append(tool_def) result.append(tool_def)
return result return result
@@ -207,8 +175,7 @@ class ToolSet:
if "type" in schema and schema["type"] in supported_types: if "type" in schema and schema["type"] in supported_types:
result["type"] = schema["type"] result["type"] = schema["type"]
if "format" in schema and schema["format"] in supported_formats.get( if "format" in schema and schema["format"] in supported_formats.get(
result["type"], result["type"], set()
set(),
): ):
result["format"] = schema["format"] result["format"] = schema["format"]
else: else:
@@ -243,15 +210,14 @@ class ToolSet:
return result return result
tools = [] tools = [
for tool in self.tools: {
d: dict[str, Any] = {
"name": tool.name, "name": tool.name,
"description": tool.description, "description": tool.description,
"parameters": convert_schema(tool.parameters),
} }
if tool.parameters: for tool in self.tools
d["parameters"] = convert_schema(tool.parameters) ]
tools.append(d)
declarations = {} declarations = {}
if tools: if tools:
+3 -9
View File
@@ -1,17 +1,11 @@
from collections.abc import AsyncGenerator
from typing import Any, Generic
import mcp import mcp
from typing import Any, Generic, AsyncGenerator
from .run_context import ContextWrapper, TContext from .run_context import TContext, ContextWrapper
from .tool import FunctionTool from .tool import FunctionTool
class BaseFunctionToolExecutor(Generic[TContext]): class BaseFunctionToolExecutor(Generic[TContext]):
@classmethod @classmethod
async def execute( async def execute(
cls, cls, tool: FunctionTool, run_context: ContextWrapper[TContext], **tool_args
tool: FunctionTool,
run_context: ContextWrapper[TContext],
**tool_args,
) -> AsyncGenerator[Any | mcp.types.CallToolResult, None]: ... ) -> AsyncGenerator[Any | mcp.types.CallToolResult, None]: ...
+8 -16
View File
@@ -1,19 +1,11 @@
from pydantic import Field from dataclasses import dataclass
from pydantic.dataclasses import dataclass from astrbot.core.provider import Provider
from astrbot.core.provider.entities import ProviderRequest
from astrbot.core.agent.run_context import ContextWrapper
from astrbot.core.platform.astr_message_event import AstrMessageEvent
from astrbot.core.star.context import Context
@dataclass(config={"arbitrary_types_allowed": True}) @dataclass
class AstrAgentContext: class AstrAgentContext:
context: Context provider: Provider
"""The star context instance""" first_provider_request: ProviderRequest
event: AstrMessageEvent curr_provider_request: ProviderRequest
"""The message event associated with the agent context.""" streaming: bool
extra: dict[str, str] = Field(default_factory=dict)
"""Customized extra data."""
AgentContextWrapper = ContextWrapper[AstrAgentContext]
-36
View File
@@ -1,36 +0,0 @@
from typing import Any
from mcp.types import CallToolResult
from astrbot.core.agent.hooks import BaseAgentRunHooks
from astrbot.core.agent.run_context import ContextWrapper
from astrbot.core.agent.tool import FunctionTool
from astrbot.core.astr_agent_context import AstrAgentContext
from astrbot.core.pipeline.context_utils import call_event_hook
from astrbot.core.star.star_handler import EventType
class MainAgentHooks(BaseAgentRunHooks[AstrAgentContext]):
async def on_agent_done(self, run_context, llm_response):
# 执行事件钩子
await call_event_hook(
run_context.context.event,
EventType.OnLLMResponseEvent,
llm_response,
)
async def on_tool_end(
self,
run_context: ContextWrapper[AstrAgentContext],
tool: FunctionTool[Any],
tool_args: dict | None,
tool_result: CallToolResult | None,
):
run_context.context.event.clear_result()
class EmptyAgentHooks(BaseAgentRunHooks[AstrAgentContext]):
pass
MAIN_AGENT_HOOKS = MainAgentHooks()
-94
View File
@@ -1,94 +0,0 @@
import traceback
from collections.abc import AsyncGenerator
from astrbot.core import logger
from astrbot.core.agent.runners.tool_loop_agent_runner import ToolLoopAgentRunner
from astrbot.core.astr_agent_context import AstrAgentContext
from astrbot.core.message.message_event_result import (
MessageChain,
MessageEventResult,
ResultContentType,
)
from astrbot.core.provider.entities import LLMResponse
AgentRunner = ToolLoopAgentRunner[AstrAgentContext]
async def run_agent(
agent_runner: AgentRunner,
max_step: int = 30,
show_tool_use: bool = True,
stream_to_general: bool = False,
show_reasoning: bool = False,
) -> AsyncGenerator[MessageChain | None, None]:
step_idx = 0
astr_event = agent_runner.run_context.context.event
while step_idx < max_step:
step_idx += 1
try:
async for resp in agent_runner.step():
if astr_event.is_stopped():
return
if resp.type == "tool_call_result":
msg_chain = resp.data["chain"]
if msg_chain.type == "tool_direct_result":
# tool_direct_result 用于标记 llm tool 需要直接发送给用户的内容
await astr_event.send(resp.data["chain"])
continue
# 对于其他情况,暂时先不处理
continue
elif resp.type == "tool_call":
if agent_runner.streaming:
# 用来标记流式响应需要分节
yield MessageChain(chain=[], type="break")
if show_tool_use:
await astr_event.send(resp.data["chain"])
continue
if stream_to_general and resp.type == "streaming_delta":
continue
if stream_to_general or not agent_runner.streaming:
content_typ = (
ResultContentType.LLM_RESULT
if resp.type == "llm_result"
else ResultContentType.GENERAL_RESULT
)
astr_event.set_result(
MessageEventResult(
chain=resp.data["chain"].chain,
result_content_type=content_typ,
),
)
yield
astr_event.clear_result()
elif resp.type == "streaming_delta":
chain = resp.data["chain"]
if chain.type == "reasoning" and not show_reasoning:
# display the reasoning content only when configured
continue
yield resp.data["chain"] # MessageChain
if agent_runner.done():
break
except Exception as e:
logger.error(traceback.format_exc())
err_msg = f"\n\nAstrBot 请求失败。\n错误类型: {type(e).__name__}\n错误信息: {e!s}\n\n请在平台日志查看和分享错误详情。\n"
error_llm_response = LLMResponse(
role="err",
completion_text=err_msg,
)
try:
await agent_runner.agent_hooks.on_agent_done(
agent_runner.run_context, error_llm_response
)
except Exception:
logger.exception("Error in on_agent_done hook")
if agent_runner.streaming:
yield MessageChain().message(err_msg)
else:
astr_event.set_result(MessageEventResult().message(err_msg))
return
-250
View File
@@ -1,250 +0,0 @@
import asyncio
import inspect
import traceback
import typing as T
import mcp
from astrbot import logger
from astrbot.core.agent.handoff import HandoffTool
from astrbot.core.agent.mcp_client import MCPTool
from astrbot.core.agent.run_context import ContextWrapper
from astrbot.core.agent.tool import FunctionTool, ToolSet
from astrbot.core.agent.tool_executor import BaseFunctionToolExecutor
from astrbot.core.astr_agent_context import AstrAgentContext
from astrbot.core.message.message_event_result import (
CommandResult,
MessageChain,
MessageEventResult,
)
from astrbot.core.provider.register import llm_tools
class FunctionToolExecutor(BaseFunctionToolExecutor[AstrAgentContext]):
@classmethod
async def execute(cls, tool, run_context, **tool_args):
"""执行函数调用。
Args:
event (AstrMessageEvent): 事件对象, 当 origin 为 local 时必须提供。
**kwargs: 函数调用的参数。
Returns:
AsyncGenerator[None | mcp.types.CallToolResult, None]
"""
if isinstance(tool, HandoffTool):
async for r in cls._execute_handoff(tool, run_context, **tool_args):
yield r
return
elif isinstance(tool, MCPTool):
async for r in cls._execute_mcp(tool, run_context, **tool_args):
yield r
return
else:
async for r in cls._execute_local(tool, run_context, **tool_args):
yield r
return
@classmethod
async def _execute_handoff(
cls,
tool: HandoffTool,
run_context: ContextWrapper[AstrAgentContext],
**tool_args,
):
input_ = tool_args.get("input")
# make toolset for the agent
tools = tool.agent.tools
if tools:
toolset = ToolSet()
for t in tools:
if isinstance(t, str):
_t = llm_tools.get_func(t)
if _t:
toolset.add_tool(_t)
elif isinstance(t, FunctionTool):
toolset.add_tool(t)
else:
toolset = None
ctx = run_context.context.context
event = run_context.context.event
umo = event.unified_msg_origin
prov_id = await ctx.get_current_chat_provider_id(umo)
llm_resp = await ctx.tool_loop_agent(
event=event,
chat_provider_id=prov_id,
prompt=input_,
system_prompt=tool.agent.instructions,
tools=toolset,
max_steps=30,
run_hooks=tool.agent.run_hooks,
)
yield mcp.types.CallToolResult(
content=[mcp.types.TextContent(type="text", text=llm_resp.completion_text)]
)
@classmethod
async def _execute_local(
cls,
tool: FunctionTool,
run_context: ContextWrapper[AstrAgentContext],
**tool_args,
):
event = run_context.context.event
if not event:
raise ValueError("Event must be provided for local function tools.")
is_override_call = False
for ty in type(tool).mro():
if "call" in ty.__dict__ and ty.__dict__["call"] is not FunctionTool.call:
is_override_call = True
break
# 检查 tool 下有没有 run 方法
if not tool.handler and not hasattr(tool, "run") and not is_override_call:
raise ValueError("Tool must have a valid handler or override 'run' method.")
awaitable = None
method_name = ""
if tool.handler:
awaitable = tool.handler
method_name = "decorator_handler"
elif is_override_call:
awaitable = tool.call
method_name = "call"
elif hasattr(tool, "run"):
awaitable = getattr(tool, "run")
method_name = "run"
if awaitable is None:
raise ValueError("Tool must have a valid handler or override 'run' method.")
wrapper = call_local_llm_tool(
context=run_context,
handler=awaitable,
method_name=method_name,
**tool_args,
)
while True:
try:
resp = await asyncio.wait_for(
anext(wrapper),
timeout=run_context.tool_call_timeout,
)
if resp is not None:
if isinstance(resp, mcp.types.CallToolResult):
yield resp
else:
text_content = mcp.types.TextContent(
type="text",
text=str(resp),
)
yield mcp.types.CallToolResult(content=[text_content])
else:
# NOTE: Tool 在这里直接请求发送消息给用户
# TODO: 是否需要判断 event.get_result() 是否为空?
# 如果为空,则说明没有发送消息给用户,并且返回值为空,将返回一个特殊的 TextContent,其内容如"工具没有返回内容"
if res := run_context.context.event.get_result():
if res.chain:
try:
await event.send(
MessageChain(
chain=res.chain,
type="tool_direct_result",
)
)
except Exception as e:
logger.error(
f"Tool 直接发送消息失败: {e}",
exc_info=True,
)
yield None
except asyncio.TimeoutError:
raise Exception(
f"tool {tool.name} execution timeout after {run_context.tool_call_timeout} seconds.",
)
except StopAsyncIteration:
break
@classmethod
async def _execute_mcp(
cls,
tool: FunctionTool,
run_context: ContextWrapper[AstrAgentContext],
**tool_args,
):
res = await tool.call(run_context, **tool_args)
if not res:
return
yield res
async def call_local_llm_tool(
context: ContextWrapper[AstrAgentContext],
handler: T.Callable[
...,
T.Awaitable[MessageEventResult | mcp.types.CallToolResult | str | None]
| T.AsyncGenerator[MessageEventResult | CommandResult | str | None, None],
],
method_name: str,
*args,
**kwargs,
) -> T.AsyncGenerator[T.Any, None]:
"""执行本地 LLM 工具的处理函数并处理其返回结果"""
ready_to_call = None # 一个协程或者异步生成器
trace_ = None
event = context.context.event
try:
if method_name == "run" or method_name == "decorator_handler":
ready_to_call = handler(event, *args, **kwargs)
elif method_name == "call":
ready_to_call = handler(context, *args, **kwargs)
else:
raise ValueError(f"未知的方法名: {method_name}")
except ValueError as e:
logger.error(f"调用本地 LLM 工具时出错: {e}", exc_info=True)
except TypeError:
logger.error("处理函数参数不匹配,请检查 handler 的定义。", exc_info=True)
except Exception as e:
trace_ = traceback.format_exc()
logger.error(f"调用本地 LLM 工具时出错: {e}\n{trace_}")
if not ready_to_call:
return
if inspect.isasyncgen(ready_to_call):
_has_yielded = False
try:
async for ret in ready_to_call:
# 这里逐步执行异步生成器, 对于每个 yield 返回的 ret, 执行下面的代码
# 返回值只能是 MessageEventResult 或者 None(无返回值)
_has_yielded = True
if isinstance(ret, (MessageEventResult, CommandResult)):
# 如果返回值是 MessageEventResult, 设置结果并继续
event.set_result(ret)
yield
else:
# 如果返回值是 None, 则不设置结果并继续
# 继续执行后续阶段
yield ret
if not _has_yielded:
# 如果这个异步生成器没有执行到 yield 分支
yield
except Exception as e:
logger.error(f"Previous Error: {trace_}")
raise e
elif inspect.iscoroutine(ready_to_call):
# 如果只是一个协程, 直接执行
ret = await ready_to_call
if isinstance(ret, (MessageEventResult, CommandResult)):
event.set_result(ret)
yield
else:
yield ret
+69 -68
View File
@@ -1,14 +1,12 @@
import os import os
import uuid import uuid
from typing import TypedDict, TypeVar
from astrbot.core import AstrBotConfig, logger from astrbot.core import AstrBotConfig, logger
from astrbot.core.utils.shared_preferences import SharedPreferences
from astrbot.core.config.astrbot_config import ASTRBOT_CONFIG_PATH from astrbot.core.config.astrbot_config import ASTRBOT_CONFIG_PATH
from astrbot.core.config.default import DEFAULT_CONFIG from astrbot.core.config.default import DEFAULT_CONFIG
from astrbot.core.platform.message_session import MessageSession from astrbot.core.platform.message_session import MessageSession
from astrbot.core.umop_config_router import UmopConfigRouter
from astrbot.core.utils.astrbot_path import get_astrbot_config_path from astrbot.core.utils.astrbot_path import get_astrbot_config_path
from astrbot.core.utils.shared_preferences import SharedPreferences from typing import TypeVar, TypedDict
_VT = TypeVar("_VT") _VT = TypeVar("_VT")
@@ -17,12 +15,14 @@ class ConfInfo(TypedDict):
"""Configuration information for a specific session or platform.""" """Configuration information for a specific session or platform."""
id: str # UUID of the configuration or "default" id: str # UUID of the configuration or "default"
umop: list[str] # Unified Message Origin Pattern
name: str name: str
path: str # File name to the configuration file path: str # File name to the configuration file
DEFAULT_CONFIG_CONF_INFO = ConfInfo( DEFAULT_CONFIG_CONF_INFO = ConfInfo(
id="default", id="default",
umop=["::"],
name="default", name="default",
path=ASTRBOT_CONFIG_PATH, path=ASTRBOT_CONFIG_PATH,
) )
@@ -31,35 +31,18 @@ DEFAULT_CONFIG_CONF_INFO = ConfInfo(
class AstrBotConfigManager: class AstrBotConfigManager:
"""A class to manage the system configuration of AstrBot, aka ACM""" """A class to manage the system configuration of AstrBot, aka ACM"""
def __init__( def __init__(self, default_config: AstrBotConfig, sp: SharedPreferences):
self,
default_config: AstrBotConfig,
ucr: UmopConfigRouter,
sp: SharedPreferences,
):
self.sp = sp self.sp = sp
self.ucr = ucr
self.confs: dict[str, AstrBotConfig] = {} self.confs: dict[str, AstrBotConfig] = {}
"""uuid / "default" -> AstrBotConfig""" """uuid / "default" -> AstrBotConfig"""
self.confs["default"] = default_config self.confs["default"] = default_config
self.abconf_data = None
self._load_all_configs() self._load_all_configs()
def _get_abconf_data(self) -> dict:
"""获取所有的 abconf 数据"""
if self.abconf_data is None:
self.abconf_data = self.sp.get(
"abconf_mapping",
{},
scope="global",
scope_id="global",
)
return self.abconf_data
def _load_all_configs(self): def _load_all_configs(self):
"""Load all configurations from the shared preferences.""" """Load all configurations from the shared preferences."""
abconf_data = self._get_abconf_data() abconf_data = self.sp.get(
self.abconf_data = abconf_data "abconf_mapping", {}, scope="global", scope_id="global"
)
for uuid_, meta in abconf_data.items(): for uuid_, meta in abconf_data.items():
filename = meta["path"] filename = meta["path"]
conf_path = os.path.join(get_astrbot_config_path(), filename) conf_path = os.path.join(get_astrbot_config_path(), filename)
@@ -68,20 +51,30 @@ class AstrBotConfigManager:
self.confs[uuid_] = conf self.confs[uuid_] = conf
else: else:
logger.warning( logger.warning(
f"Config file {conf_path} for UUID {uuid_} does not exist, skipping.", f"Config file {conf_path} for UUID {uuid_} does not exist, skipping."
) )
continue continue
def _is_umo_match(self, p1: str, p2: str) -> bool:
"""判断 p2 umo 是否逻辑包含于 p1 umo"""
p1_ls = p1.split(":")
p2_ls = p2.split(":")
if len(p1_ls) != 3 or len(p2_ls) != 3:
return False # 非法格式
return all(p == "" or p == "*" or p == t for p, t in zip(p1_ls, p2_ls))
def _load_conf_mapping(self, umo: str | MessageSession) -> ConfInfo: def _load_conf_mapping(self, umo: str | MessageSession) -> ConfInfo:
"""获取指定 umo 的配置文件 uuid, 如果不存在则返回默认配置(返回 "default") """获取指定 umo 的配置文件 uuid, 如果不存在则返回默认配置(返回 "default")
Returns: Returns:
ConfInfo: 包含配置文件的 uuid, 路径和名称等信息, 是一个 dict 类型 ConfInfo: 包含配置文件的 uuid, 路径和名称等信息, 是一个 dict 类型
""" """
# uuid -> { "path": str, "name": str } # uuid -> { "umop": list, "path": str, "name": str }
abconf_data = self._get_abconf_data() abconf_data = self.sp.get(
"abconf_mapping", {}, scope="global", scope_id="global"
)
if isinstance(umo, MessageSession): if isinstance(umo, MessageSession):
umo = str(umo) umo = str(umo)
else: else:
@@ -90,13 +83,10 @@ class AstrBotConfigManager:
except Exception: except Exception:
return DEFAULT_CONFIG_CONF_INFO return DEFAULT_CONFIG_CONF_INFO
conf_id = self.ucr.get_conf_id_for_umop(umo) for uuid_, meta in abconf_data.items():
if conf_id: for pattern in meta["umop"]:
meta = abconf_data.get(conf_id) if self._is_umo_match(pattern, umo):
if meta and isinstance(meta, dict): return ConfInfo(**meta, id=uuid_)
# the bind relation between umo and conf is defined in ucr now, so we remove "umop" here
meta.pop("umop", None)
return ConfInfo(**meta, id=conf_id)
return DEFAULT_CONFIG_CONF_INFO return DEFAULT_CONFIG_CONF_INFO
@@ -104,22 +94,27 @@ class AstrBotConfigManager:
self, self,
abconf_path: str, abconf_path: str,
abconf_id: str, abconf_id: str,
umo_parts: list[str] | list[MessageSession],
abconf_name: str | None = None, abconf_name: str | None = None,
) -> None: ) -> None:
"""保存配置文件的映射关系""" """保存配置文件的映射关系"""
for part in umo_parts:
if isinstance(part, MessageSession):
part = str(part)
elif not isinstance(part, str):
raise ValueError(
"umo_parts must be a list of strings or MessageSession instances"
)
abconf_data = self.sp.get( abconf_data = self.sp.get(
"abconf_mapping", "abconf_mapping", {}, scope="global", scope_id="global"
{},
scope="global",
scope_id="global",
) )
random_word = abconf_name or uuid.uuid4().hex[:8] random_word = abconf_name or uuid.uuid4().hex[:8]
abconf_data[abconf_id] = { abconf_data[abconf_id] = {
"umop": umo_parts,
"path": abconf_path, "path": abconf_path,
"name": random_word, "name": random_word,
} }
self.sp.put("abconf_mapping", abconf_data, scope="global", scope_id="global") self.sp.put("abconf_mapping", abconf_data, scope="global", scope_id="global")
self.abconf_data = abconf_data
def get_conf(self, umo: str | MessageSession | None) -> AstrBotConfig: def get_conf(self, umo: str | MessageSession | None) -> AstrBotConfig:
"""获取指定 umo 的配置文件。如果不存在,则 fallback 到默认配置文件。""" """获取指定 umo 的配置文件。如果不存在,则 fallback 到默认配置文件。"""
@@ -151,26 +146,31 @@ class AstrBotConfigManager:
def get_conf_list(self) -> list[ConfInfo]: def get_conf_list(self) -> list[ConfInfo]:
"""获取所有配置文件的元数据列表""" """获取所有配置文件的元数据列表"""
conf_list = [] conf_list = []
abconf_mapping = self._get_abconf_data()
for uuid_, meta in abconf_mapping.items():
if not isinstance(meta, dict):
continue
meta.pop("umop", None)
conf_list.append(ConfInfo(**meta, id=uuid_))
conf_list.append(DEFAULT_CONFIG_CONF_INFO) conf_list.append(DEFAULT_CONFIG_CONF_INFO)
abconf_mapping = self.sp.get(
"abconf_mapping", {}, scope="global", scope_id="global"
)
for uuid_, meta in abconf_mapping.items():
conf_list.append(ConfInfo(**meta, id=uuid_))
return conf_list return conf_list
def create_conf( def create_conf(
self, self,
umo_parts: list[str] | list[MessageSession],
config: dict = DEFAULT_CONFIG, config: dict = DEFAULT_CONFIG,
name: str | None = None, name: str | None = None,
) -> str: ) -> str:
"""
umo 由三个部分组成 [platform_id]:[message_type]:[session_id]。
umo_parts 可以是 "::" (代表所有), 可以是 "[platform_id]::" (代表指定平台下的所有类型消息和会话)。
"""
conf_uuid = str(uuid.uuid4()) conf_uuid = str(uuid.uuid4())
conf_file_name = f"abconf_{conf_uuid}.json" conf_file_name = f"abconf_{conf_uuid}.json"
conf_path = os.path.join(get_astrbot_config_path(), conf_file_name) conf_path = os.path.join(get_astrbot_config_path(), conf_file_name)
conf = AstrBotConfig(config_path=conf_path, default_config=config) conf = AstrBotConfig(config_path=conf_path, default_config=config)
conf.save_config() conf.save_config()
self._save_conf_mapping(conf_file_name, conf_uuid, abconf_name=name) self._save_conf_mapping(conf_file_name, conf_uuid, umo_parts, abconf_name=name)
self.confs[conf_uuid] = conf self.confs[conf_uuid] = conf
return conf_uuid return conf_uuid
@@ -185,17 +185,13 @@ class AstrBotConfigManager:
Raises: Raises:
ValueError: 如果试图删除默认配置文件 ValueError: 如果试图删除默认配置文件
""" """
if conf_id == "default": if conf_id == "default":
raise ValueError("不能删除默认配置文件") raise ValueError("不能删除默认配置文件")
# 从映射中移除 # 从映射中移除
abconf_data = self.sp.get( abconf_data = self.sp.get(
"abconf_mapping", "abconf_mapping", {}, scope="global", scope_id="global"
{},
scope="global",
scope_id="global",
) )
if conf_id not in abconf_data: if conf_id not in abconf_data:
logger.warning(f"配置文件 {conf_id} 不存在于映射中") logger.warning(f"配置文件 {conf_id} 不存在于映射中")
@@ -203,8 +199,7 @@ class AstrBotConfigManager:
# 获取配置文件路径 # 获取配置文件路径
conf_path = os.path.join( conf_path = os.path.join(
get_astrbot_config_path(), get_astrbot_config_path(), abconf_data[conf_id]["path"]
abconf_data[conf_id]["path"],
) )
# 删除配置文件 # 删除配置文件
@@ -223,30 +218,28 @@ class AstrBotConfigManager:
# 从映射中移除 # 从映射中移除
del abconf_data[conf_id] del abconf_data[conf_id]
self.sp.put("abconf_mapping", abconf_data, scope="global", scope_id="global") self.sp.put("abconf_mapping", abconf_data, scope="global", scope_id="global")
self.abconf_data = abconf_data
logger.info(f"成功删除配置文件 {conf_id}") logger.info(f"成功删除配置文件 {conf_id}")
return True return True
def update_conf_info(self, conf_id: str, name: str | None = None) -> bool: def update_conf_info(
self, conf_id: str, name: str | None = None, umo_parts: list[str] | None = None
) -> bool:
"""更新配置文件信息 """更新配置文件信息
Args: Args:
conf_id: 配置文件的 UUID conf_id: 配置文件的 UUID
name: 新的配置文件名称 (可选) name: 新的配置文件名称 (可选)
umo_parts: 新的 UMO 部分列表 (可选)
Returns: Returns:
bool: 更新是否成功 bool: 更新是否成功
""" """
if conf_id == "default": if conf_id == "default":
raise ValueError("不能更新默认配置文件的信息") raise ValueError("不能更新默认配置文件的信息")
abconf_data = self.sp.get( abconf_data = self.sp.get(
"abconf_mapping", "abconf_mapping", {}, scope="global", scope_id="global"
{},
scope="global",
scope_id="global",
) )
if conf_id not in abconf_data: if conf_id not in abconf_data:
logger.warning(f"配置文件 {conf_id} 不存在于映射中") logger.warning(f"配置文件 {conf_id} 不存在于映射中")
@@ -256,17 +249,25 @@ class AstrBotConfigManager:
if name is not None: if name is not None:
abconf_data[conf_id]["name"] = name abconf_data[conf_id]["name"] = name
# 更新 UMO 部分
if umo_parts is not None:
# 验证 UMO 部分格式
for part in umo_parts:
if isinstance(part, MessageSession):
part = str(part)
elif not isinstance(part, str):
raise ValueError(
"umo_parts must be a list of strings or MessageSession instances"
)
abconf_data[conf_id]["umop"] = umo_parts
# 保存更新 # 保存更新
self.sp.put("abconf_mapping", abconf_data, scope="global", scope_id="global") self.sp.put("abconf_mapping", abconf_data, scope="global", scope_id="global")
self.abconf_data = abconf_data
logger.info(f"成功更新配置文件 {conf_id} 的信息") logger.info(f"成功更新配置文件 {conf_id} 的信息")
return True return True
def g( def g(
self, self, umo: str | None = None, key: str | None = None, default: _VT = None
umo: str | None = None,
key: str | None = None,
default: _VT = None,
) -> _VT: ) -> _VT:
"""获取配置项。umo 为 None 时使用默认配置""" """获取配置项。umo 为 None 时使用默认配置"""
if umo is None: if umo is None:
+2 -2
View File
@@ -1,9 +1,9 @@
from .default import DEFAULT_CONFIG, VERSION, DB_PATH
from .astrbot_config import * from .astrbot_config import *
from .default import DB_PATH, DEFAULT_CONFIG, VERSION
__all__ = [ __all__ = [
"DB_PATH",
"DEFAULT_CONFIG", "DEFAULT_CONFIG",
"VERSION", "VERSION",
"DB_PATH",
"AstrBotConfig", "AstrBotConfig",
] ]
+26 -32
View File
@@ -1,11 +1,10 @@
import enum import os
import json import json
import logging import logging
import os import enum
from astrbot.core.utils.astrbot_path import get_astrbot_data_path
from .default import DEFAULT_CONFIG, DEFAULT_VALUE_MAP from .default import DEFAULT_CONFIG, DEFAULT_VALUE_MAP
from typing import Dict
from astrbot.core.utils.astrbot_path import get_astrbot_data_path
ASTRBOT_CONFIG_PATH = os.path.join(get_astrbot_data_path(), "cmd_config.json") ASTRBOT_CONFIG_PATH = os.path.join(get_astrbot_data_path(), "cmd_config.json")
logger = logging.getLogger("astrbot") logger = logging.getLogger("astrbot")
@@ -24,15 +23,11 @@ class AstrBotConfig(dict):
- 如果传入了 schema,将会通过 schema 解析出 default_config,此时传入的 default_config 会被忽略。 - 如果传入了 schema,将会通过 schema 解析出 default_config,此时传入的 default_config 会被忽略。
""" """
config_path: str
default_config: dict
schema: dict | None
def __init__( def __init__(
self, self,
config_path: str = ASTRBOT_CONFIG_PATH, config_path: str = ASTRBOT_CONFIG_PATH,
default_config: dict = DEFAULT_CONFIG, default_config: dict = DEFAULT_CONFIG,
schema: dict | None = None, schema: dict = None,
): ):
super().__init__() super().__init__()
@@ -50,7 +45,7 @@ class AstrBotConfig(dict):
json.dump(default_config, f, indent=4, ensure_ascii=False) json.dump(default_config, f, indent=4, ensure_ascii=False)
object.__setattr__(self, "first_deploy", True) # 标记第一次部署 object.__setattr__(self, "first_deploy", True) # 标记第一次部署
with open(config_path, encoding="utf-8-sig") as f: with open(config_path, "r", encoding="utf-8-sig") as f:
conf_str = f.read() conf_str = f.read()
conf = json.loads(conf_str) conf = json.loads(conf_str)
@@ -70,7 +65,7 @@ class AstrBotConfig(dict):
for k, v in schema.items(): for k, v in schema.items():
if v["type"] not in DEFAULT_VALUE_MAP: if v["type"] not in DEFAULT_VALUE_MAP:
raise TypeError( raise TypeError(
f"不受支持的配置类型 {v['type']}。支持的类型有:{DEFAULT_VALUE_MAP.keys()}", f"不受支持的配置类型 {v['type']}。支持的类型有:{DEFAULT_VALUE_MAP.keys()}"
) )
if "default" in v: if "default" in v:
default = v["default"] default = v["default"]
@@ -87,7 +82,7 @@ class AstrBotConfig(dict):
return conf return conf
def check_config_integrity(self, refer_conf: dict, conf: dict, path=""): def check_config_integrity(self, refer_conf: Dict, conf: Dict, path=""):
"""检查配置完整性,如果有新的配置项或顺序不一致则返回 True""" """检查配置完整性,如果有新的配置项或顺序不一致则返回 True"""
has_new = False has_new = False
@@ -102,28 +97,27 @@ class AstrBotConfig(dict):
logger.info(f"检查到配置项 {path_} 不存在,已插入默认值 {value}") logger.info(f"检查到配置项 {path_} 不存在,已插入默认值 {value}")
new_conf[key] = value new_conf[key] = value
has_new = True has_new = True
elif conf[key] is None: else:
# 配置项为 None,使用默认值 if conf[key] is None:
new_conf[key] = value # 配置项为 None,使用默认值
has_new = True
elif isinstance(value, dict):
# 递归检查子配置项
if not isinstance(conf[key], dict):
# 类型不匹配,使用默认值
new_conf[key] = value new_conf[key] = value
has_new = True has_new = True
elif isinstance(value, dict):
# 递归检查子配置项
if not isinstance(conf[key], dict):
# 类型不匹配,使用默认值
new_conf[key] = value
has_new = True
else:
# 递归检查并同步顺序
child_has_new = self.check_config_integrity(
value, conf[key], path + "." + key if path else key
)
new_conf[key] = conf[key]
has_new |= child_has_new
else: else:
# 递归检查并同步顺序 # 直接使用现有配置
child_has_new = self.check_config_integrity(
value,
conf[key],
path + "." + key if path else key,
)
new_conf[key] = conf[key] new_conf[key] = conf[key]
has_new |= child_has_new
else:
# 直接使用现有配置
new_conf[key] = conf[key]
# 检查是否存在参考配置中没有的配置项 # 检查是否存在参考配置中没有的配置项
for key in list(conf.keys()): for key in list(conf.keys()):
@@ -146,7 +140,7 @@ class AstrBotConfig(dict):
return has_new return has_new
def save_config(self, replace_config: dict | None = None): def save_config(self, replace_config: Dict = None):
"""将配置写入文件 """将配置写入文件
如果传入 replace_config,则将配置替换为 replace_config 如果传入 replace_config,则将配置替换为 replace_config
File diff suppressed because it is too large Load Diff
-110
View File
@@ -1,110 +0,0 @@
"""
配置元数据国际化工具
提供配置元数据的国际化键转换功能
"""
from typing import Any
class ConfigMetadataI18n:
"""配置元数据国际化转换器"""
@staticmethod
def _get_i18n_key(group: str, section: str, field: str, attr: str) -> str:
"""
生成国际化键
Args:
group: 配置组,如 'ai_group', 'platform_group'
section: 配置节,如 'agent_runner', 'general'
field: 字段名,如 'enable', 'default_provider'
attr: 属性类型,如 'description', 'hint', 'labels'
Returns:
国际化键,格式如: 'ai_group.agent_runner.enable.description'
"""
if field:
return f"{group}.{section}.{field}.{attr}"
else:
return f"{group}.{section}.{attr}"
@staticmethod
def convert_to_i18n_keys(metadata: dict[str, Any]) -> dict[str, Any]:
"""
将配置元数据转换为使用国际化键
Args:
metadata: 原始配置元数据字典
Returns:
使用国际化键的配置元数据字典
"""
result = {}
for group_key, group_data in metadata.items():
group_result = {
"name": f"{group_key}.name",
"metadata": {},
}
for section_key, section_data in group_data.get("metadata", {}).items():
section_result = {
"description": f"{group_key}.{section_key}.description",
"type": section_data.get("type"),
}
# 复制其他属性
for key in ["items", "condition", "_special", "invisible"]:
if key in section_data:
section_result[key] = section_data[key]
# 处理 hint
if "hint" in section_data:
section_result["hint"] = f"{group_key}.{section_key}.hint"
# 处理 items 中的字段
if "items" in section_data and isinstance(section_data["items"], dict):
items_result = {}
for field_key, field_data in section_data["items"].items():
# 处理嵌套的点号字段名(如 provider_settings.enable
field_name = field_key
field_result = {}
# 复制基本属性
for attr in [
"type",
"condition",
"_special",
"invisible",
"options",
]:
if attr in field_data:
field_result[attr] = field_data[attr]
# 转换文本属性为国际化键
if "description" in field_data:
field_result["description"] = (
f"{group_key}.{section_key}.{field_name}.description"
)
if "hint" in field_data:
field_result["hint"] = (
f"{group_key}.{section_key}.{field_name}.hint"
)
if "labels" in field_data:
field_result["labels"] = (
f"{group_key}.{section_key}.{field_name}.labels"
)
items_result[field_key] = field_result
section_result["items"] = items_result
group_result["metadata"][section_key] = section_result
result[group_key] = group_result
return result
+35 -147
View File
@@ -1,14 +1,13 @@
"""AstrBot 会话-对话管理器, 维护两个本地存储, 其中一个是 json 格式的shared_preferences, 另外一个是数据库. """
AstrBot 会话-对话管理器, 维护两个本地存储, 其中一个是 json 格式的shared_preferences, 另外一个是数据库
在 AstrBot 中, 会话和对话是独立的, 会话用于标记对话窗口, 例如群聊"123456789"可以建立一个会话, 在 AstrBot 中, 会话和对话是独立的, 会话用于标记对话窗口, 例如群聊"123456789"可以建立一个会话,
在一个会话中可以建立多个对话, 并且支持对话的切换和删除 在一个会话中可以建立多个对话, 并且支持对话的切换和删除
""" """
import json import json
from collections.abc import Awaitable, Callable
from astrbot.core import sp from astrbot.core import sp
from astrbot.core.agent.message import AssistantMessageSegment, UserMessageSegment from typing import Dict, List
from astrbot.core.db import BaseDatabase from astrbot.core.db import BaseDatabase
from astrbot.core.db.po import Conversation, ConversationV2 from astrbot.core.db.po import Conversation, ConversationV2
@@ -17,45 +16,10 @@ class ConversationManager:
"""负责管理会话与 LLM 的对话,某个会话当前正在用哪个对话。""" """负责管理会话与 LLM 的对话,某个会话当前正在用哪个对话。"""
def __init__(self, db_helper: BaseDatabase): def __init__(self, db_helper: BaseDatabase):
self.session_conversations: dict[str, str] = {} self.session_conversations: Dict[str, str] = {}
self.db = db_helper self.db = db_helper
self.save_interval = 60 # 每 60 秒保存一次 self.save_interval = 60 # 每 60 秒保存一次
# 会话删除回调函数列表(用于级联清理,如知识库配置)
self._on_session_deleted_callbacks: list[Callable[[str], Awaitable[None]]] = []
def register_on_session_deleted(
self,
callback: Callable[[str], Awaitable[None]],
) -> None:
"""注册会话删除回调函数.
其他模块可以注册回调来响应会话删除事件,实现级联清理。
例如:知识库模块可以注册回调来清理会话的知识库配置。
Args:
callback: 回调函数,接收会话ID (unified_msg_origin) 作为参数
"""
self._on_session_deleted_callbacks.append(callback)
async def _trigger_session_deleted(self, unified_msg_origin: str) -> None:
"""触发会话删除回调.
Args:
unified_msg_origin: 会话ID
"""
for callback in self._on_session_deleted_callbacks:
try:
await callback(unified_msg_origin)
except Exception as e:
from astrbot.core import logger
logger.error(
f"会话删除回调执行失败 (session: {unified_msg_origin}): {e}",
)
def _convert_conv_from_v2_to_v1(self, conv_v2: ConversationV2) -> Conversation: def _convert_conv_from_v2_to_v1(self, conv_v2: ConversationV2) -> Conversation:
"""将 ConversationV2 对象转换为 Conversation 对象""" """将 ConversationV2 对象转换为 Conversation 对象"""
created_at = int(conv_v2.created_at.timestamp()) created_at = int(conv_v2.created_at.timestamp())
@@ -79,13 +43,12 @@ class ConversationManager:
title: str | None = None, title: str | None = None,
persona_id: str | None = None, persona_id: str | None = None,
) -> str: ) -> str:
"""新建对话,并将当前会话的对话转移到新对话. """新建对话,并将当前会话的对话转移到新对话
Args: Args:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
Returns: Returns:
conversation_id (str): 对话 ID, 是 uuid 格式的字符串 conversation_id (str): 对话 ID, 是 uuid 格式的字符串
""" """
if not platform_id: if not platform_id:
# 如果没有提供 platform_id,则从 unified_msg_origin 中解析 # 如果没有提供 platform_id,则从 unified_msg_origin 中解析
@@ -111,46 +74,30 @@ class ConversationManager:
Args: Args:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
conversation_id (str): 对话 ID, 是 uuid 格式的字符串 conversation_id (str): 对话 ID, 是 uuid 格式的字符串
""" """
self.session_conversations[unified_msg_origin] = conversation_id self.session_conversations[unified_msg_origin] = conversation_id
await sp.session_put(unified_msg_origin, "sel_conv_id", conversation_id) await sp.session_put(unified_msg_origin, "sel_conv_id", conversation_id)
async def delete_conversation( async def delete_conversation(
self, self, unified_msg_origin: str, conversation_id: str | None = None
unified_msg_origin: str,
conversation_id: str | None = None,
): ):
"""删除会话的对话,当 conversation_id 为 None 时删除会话当前的对话 """删除会话的对话,当 conversation_id 为 None 时删除会话当前的对话
Args: Args:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
conversation_id (str): 对话 ID, 是 uuid 格式的字符串 conversation_id (str): 对话 ID, 是 uuid 格式的字符串
""" """
f = False
if not conversation_id: if not conversation_id:
conversation_id = self.session_conversations.get(unified_msg_origin) conversation_id = self.session_conversations.get(unified_msg_origin)
if conversation_id:
f = True
if conversation_id: if conversation_id:
await self.db.delete_conversation(cid=conversation_id) await self.db.delete_conversation(cid=conversation_id)
curr_cid = await self.get_curr_conversation_id(unified_msg_origin) if f:
if curr_cid == conversation_id:
self.session_conversations.pop(unified_msg_origin, None) self.session_conversations.pop(unified_msg_origin, None)
await sp.session_remove(unified_msg_origin, "sel_conv_id") await sp.session_remove(unified_msg_origin, "sel_conv_id")
async def delete_conversations_by_user_id(self, unified_msg_origin: str):
"""删除会话的所有对话
Args:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
"""
await self.db.delete_conversations_by_user_id(user_id=unified_msg_origin)
self.session_conversations.pop(unified_msg_origin, None)
await sp.session_remove(unified_msg_origin, "sel_conv_id")
# 触发会话删除回调(级联清理)
await self._trigger_session_deleted(unified_msg_origin)
async def get_curr_conversation_id(self, unified_msg_origin: str) -> str | None: async def get_curr_conversation_id(self, unified_msg_origin: str) -> str | None:
"""获取会话当前的对话 ID """获取会话当前的对话 ID
@@ -158,7 +105,6 @@ class ConversationManager:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
Returns: Returns:
conversation_id (str): 对话 ID, 是 uuid 格式的字符串 conversation_id (str): 对话 ID, 是 uuid 格式的字符串
""" """
ret = self.session_conversations.get(unified_msg_origin, None) ret = self.session_conversations.get(unified_msg_origin, None)
if not ret: if not ret:
@@ -173,15 +119,13 @@ class ConversationManager:
conversation_id: str, conversation_id: str,
create_if_not_exists: bool = False, create_if_not_exists: bool = False,
) -> Conversation | None: ) -> Conversation | None:
"""获取会话的对话. """获取会话的对话
Args: Args:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
conversation_id (str): 对话 ID, 是 uuid 格式的字符串 conversation_id (str): 对话 ID, 是 uuid 格式的字符串
create_if_not_exists (bool): 如果对话不存在,是否创建一个新的对话
Returns: Returns:
conversation (Conversation): 对话对象 conversation (Conversation): 对话对象
""" """
conv = await self.db.get_conversation_by_id(cid=conversation_id) conv = await self.db.get_conversation_by_id(cid=conversation_id)
if not conv and create_if_not_exists: if not conv and create_if_not_exists:
@@ -194,22 +138,18 @@ class ConversationManager:
return conv_res return conv_res
async def get_conversations( async def get_conversations(
self, self, unified_msg_origin: str | None = None, platform_id: str | None = None
unified_msg_origin: str | None = None, ) -> List[Conversation]:
platform_id: str | None = None, """获取对话列表
) -> list[Conversation]:
"""获取对话列表.
Args: Args:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id,可选 unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id,可选
platform_id (str): 平台 ID, 可选参数, 用于过滤对话 platform_id (str): 平台 ID, 可选参数, 用于过滤对话
Returns: Returns:
conversations (List[Conversation]): 对话对象列表 conversations (List[Conversation]): 对话对象列表
""" """
convs = await self.db.get_conversations( convs = await self.db.get_conversations(
user_id=unified_msg_origin, user_id=unified_msg_origin, platform_id=platform_id
platform_id=platform_id,
) )
convs_res = [] convs_res = []
for conv in convs: for conv in convs:
@@ -225,7 +165,7 @@ class ConversationManager:
search_query: str = "", search_query: str = "",
**kwargs, **kwargs,
) -> tuple[list[Conversation], int]: ) -> tuple[list[Conversation], int]:
"""获取过滤后的对话列表. """获取过滤后的对话列表
Args: Args:
page (int): 页码, 默认为 1 page (int): 页码, 默认为 1
@@ -234,7 +174,6 @@ class ConversationManager:
search_query (str): 搜索查询字符串, 可选 search_query (str): 搜索查询字符串, 可选
Returns: Returns:
conversations (list[Conversation]): 对话对象列表 conversations (list[Conversation]): 对话对象列表
""" """
convs, cnt = await self.db.get_filtered_conversations( convs, cnt = await self.db.get_filtered_conversations(
page=page, page=page,
@@ -256,14 +195,13 @@ class ConversationManager:
history: list[dict] | None = None, history: list[dict] | None = None,
title: str | None = None, title: str | None = None,
persona_id: str | None = None, persona_id: str | None = None,
) -> None: ):
"""更新会话的对话. """更新会话的对话
Args: Args:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
conversation_id (str): 对话 ID, 是 uuid 格式的字符串 conversation_id (str): 对话 ID, 是 uuid 格式的字符串
history (List[Dict]): 对话历史记录, 是一个字典列表, 每个字典包含 role 和 content 字段 history (List[Dict]): 对话历史记录, 是一个字典列表, 每个字典包含 role 和 content 字段
""" """
if not conversation_id: if not conversation_id:
# 如果没有提供 conversation_id,则获取当前的 # 如果没有提供 conversation_id,则获取当前的
@@ -277,20 +215,16 @@ class ConversationManager:
) )
async def update_conversation_title( async def update_conversation_title(
self, self, unified_msg_origin: str, title: str, conversation_id: str | None = None
unified_msg_origin: str, ):
title: str, """更新会话的对话标题
conversation_id: str | None = None,
) -> None:
"""更新会话的对话标题.
Args: Args:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
title (str): 对话标题 title (str): 对话标题
conversation_id (str): 对话 ID, 是 uuid 格式的字符串
Deprecated: Deprecated:
Use `update_conversation` with `title` parameter instead. Use `update_conversation` with `title` parameter instead.
""" """
await self.update_conversation( await self.update_conversation(
unified_msg_origin=unified_msg_origin, unified_msg_origin=unified_msg_origin,
@@ -303,16 +237,15 @@ class ConversationManager:
unified_msg_origin: str, unified_msg_origin: str,
persona_id: str, persona_id: str,
conversation_id: str | None = None, conversation_id: str | None = None,
) -> None: ):
"""更新会话的对话 Persona ID. """更新会话的对话 Persona ID
Args: Args:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
persona_id (str): 对话 Persona ID persona_id (str): 对话 Persona ID
conversation_id (str): 对话 ID, 是 uuid 格式的字符串
Deprecated: Deprecated:
Use `update_conversation` with `persona_id` parameter instead. Use `update_conversation` with `persona_id` parameter instead.
""" """
await self.update_conversation( await self.update_conversation(
unified_msg_origin=unified_msg_origin, unified_msg_origin=unified_msg_origin,
@@ -320,85 +253,40 @@ class ConversationManager:
persona_id=persona_id, persona_id=persona_id,
) )
async def add_message_pair(
self,
cid: str,
user_message: UserMessageSegment | dict,
assistant_message: AssistantMessageSegment | dict,
) -> None:
"""Add a user-assistant message pair to the conversation history.
Args:
cid (str): Conversation ID
user_message (UserMessageSegment | dict): OpenAI-format user message object or dict
assistant_message (AssistantMessageSegment | dict): OpenAI-format assistant message object or dict
Raises:
Exception: If the conversation with the given ID is not found
"""
conv = await self.db.get_conversation_by_id(cid=cid)
if not conv:
raise Exception(f"Conversation with id {cid} not found")
history = conv.content or []
if isinstance(user_message, UserMessageSegment):
user_msg_dict = user_message.model_dump()
else:
user_msg_dict = user_message
if isinstance(assistant_message, AssistantMessageSegment):
assistant_msg_dict = assistant_message.model_dump()
else:
assistant_msg_dict = assistant_message
history.append(user_msg_dict)
history.append(assistant_msg_dict)
await self.db.update_conversation(
cid=cid,
content=history,
)
async def get_human_readable_context( async def get_human_readable_context(
self, self, unified_msg_origin, conversation_id, page=1, page_size=10
unified_msg_origin: str, ):
conversation_id: str, """获取人类可读的上下文
page: int = 1,
page_size: int = 10,
) -> tuple[list[str], int]:
"""获取人类可读的上下文.
Args: Args:
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
conversation_id (str): 对话 ID, 是 uuid 格式的字符串 conversation_id (str): 对话 ID, 是 uuid 格式的字符串
page (int): 页码 page (int): 页码
page_size (int): 每页大小 page_size (int): 每页大小
""" """
conversation = await self.get_conversation(unified_msg_origin, conversation_id) conversation = await self.get_conversation(unified_msg_origin, conversation_id)
if not conversation:
return [], 0
history = json.loads(conversation.history) history = json.loads(conversation.history)
# contexts_groups 存放按顺序的段落(每个段落是一个 str 列表), contexts = []
# 之后会被展平成一个扁平的 str 列表返回。 temp_contexts = []
contexts_groups: list[list[str]] = []
temp_contexts: list[str] = []
for record in history: for record in history:
if record["role"] == "user": if record["role"] == "user":
temp_contexts.append(f"User: {record['content']}") temp_contexts.append(f"User: {record['content']}")
elif record["role"] == "assistant": elif record["role"] == "assistant":
if record.get("content"): if "content" in record and record["content"]:
temp_contexts.append(f"Assistant: {record['content']}") temp_contexts.append(f"Assistant: {record['content']}")
elif "tool_calls" in record: elif "tool_calls" in record:
tool_calls_str = json.dumps( tool_calls_str = json.dumps(
record["tool_calls"], record["tool_calls"], ensure_ascii=False
ensure_ascii=False,
) )
temp_contexts.append(f"Assistant: [函数调用] {tool_calls_str}") temp_contexts.append(f"Assistant: [函数调用] {tool_calls_str}")
else: else:
temp_contexts.append("Assistant: [未知的内容]") temp_contexts.append("Assistant: [未知的内容]")
contexts_groups.insert(0, temp_contexts) contexts.insert(0, temp_contexts)
temp_contexts = [] temp_contexts = []
# 展平分组后的 contexts 列表为单层字符串列表 # 展平 contexts 列表
contexts = [item for sublist in contexts_groups for item in sublist] contexts = [item for sublist in contexts for item in sublist]
# 计算分页 # 计算分页
paged_contexts = contexts[(page - 1) * page_size : page * page_size] paged_contexts = contexts[(page - 1) * page_size : page * page_size]
+57 -96
View File
@@ -1,5 +1,5 @@
"""Astrbot 核心生命周期管理类, 负责管理 AstrBot 的启动、停止、重启等操作. """
Astrbot 核心生命周期管理类, 负责管理 AstrBot 的启动、停止、重启等操作。
该类负责初始化各个组件, 包括 ProviderManager、PlatformManager、ConversationManager、PluginManager、PipelineScheduler、EventBus等。 该类负责初始化各个组件, 包括 ProviderManager、PlatformManager、ConversationManager、PluginManager、PipelineScheduler、EventBus等。
该类还负责加载和执行插件, 以及处理事件总线的分发。 该类还负责加载和执行插件, 以及处理事件总线的分发。
@@ -9,45 +9,42 @@
3. 执行启动完成事件钩子 3. 执行启动完成事件钩子
""" """
import asyncio
import os
import threading
import time
import traceback import traceback
import asyncio
import time
import threading
import os
from .event_bus import EventBus
from . import astrbot_config, html_renderer
from asyncio import Queue from asyncio import Queue
from typing import List
from astrbot.api import logger, sp from astrbot.core.pipeline.scheduler import PipelineScheduler, PipelineContext
from astrbot.core.star import PluginManager
from astrbot.core.platform.manager import PlatformManager
from astrbot.core.star.context import Context
from astrbot.core.persona_mgr import PersonaManager
from astrbot.core.provider.manager import ProviderManager
from astrbot.core import LogBroker from astrbot.core import LogBroker
from astrbot.core.astrbot_config_mgr import AstrBotConfigManager from astrbot.core.db import BaseDatabase
from astrbot.core.updator import AstrBotUpdator
from astrbot.core import logger, sp
from astrbot.core.config.default import VERSION from astrbot.core.config.default import VERSION
from astrbot.core.conversation_mgr import ConversationManager from astrbot.core.conversation_mgr import ConversationManager
from astrbot.core.db import BaseDatabase
from astrbot.core.knowledge_base.kb_mgr import KnowledgeBaseManager
from astrbot.core.persona_mgr import PersonaManager
from astrbot.core.pipeline.scheduler import PipelineContext, PipelineScheduler
from astrbot.core.platform.manager import PlatformManager
from astrbot.core.platform_message_history_mgr import PlatformMessageHistoryManager from astrbot.core.platform_message_history_mgr import PlatformMessageHistoryManager
from astrbot.core.provider.manager import ProviderManager from astrbot.core.astrbot_config_mgr import AstrBotConfigManager
from astrbot.core.star import PluginManager from astrbot.core.star.star_handler import star_handlers_registry, EventType
from astrbot.core.star.context import Context from astrbot.core.star.star_handler import star_map
from astrbot.core.star.star_handler import EventType, star_handlers_registry, star_map
from astrbot.core.umop_config_router import UmopConfigRouter
from astrbot.core.updator import AstrBotUpdator
from astrbot.core.utils.migra_helper import migra
from . import astrbot_config, html_renderer
from .event_bus import EventBus
class AstrBotCoreLifecycle: class AstrBotCoreLifecycle:
"""AstrBot 核心生命周期管理类, 负责管理 AstrBot 的启动、停止、重启等操作. """
AstrBot 核心生命周期管理类, 负责管理 AstrBot 的启动、停止、重启等操作。
该类负责初始化各个组件, 包括 ProviderManager、PlatformManager、ConversationManager、PluginManager、PipelineScheduler、 该类负责初始化各个组件, 包括 ProviderManager、PlatformManager、ConversationManager、PluginManager、PipelineScheduler、
EventBus 等。 EventBus 等。
该类还负责加载和执行插件, 以及处理事件总线的分发。 该类还负责加载和执行插件, 以及处理事件总线的分发。
""" """
def __init__(self, log_broker: LogBroker, db: BaseDatabase) -> None: def __init__(self, log_broker: LogBroker, db: BaseDatabase):
self.log_broker = log_broker # 初始化日志代理 self.log_broker = log_broker # 初始化日志代理
self.astrbot_config = astrbot_config # 初始化配置 self.astrbot_config = astrbot_config # 初始化配置
self.db = db # 初始化数据库 self.db = db # 初始化数据库
@@ -71,11 +68,11 @@ class AstrBotCoreLifecycle:
del os.environ["no_proxy"] del os.environ["no_proxy"]
logger.debug("HTTP proxy cleared") logger.debug("HTTP proxy cleared")
async def initialize(self) -> None: async def initialize(self):
"""初始化 AstrBot 核心生命周期管理类.
负责初始化各个组件, 包括 ProviderManager、PlatformManager、ConversationManager、PluginManager、PipelineScheduler、EventBus、AstrBotUpdator等。
""" """
初始化 AstrBot 核心生命周期管理类, 负责初始化各个组件, 包括 ProviderManager、PlatformManager、ConversationManager、PluginManager、PipelineScheduler、EventBus、AstrBotUpdator等。
"""
# 初始化日志代理 # 初始化日志代理
logger.info("AstrBot v" + VERSION) logger.info("AstrBot v" + VERSION)
if os.environ.get("TESTING", ""): if os.environ.get("TESTING", ""):
@@ -87,28 +84,11 @@ class AstrBotCoreLifecycle:
await html_renderer.initialize() await html_renderer.initialize()
# 初始化 UMOP 配置路由器
self.umop_config_router = UmopConfigRouter(sp=sp)
# 初始化 AstrBot 配置管理器 # 初始化 AstrBot 配置管理器
self.astrbot_config_mgr = AstrBotConfigManager( self.astrbot_config_mgr = AstrBotConfigManager(
default_config=self.astrbot_config, default_config=self.astrbot_config, sp=sp
ucr=self.umop_config_router,
sp=sp,
) )
# apply migration
try:
await migra(
self.db,
self.astrbot_config_mgr,
self.umop_config_router,
self.astrbot_config_mgr,
)
except Exception as e:
logger.error(f"AstrBot migration failed: {e!s}")
logger.error(traceback.format_exc())
# 初始化事件队列 # 初始化事件队列
self.event_queue = Queue() self.event_queue = Queue()
@@ -118,9 +98,7 @@ class AstrBotCoreLifecycle:
# 初始化供应商管理器 # 初始化供应商管理器
self.provider_manager = ProviderManager( self.provider_manager = ProviderManager(
self.astrbot_config_mgr, self.astrbot_config_mgr, self.db, self.persona_mgr
self.db,
self.persona_mgr,
) )
# 初始化平台管理器 # 初始化平台管理器
@@ -132,9 +110,6 @@ class AstrBotCoreLifecycle:
# 初始化平台消息历史管理器 # 初始化平台消息历史管理器
self.platform_message_history_manager = PlatformMessageHistoryManager(self.db) self.platform_message_history_manager = PlatformMessageHistoryManager(self.db)
# 初始化知识库管理器
self.kb_manager = KnowledgeBaseManager(self.provider_manager)
# 初始化提供给插件的上下文 # 初始化提供给插件的上下文
self.star_context = Context( self.star_context = Context(
self.event_queue, self.event_queue,
@@ -146,7 +121,6 @@ class AstrBotCoreLifecycle:
self.platform_message_history_manager, self.platform_message_history_manager,
self.persona_mgr, self.persona_mgr,
self.astrbot_config_mgr, self.astrbot_config_mgr,
self.kb_manager,
) )
# 初始化插件管理器 # 初始化插件管理器
@@ -158,9 +132,8 @@ class AstrBotCoreLifecycle:
# 根据配置实例化各个 Provider # 根据配置实例化各个 Provider
await self.provider_manager.initialize() await self.provider_manager.initialize()
await self.kb_manager.initialize()
# 初始化消息事件流水线调度器 # 初始化消息事件流水线调度器
self.pipeline_scheduler_mapping = await self.load_pipeline_scheduler() self.pipeline_scheduler_mapping = await self.load_pipeline_scheduler()
# 初始化更新器 # 初始化更新器
@@ -168,16 +141,14 @@ class AstrBotCoreLifecycle:
# 初始化事件总线 # 初始化事件总线
self.event_bus = EventBus( self.event_bus = EventBus(
self.event_queue, self.event_queue, self.pipeline_scheduler_mapping, self.astrbot_config_mgr
self.pipeline_scheduler_mapping,
self.astrbot_config_mgr,
) )
# 记录启动时间 # 记录启动时间
self.start_time = int(time.time()) self.start_time = int(time.time())
# 初始化当前任务列表 # 初始化当前任务列表
self.curr_tasks: list[asyncio.Task] = [] self.curr_tasks: List[asyncio.Task] = []
# 根据配置实例化各个平台适配器 # 根据配置实例化各个平台适配器
await self.platform_manager.initialize() await self.platform_manager.initialize()
@@ -185,34 +156,33 @@ class AstrBotCoreLifecycle:
# 初始化关闭控制面板的事件 # 初始化关闭控制面板的事件
self.dashboard_shutdown_event = asyncio.Event() self.dashboard_shutdown_event = asyncio.Event()
def _load(self) -> None: def _load(self):
"""加载事件总线和任务并初始化.""" """加载事件总线和任务并初始化"""
# 创建一个异步任务来执行事件总线的 dispatch() 方法 # 创建一个异步任务来执行事件总线的 dispatch() 方法
# dispatch是一个无限循环的协程, 从事件队列中获取事件并处理 # dispatch是一个无限循环的协程, 从事件队列中获取事件并处理
event_bus_task = asyncio.create_task( event_bus_task = asyncio.create_task(
self.event_bus.dispatch(), self.event_bus.dispatch(), name="event_bus"
name="event_bus",
) )
# 把插件中注册的所有协程函数注册到事件总线中并执行 # 把插件中注册的所有协程函数注册到事件总线中并执行
extra_tasks = [] extra_tasks = []
for task in self.star_context._register_tasks: for task in self.star_context._register_tasks:
extra_tasks.append(asyncio.create_task(task, name=task.__name__)) # type: ignore extra_tasks.append(asyncio.create_task(task, name=task.__name__))
tasks_ = [event_bus_task, *extra_tasks] tasks_ = [event_bus_task, *extra_tasks]
for task in tasks_: for task in tasks_:
self.curr_tasks.append( self.curr_tasks.append(
asyncio.create_task(self._task_wrapper(task), name=task.get_name()), asyncio.create_task(self._task_wrapper(task), name=task.get_name())
) )
self.start_time = int(time.time()) self.start_time = int(time.time())
async def _task_wrapper(self, task: asyncio.Task) -> None: async def _task_wrapper(self, task: asyncio.Task):
"""异步任务包装器, 用于处理异步任务执行中出现的各种异常. """异步任务包装器, 用于处理异步任务执行中出现的各种异常
Args: Args:
task (asyncio.Task): 要执行的异步任务 task (asyncio.Task): 要执行的异步任务
""" """
try: try:
await task await task
@@ -225,22 +195,19 @@ class AstrBotCoreLifecycle:
logger.error(f"| {line}") logger.error(f"| {line}")
logger.error("-------") logger.error("-------")
async def start(self) -> None: async def start(self):
"""启动 AstrBot 核心生命周期管理类. """启动 AstrBot 核心生命周期管理类, 用load加载事件总线和任务并初始化, 执行启动完成事件钩子"""
用load加载事件总线和任务并初始化, 执行启动完成事件钩子
"""
self._load() self._load()
logger.info("AstrBot 启动完成。") logger.info("AstrBot 启动完成。")
# 执行启动完成事件钩子 # 执行启动完成事件钩子
handlers = star_handlers_registry.get_handlers_by_event_type( handlers = star_handlers_registry.get_handlers_by_event_type(
EventType.OnAstrBotLoadedEvent, EventType.OnAstrBotLoadedEvent
) )
for handler in handlers: for handler in handlers:
try: try:
logger.info( logger.info(
f"hook(on_astrbot_loaded) -> {star_map[handler.handler_module_path].name} - {handler.handler_name}", f"hook(on_astrbot_loaded) -> {star_map[handler.handler_module_path].name} - {handler.handler_name}"
) )
await handler.handler() await handler.handler()
except BaseException: except BaseException:
@@ -249,8 +216,8 @@ class AstrBotCoreLifecycle:
# 同时运行curr_tasks中的所有任务 # 同时运行curr_tasks中的所有任务
await asyncio.gather(*self.curr_tasks, return_exceptions=True) await asyncio.gather(*self.curr_tasks, return_exceptions=True)
async def stop(self) -> None: async def stop(self):
"""停止 AstrBot 核心生命周期管理类, 取消所有当前任务并终止各个管理器.""" """停止 AstrBot 核心生命周期管理类, 取消所有当前任务并终止各个管理器"""
# 请求停止所有正在运行的异步任务 # 请求停止所有正在运行的异步任务
for task in self.curr_tasks: for task in self.curr_tasks:
task.cancel() task.cancel()
@@ -261,12 +228,11 @@ class AstrBotCoreLifecycle:
except Exception as e: except Exception as e:
logger.warning(traceback.format_exc()) logger.warning(traceback.format_exc())
logger.warning( logger.warning(
f"插件 {plugin.name} 未被正常终止 {e!s}, 可能会导致资源泄露等问题。", f"插件 {plugin.name} 未被正常终止 {e!s}, 可能会导致资源泄露等问题。"
) )
await self.provider_manager.terminate() await self.provider_manager.terminate()
await self.platform_manager.terminate() await self.platform_manager.terminate()
await self.kb_manager.terminate()
self.dashboard_shutdown_event.set() self.dashboard_shutdown_event.set()
# 再次遍历curr_tasks等待每个任务真正结束 # 再次遍历curr_tasks等待每个任务真正结束
@@ -278,19 +244,16 @@ class AstrBotCoreLifecycle:
except Exception as e: except Exception as e:
logger.error(f"任务 {task.get_name()} 发生错误: {e}") logger.error(f"任务 {task.get_name()} 发生错误: {e}")
async def restart(self) -> None: async def restart(self):
"""重启 AstrBot 核心生命周期管理类, 终止各个管理器并重新加载平台实例""" """重启 AstrBot 核心生命周期管理类, 终止各个管理器并重新加载平台实例"""
await self.provider_manager.terminate() await self.provider_manager.terminate()
await self.platform_manager.terminate() await self.platform_manager.terminate()
await self.kb_manager.terminate()
self.dashboard_shutdown_event.set() self.dashboard_shutdown_event.set()
threading.Thread( threading.Thread(
target=self.astrbot_updator._reboot, target=self.astrbot_updator._reboot, name="restart", daemon=True
name="restart",
daemon=True,
).start() ).start()
def load_platform(self) -> list[asyncio.Task]: def load_platform(self) -> List[asyncio.Task]:
"""加载平台实例并返回所有平台实例的异步任务列表""" """加载平台实例并返回所有平台实例的异步任务列表"""
tasks = [] tasks = []
platform_insts = self.platform_manager.get_insts() platform_insts = self.platform_manager.get_insts()
@@ -299,38 +262,36 @@ class AstrBotCoreLifecycle:
asyncio.create_task( asyncio.create_task(
platform_inst.run(), platform_inst.run(),
name=f"{platform_inst.meta().id}({platform_inst.meta().name})", name=f"{platform_inst.meta().id}({platform_inst.meta().name})",
), )
) )
return tasks return tasks
async def load_pipeline_scheduler(self) -> dict[str, PipelineScheduler]: async def load_pipeline_scheduler(self) -> dict[str, PipelineScheduler]:
"""加载消息事件流水线调度器. """加载消息事件流水线调度器
Returns: Returns:
dict[str, PipelineScheduler]: 平台 ID 到流水线调度器的映射 dict[str, PipelineScheduler]: 平台 ID 到流水线调度器的映射
""" """
mapping = {} mapping = {}
for conf_id, ab_config in self.astrbot_config_mgr.confs.items(): for conf_id, ab_config in self.astrbot_config_mgr.confs.items():
scheduler = PipelineScheduler( scheduler = PipelineScheduler(
PipelineContext(ab_config, self.plugin_manager, conf_id), PipelineContext(ab_config, self.plugin_manager, conf_id)
) )
await scheduler.initialize() await scheduler.initialize()
mapping[conf_id] = scheduler mapping[conf_id] = scheduler
return mapping return mapping
async def reload_pipeline_scheduler(self, conf_id: str) -> None: async def reload_pipeline_scheduler(self, conf_id: str):
"""重新加载消息事件流水线调度器. """重新加载消息事件流水线调度器
Returns: Returns:
dict[str, PipelineScheduler]: 平台 ID 到流水线调度器的映射 dict[str, PipelineScheduler]: 平台 ID 到流水线调度器的映射
""" """
ab_config = self.astrbot_config_mgr.confs.get(conf_id) ab_config = self.astrbot_config_mgr.confs.get(conf_id)
if not ab_config: if not ab_config:
raise ValueError(f"配置文件 {conf_id} 不存在") raise ValueError(f"配置文件 {conf_id} 不存在")
scheduler = PipelineScheduler( scheduler = PipelineScheduler(
PipelineContext(ab_config, self.plugin_manager, conf_id), PipelineContext(ab_config, self.plugin_manager, conf_id)
) )
await scheduler.initialize() await scheduler.initialize()
self.pipeline_scheduler_mapping[conf_id] = scheduler self.pipeline_scheduler_mapping[conf_id] = scheduler
+24 -132
View File
@@ -1,27 +1,27 @@
import abc import abc
import datetime import datetime
import typing as T import typing as T
from contextlib import asynccontextmanager
from dataclasses import dataclass
from deprecated import deprecated from deprecated import deprecated
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine from dataclasses import dataclass
from astrbot.core.db.po import ( from astrbot.core.db.po import (
Attachment,
ConversationV2,
Persona,
PlatformMessageHistory,
PlatformSession,
PlatformStat,
Preference,
Stats, Stats,
PlatformStat,
ConversationV2,
PlatformMessageHistory,
Attachment,
Persona,
Preference,
) )
from contextlib import asynccontextmanager
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.orm import sessionmaker
@dataclass @dataclass
class BaseDatabase(abc.ABC): class BaseDatabase(abc.ABC):
"""数据库基类""" """
数据库基类
"""
DATABASE_URL = "" DATABASE_URL = ""
@@ -31,14 +31,13 @@ class BaseDatabase(abc.ABC):
echo=False, echo=False,
future=True, future=True,
) )
self.AsyncSessionLocal = async_sessionmaker( self.AsyncSessionLocal = sessionmaker(
self.engine, self.engine, class_=AsyncSession, expire_on_commit=False
class_=AsyncSession,
expire_on_commit=False,
) )
async def initialize(self): async def initialize(self):
"""初始化数据库连接""" """初始化数据库连接"""
pass
@asynccontextmanager @asynccontextmanager
async def get_db(self) -> T.AsyncGenerator[AsyncSession, None]: async def get_db(self) -> T.AsyncGenerator[AsyncSession, None]:
@@ -92,9 +91,7 @@ class BaseDatabase(abc.ABC):
@abc.abstractmethod @abc.abstractmethod
async def get_conversations( async def get_conversations(
self, self, user_id: str | None = None, platform_id: str | None = None
user_id: str | None = None,
platform_id: str | None = None,
) -> list[ConversationV2]: ) -> list[ConversationV2]:
"""Get all conversations for a specific user and platform_id(optional). """Get all conversations for a specific user and platform_id(optional).
@@ -109,9 +106,7 @@ class BaseDatabase(abc.ABC):
@abc.abstractmethod @abc.abstractmethod
async def get_all_conversations( async def get_all_conversations(
self, self, page: int = 1, page_size: int = 20
page: int = 1,
page_size: int = 20,
) -> list[ConversationV2]: ) -> list[ConversationV2]:
"""Get all conversations with pagination.""" """Get all conversations with pagination."""
... ...
@@ -159,31 +154,23 @@ class BaseDatabase(abc.ABC):
"""Delete a conversation by its ID.""" """Delete a conversation by its ID."""
... ...
@abc.abstractmethod
async def delete_conversations_by_user_id(self, user_id: str) -> None:
"""Delete all conversations for a specific user."""
...
@abc.abstractmethod @abc.abstractmethod
async def insert_platform_message_history( async def insert_platform_message_history(
self, self,
platform_id: str, platform_id: str,
user_id: str, user_id: str,
content: dict, content: list[dict],
sender_id: str | None = None, sender_id: str | None = None,
sender_name: str | None = None, sender_name: str | None = None,
) -> PlatformMessageHistory: ) -> None:
"""Insert a new platform message history record.""" """Insert a new platform message history record."""
... ...
@abc.abstractmethod @abc.abstractmethod
async def delete_platform_message_offset( async def delete_platform_message_offset(
self, self, platform_id: str, user_id: str, offset_sec: int = 86400
platform_id: str,
user_id: str,
offset_sec: int = 86400,
) -> None: ) -> None:
"""Delete platform message history records newer than the specified offset.""" """Delete platform message history records older than the specified offset."""
... ...
@abc.abstractmethod @abc.abstractmethod
@@ -197,14 +184,6 @@ class BaseDatabase(abc.ABC):
"""Get platform message history for a specific user.""" """Get platform message history for a specific user."""
... ...
@abc.abstractmethod
async def get_platform_message_history_by_id(
self,
message_id: int,
) -> PlatformMessageHistory | None:
"""Get a platform message history record by its ID."""
...
@abc.abstractmethod @abc.abstractmethod
async def insert_attachment( async def insert_attachment(
self, self,
@@ -220,27 +199,6 @@ class BaseDatabase(abc.ABC):
"""Get an attachment by its ID.""" """Get an attachment by its ID."""
... ...
@abc.abstractmethod
async def get_attachments(self, attachment_ids: list[str]) -> list[Attachment]:
"""Get multiple attachments by their IDs."""
...
@abc.abstractmethod
async def delete_attachment(self, attachment_id: str) -> bool:
"""Delete an attachment by its ID.
Returns True if the attachment was deleted, False if it was not found.
"""
...
@abc.abstractmethod
async def delete_attachments(self, attachment_ids: list[str]) -> int:
"""Delete multiple attachments by their IDs.
Returns the number of attachments deleted.
"""
...
@abc.abstractmethod @abc.abstractmethod
async def insert_persona( async def insert_persona(
self, self,
@@ -280,11 +238,7 @@ class BaseDatabase(abc.ABC):
@abc.abstractmethod @abc.abstractmethod
async def insert_preference_or_update( async def insert_preference_or_update(
self, self, scope: str, scope_id: str, key: str, value: dict
scope: str,
scope_id: str,
key: str,
value: dict,
) -> Preference: ) -> Preference:
"""Insert a new preference record.""" """Insert a new preference record."""
... ...
@@ -296,10 +250,7 @@ class BaseDatabase(abc.ABC):
@abc.abstractmethod @abc.abstractmethod
async def get_preferences( async def get_preferences(
self, self, scope: str, scope_id: str | None = None, key: str | None = None
scope: str,
scope_id: str | None = None,
key: str | None = None,
) -> list[Preference]: ) -> list[Preference]:
"""Get all preferences for a specific scope ID or key.""" """Get all preferences for a specific scope ID or key."""
... ...
@@ -331,62 +282,3 @@ class BaseDatabase(abc.ABC):
# async def get_llm_messages(self, cid: str) -> list[LLMMessage]: # async def get_llm_messages(self, cid: str) -> list[LLMMessage]:
# """Get all LLM messages for a specific conversation.""" # """Get all LLM messages for a specific conversation."""
# ... # ...
@abc.abstractmethod
async def get_session_conversations(
self,
page: int = 1,
page_size: int = 20,
search_query: str | None = None,
platform: str | None = None,
) -> tuple[list[dict], int]:
"""Get paginated session conversations with joined conversation and persona details, support search and platform filter."""
...
# ====
# Platform Session Management
# ====
@abc.abstractmethod
async def create_platform_session(
self,
creator: str,
platform_id: str = "webchat",
session_id: str | None = None,
display_name: str | None = None,
is_group: int = 0,
) -> PlatformSession:
"""Create a new Platform session."""
...
@abc.abstractmethod
async def get_platform_session_by_id(
self, session_id: str
) -> PlatformSession | None:
"""Get a Platform session by its ID."""
...
@abc.abstractmethod
async def get_platform_sessions_by_creator(
self,
creator: str,
platform_id: str | None = None,
page: int = 1,
page_size: int = 20,
) -> list[PlatformSession]:
"""Get all Platform sessions for a specific creator (username) and optionally platform."""
...
@abc.abstractmethod
async def update_platform_session(
self,
session_id: str,
display_name: str | None = None,
) -> None:
"""Update a Platform session's updated_at timestamp and optionally display_name."""
...
@abc.abstractmethod
async def delete_platform_session(self, session_id: str) -> None:
"""Delete a Platform session by its ID."""
...
+14 -19
View File
@@ -1,33 +1,27 @@
import os import os
from astrbot.api import logger, sp
from astrbot.core.config import AstrBotConfig
from astrbot.core.db import BaseDatabase
from astrbot.core.utils.astrbot_path import get_astrbot_data_path from astrbot.core.utils.astrbot_path import get_astrbot_data_path
from astrbot.core.db import BaseDatabase
from astrbot.core.config import AstrBotConfig
from astrbot.api import logger, sp
from .migra_3_to_4 import ( from .migra_3_to_4 import (
migration_conversation_table, migration_conversation_table,
migration_persona_data,
migration_platform_table, migration_platform_table,
migration_preferences,
migration_webchat_data, migration_webchat_data,
migration_persona_data,
migration_preferences,
) )
async def check_migration_needed_v4(db_helper: BaseDatabase) -> bool: async def check_migration_needed_v4(db_helper: BaseDatabase) -> bool:
"""检查是否需要进行数据库迁移 """
检查是否需要进行数据库迁移
如果存在 data_v3.db 并且 preference 中没有 migration_done_v4,则需要进行迁移。 如果存在 data_v3.db 并且 preference 中没有 migration_done_v4,则需要进行迁移。
""" """
# 仅当 data 目录下存在旧版本数据(data_v3.db 文件)时才考虑迁移 data_v3_exists = os.path.exists(get_astrbot_data_path())
data_dir = get_astrbot_data_path() if not data_v3_exists:
data_v3_db = os.path.join(data_dir, "data_v3.db")
if not os.path.exists(data_v3_db):
return False return False
migration_done = await db_helper.get_preference( migration_done = await db_helper.get_preference(
"global", "global", "global", "migration_done_v4"
"global",
"migration_done_v4",
) )
if migration_done: if migration_done:
return False return False
@@ -38,8 +32,9 @@ async def do_migration_v4(
db_helper: BaseDatabase, db_helper: BaseDatabase,
platform_id_map: dict[str, dict[str, str]], platform_id_map: dict[str, dict[str, str]],
astrbot_config: AstrBotConfig, astrbot_config: AstrBotConfig,
) -> None: ):
"""执行数据库迁移 """
执行数据库迁移
迁移旧的 webchat_conversation 表到新的 conversation 表。 迁移旧的 webchat_conversation 表到新的 conversation 表。
迁移旧的 platform 到新的 platform_stats 表。 迁移旧的 platform 到新的 platform_stats 表。
""" """
@@ -58,7 +53,7 @@ async def do_migration_v4(
await migration_webchat_data(db_helper, platform_id_map) await migration_webchat_data(db_helper, platform_id_map)
# 执行偏好设置迁移 # 执行偏好设置迁移
await migration_preferences(db_helper, platform_id_map) await migration_preferences(db_helper,platform_id_map)
# 执行平台统计表迁移 # 执行平台统计表迁移
await migration_platform_table(db_helper, platform_id_map) await migration_platform_table(db_helper, platform_id_map)
+34 -55
View File
@@ -1,18 +1,15 @@
import datetime
import json import json
import datetime
from sqlalchemy import text from .. import BaseDatabase
from sqlalchemy.ext.asyncio import AsyncSession from .sqlite_v3 import SQLiteDatabase as SQLiteV3DatabaseV3
from .shared_preferences_v3 import sp as sp_v3
from astrbot.core.config.default import DB_PATH
from astrbot.api import logger, sp from astrbot.api import logger, sp
from astrbot.core.config import AstrBotConfig from astrbot.core.config import AstrBotConfig
from astrbot.core.config.default import DB_PATH
from astrbot.core.db.po import ConversationV2, PlatformMessageHistory
from astrbot.core.platform.astr_message_event import MessageSesion from astrbot.core.platform.astr_message_event import MessageSesion
from sqlalchemy.ext.asyncio import AsyncSession
from .. import BaseDatabase from astrbot.core.db.po import ConversationV2, PlatformMessageHistory
from .shared_preferences_v3 import sp as sp_v3 from sqlalchemy import text
from .sqlite_v3 import SQLiteDatabase as SQLiteV3DatabaseV3
""" """
1. 迁移旧的 webchat_conversation 表到新的 conversation 1. 迁移旧的 webchat_conversation 表到新的 conversation
@@ -21,8 +18,7 @@ from .sqlite_v3 import SQLiteDatabase as SQLiteV3DatabaseV3
def get_platform_id( def get_platform_id(
platform_id_map: dict[str, dict[str, str]], platform_id_map: dict[str, dict[str, str]], old_platform_name: str
old_platform_name: str,
) -> str: ) -> str:
return platform_id_map.get( return platform_id_map.get(
old_platform_name, old_platform_name,
@@ -31,8 +27,7 @@ def get_platform_id(
def get_platform_type( def get_platform_type(
platform_id_map: dict[str, dict[str, str]], platform_id_map: dict[str, dict[str, str]], old_platform_name: str
old_platform_name: str,
) -> str: ) -> str:
return platform_id_map.get( return platform_id_map.get(
old_platform_name, old_platform_name,
@@ -41,15 +36,13 @@ def get_platform_type(
async def migration_conversation_table( async def migration_conversation_table(
db_helper: BaseDatabase, db_helper: BaseDatabase, platform_id_map: dict[str, dict[str, str]]
platform_id_map: dict[str, dict[str, str]],
): ):
db_helper_v3 = SQLiteV3DatabaseV3( db_helper_v3 = SQLiteV3DatabaseV3(
db_path=DB_PATH.replace("data_v4.db", "data_v3.db"), db_path=DB_PATH.replace("data_v4.db", "data_v3.db")
) )
conversations, total_cnt = db_helper_v3.get_all_conversations( conversations, total_cnt = db_helper_v3.get_all_conversations(
page=1, page=1, page_size=10000000
page_size=10000000,
) )
logger.info(f"迁移 {total_cnt} 条旧的会话数据到新的表中...") logger.info(f"迁移 {total_cnt} 条旧的会话数据到新的表中...")
@@ -68,15 +61,13 @@ async def migration_conversation_table(
) )
if not conv: if not conv:
logger.info( logger.info(
f"未找到该条旧会话对应的具体数据: {conversation}, 跳过。", f"未找到该条旧会话对应的具体数据: {conversation}, 跳过。"
) )
continue
if ":" not in conv.user_id: if ":" not in conv.user_id:
continue continue
session = MessageSesion.from_str(session_str=conv.user_id) session = MessageSesion.from_str(session_str=conv.user_id)
platform_id = get_platform_id( platform_id = get_platform_id(
platform_id_map, platform_id_map, session.platform_name
session.platform_name,
) )
session.platform_id = platform_id # 更新平台名称为新的 ID session.platform_id = platform_id # 更新平台名称为新的 ID
conv_v2 = ConversationV2( conv_v2 = ConversationV2(
@@ -99,11 +90,10 @@ async def migration_conversation_table(
async def migration_platform_table( async def migration_platform_table(
db_helper: BaseDatabase, db_helper: BaseDatabase, platform_id_map: dict[str, dict[str, str]]
platform_id_map: dict[str, dict[str, str]],
): ):
db_helper_v3 = SQLiteV3DatabaseV3( db_helper_v3 = SQLiteV3DatabaseV3(
db_path=DB_PATH.replace("data_v4.db", "data_v3.db"), db_path=DB_PATH.replace("data_v4.db", "data_v3.db")
) )
secs_from_2023_4_10_to_now = ( secs_from_2023_4_10_to_now = (
datetime.datetime.now(datetime.timezone.utc) datetime.datetime.now(datetime.timezone.utc)
@@ -144,12 +134,10 @@ async def migration_platform_table(
if cnt == 0: if cnt == 0:
continue continue
platform_id = get_platform_id( platform_id = get_platform_id(
platform_id_map, platform_id_map, platform_stats_v3[idx].name
platform_stats_v3[idx].name,
) )
platform_type = get_platform_type( platform_type = get_platform_type(
platform_id_map, platform_id_map, platform_stats_v3[idx].name
platform_stats_v3[idx].name,
) )
try: try:
await dbsession.execute( await dbsession.execute(
@@ -161,8 +149,7 @@ async def migration_platform_table(
"""), """),
{ {
"timestamp": datetime.datetime.fromtimestamp( "timestamp": datetime.datetime.fromtimestamp(
bucket_end, bucket_end, tz=datetime.timezone.utc
tz=datetime.timezone.utc,
), ),
"platform_id": platform_id, "platform_id": platform_id,
"platform_type": platform_type, "platform_type": platform_type,
@@ -178,16 +165,14 @@ async def migration_platform_table(
async def migration_webchat_data( async def migration_webchat_data(
db_helper: BaseDatabase, db_helper: BaseDatabase, platform_id_map: dict[str, dict[str, str]]
platform_id_map: dict[str, dict[str, str]],
): ):
"""迁移 WebChat 的历史记录到新的 PlatformMessageHistory 表中""" """迁移 WebChat 的历史记录到新的 PlatformMessageHistory 表中"""
db_helper_v3 = SQLiteV3DatabaseV3( db_helper_v3 = SQLiteV3DatabaseV3(
db_path=DB_PATH.replace("data_v4.db", "data_v3.db"), db_path=DB_PATH.replace("data_v4.db", "data_v3.db")
) )
conversations, total_cnt = db_helper_v3.get_all_conversations( conversations, total_cnt = db_helper_v3.get_all_conversations(
page=1, page=1, page_size=10000000
page_size=10000000,
) )
logger.info(f"迁移 {total_cnt} 条旧的 WebChat 会话数据到新的表中...") logger.info(f"迁移 {total_cnt} 条旧的 WebChat 会话数据到新的表中...")
@@ -206,9 +191,8 @@ async def migration_webchat_data(
) )
if not conv: if not conv:
logger.info( logger.info(
f"未找到该条旧会话对应的具体数据: {conversation}, 跳过。", f"未找到该条旧会话对应的具体数据: {conversation}, 跳过。"
) )
continue
if ":" in conv.user_id: if ":" in conv.user_id:
continue continue
platform_id = "webchat" platform_id = "webchat"
@@ -234,10 +218,10 @@ async def migration_webchat_data(
async def migration_persona_data( async def migration_persona_data(
db_helper: BaseDatabase, db_helper: BaseDatabase, astrbot_config: AstrBotConfig
astrbot_config: AstrBotConfig,
): ):
"""迁移 Persona 数据到新的表中。 """
迁移 Persona 数据到新的表中
旧的 Persona 数据存储在 preference 新的 Persona 数据存储在 persona 表中 旧的 Persona 数据存储在 preference 新的 Persona 数据存储在 persona 表中
""" """
v3_persona_config: list[dict] = astrbot_config.get("persona", []) v3_persona_config: list[dict] = astrbot_config.get("persona", [])
@@ -252,15 +236,14 @@ async def migration_persona_data(
try: try:
begin_dialogs = persona.get("begin_dialogs", []) begin_dialogs = persona.get("begin_dialogs", [])
mood_imitation_dialogs = persona.get("mood_imitation_dialogs", []) mood_imitation_dialogs = persona.get("mood_imitation_dialogs", [])
parts = [] mood_prompt = ""
user_turn = True user_turn = True
for mood_dialog in mood_imitation_dialogs: for mood_dialog in mood_imitation_dialogs:
if user_turn: if user_turn:
parts.append(f"A: {mood_dialog}\n") mood_prompt += f"A: {mood_dialog}\n"
else: else:
parts.append(f"B: {mood_dialog}\n") mood_prompt += f"B: {mood_dialog}\n"
user_turn = not user_turn user_turn = not user_turn
mood_prompt = "".join(parts)
system_prompt = persona.get("prompt", "") system_prompt = persona.get("prompt", "")
if mood_prompt: if mood_prompt:
system_prompt += f"Here are few shots of dialogs, you need to imitate the tone of 'B' in the following dialogs to respond:\n {mood_prompt}" system_prompt += f"Here are few shots of dialogs, you need to imitate the tone of 'B' in the following dialogs to respond:\n {mood_prompt}"
@@ -270,15 +253,14 @@ async def migration_persona_data(
begin_dialogs=begin_dialogs, begin_dialogs=begin_dialogs,
) )
logger.info( logger.info(
f"迁移 Persona {persona['name']}({persona_new.system_prompt[:30]}...) 到新表成功。", f"迁移 Persona {persona['name']}({persona_new.system_prompt[:30]}...) 到新表成功。"
) )
except Exception as e: except Exception as e:
logger.error(f"解析 Persona 配置失败:{e}") logger.error(f"解析 Persona 配置失败:{e}")
async def migration_preferences( async def migration_preferences(
db_helper: BaseDatabase, db_helper: BaseDatabase, platform_id_map: dict[str, dict[str, str]]
platform_id_map: dict[str, dict[str, str]],
): ):
# 1. global scope migration # 1. global scope migration
keys = [ keys = [
@@ -347,13 +329,10 @@ async def migration_preferences(
for provider_type, provider_id in perf.items(): for provider_type, provider_id in perf.items():
await sp.put_async( await sp.put_async(
"umo", "umo", str(session), f"provider_perf_{provider_type}", provider_id
str(session),
f"provider_perf_{provider_type}",
provider_id,
) )
logger.info( logger.info(
f"迁移会话 {umo} 的提供商偏好到新表成功,平台 ID: {platform_id}", f"迁移会话 {umo} 的提供商偏好到新表成功,平台 ID: {platform_id}"
) )
except Exception as e: except Exception as e:
logger.error(f"迁移会话 {umo} 的提供商偏好失败: {e}", exc_info=True) logger.error(f"迁移会话 {umo} 的提供商偏好失败: {e}", exc_info=True)
@@ -1,44 +0,0 @@
from astrbot.api import logger, sp
from astrbot.core.astrbot_config_mgr import AstrBotConfigManager
from astrbot.core.umop_config_router import UmopConfigRouter
async def migrate_45_to_46(acm: AstrBotConfigManager, ucr: UmopConfigRouter):
abconf_data = acm.abconf_data
if not isinstance(abconf_data, dict):
# should be unreachable
logger.warning(
f"migrate_45_to_46: abconf_data is not a dict (type={type(abconf_data)}). Value: {abconf_data!r}",
)
return
# 如果任何一项带有 umop,则说明需要迁移
need_migration = False
for conf_id, conf_info in abconf_data.items():
if isinstance(conf_info, dict) and "umop" in conf_info:
need_migration = True
break
if not need_migration:
return
logger.info("Starting migration from version 4.5 to 4.6")
# extract umo->conf_id mapping
umo_to_conf_id = {}
for conf_id, conf_info in abconf_data.items():
if isinstance(conf_info, dict) and "umop" in conf_info:
umop_ls = conf_info.pop("umop")
if not isinstance(umop_ls, list):
continue
for umo in umop_ls:
if isinstance(umo, str) and umo not in umo_to_conf_id:
umo_to_conf_id[umo] = conf_id
# update the abconf data
await sp.global_put("abconf_mapping", abconf_data)
# update the umop config router
await ucr.update_routing_data(umo_to_conf_id)
logger.info("Migration from version 45 to 46 completed successfully")
@@ -1,131 +0,0 @@
"""Migration script for WebChat sessions.
This migration creates PlatformSession from existing platform_message_history records.
Changes:
- Creates platform_sessions table
- Adds platform_id field (default: 'webchat')
- Adds display_name field
- Session_id format: {platform_id}_{uuid}
"""
from sqlalchemy import func, select
from sqlmodel import col
from astrbot.api import logger, sp
from astrbot.core.db import BaseDatabase
from astrbot.core.db.po import ConversationV2, PlatformMessageHistory, PlatformSession
async def migrate_webchat_session(db_helper: BaseDatabase):
"""Create PlatformSession records from platform_message_history.
This migration extracts all unique user_ids from platform_message_history
where platform_id='webchat' and creates corresponding PlatformSession records.
"""
# 检查是否已经完成迁移
migration_done = await db_helper.get_preference(
"global", "global", "migration_done_webchat_session_1"
)
if migration_done:
return
logger.info("开始执行数据库迁移(WebChat 会话迁移)...")
try:
async with db_helper.get_db() as session:
# 从 platform_message_history 创建 PlatformSession
query = (
select(
col(PlatformMessageHistory.user_id),
col(PlatformMessageHistory.sender_name),
func.min(PlatformMessageHistory.created_at).label("earliest"),
func.max(PlatformMessageHistory.updated_at).label("latest"),
)
.where(col(PlatformMessageHistory.platform_id) == "webchat")
.where(col(PlatformMessageHistory.sender_id) != "bot")
.group_by(col(PlatformMessageHistory.user_id))
)
result = await session.execute(query)
webchat_users = result.all()
if not webchat_users:
logger.info("没有找到需要迁移的 WebChat 数据")
await sp.put_async(
"global", "global", "migration_done_webchat_session_1", True
)
return
logger.info(f"找到 {len(webchat_users)} 个 WebChat 会话需要迁移")
# 检查已存在的会话
existing_query = select(col(PlatformSession.session_id))
existing_result = await session.execute(existing_query)
existing_session_ids = {row[0] for row in existing_result.fetchall()}
# 查询 Conversations 表中的 title,用于设置 display_name
# 对于每个 user_id,对应的 conversation user_id 格式为: webchat:FriendMessage:webchat!astrbot!{user_id}
user_ids_to_query = [
f"webchat:FriendMessage:webchat!astrbot!{user_id}"
for user_id, _, _, _ in webchat_users
]
conv_query = select(
col(ConversationV2.user_id), col(ConversationV2.title)
).where(col(ConversationV2.user_id).in_(user_ids_to_query))
conv_result = await session.execute(conv_query)
# 创建 user_id -> title 的映射字典
title_map = {
user_id.replace("webchat:FriendMessage:webchat!astrbot!", ""): title
for user_id, title in conv_result.fetchall()
}
# 批量创建 PlatformSession 记录
sessions_to_add = []
skipped_count = 0
for user_id, sender_name, created_at, updated_at in webchat_users:
# user_id 就是 webchat_conv_id (session_id)
session_id = user_id
# sender_name 通常是 username,但可能为 None
creator = sender_name if sender_name else "guest"
# 检查是否已经存在该会话
if session_id in existing_session_ids:
logger.debug(f"会话 {session_id} 已存在,跳过")
skipped_count += 1
continue
# 从 Conversations 表中获取 display_name
display_name = title_map.get(user_id)
# 创建新的 PlatformSession(保留原有的时间戳)
new_session = PlatformSession(
session_id=session_id,
platform_id="webchat",
creator=creator,
is_group=0,
created_at=created_at,
updated_at=updated_at,
display_name=display_name,
)
sessions_to_add.append(new_session)
# 批量插入
if sessions_to_add:
session.add_all(sessions_to_add)
await session.commit()
logger.info(
f"WebChat 会话迁移完成!成功迁移: {len(sessions_to_add)}, 跳过: {skipped_count}",
)
else:
logger.info("没有新会话需要迁移")
# 标记迁移完成
await sp.put_async("global", "global", "migration_done_webchat_session_1", True)
except Exception as e:
logger.error(f"迁移过程中发生错误: {e}", exc_info=True)
raise
@@ -1,12 +1,10 @@
import json import json
import os import os
from typing import TypeVar from typing import TypeVar
from astrbot.core.utils.astrbot_path import get_astrbot_data_path from astrbot.core.utils.astrbot_path import get_astrbot_data_path
_VT = TypeVar("_VT") _VT = TypeVar("_VT")
class SharedPreferences: class SharedPreferences:
def __init__(self, path=None): def __init__(self, path=None):
if path is None: if path is None:
@@ -17,7 +15,7 @@ class SharedPreferences:
def _load_preferences(self): def _load_preferences(self):
if os.path.exists(self.path): if os.path.exists(self.path):
try: try:
with open(self.path) as f: with open(self.path, "r") as f:
return json.load(f) return json.load(f)
except json.JSONDecodeError: except json.JSONDecodeError:
os.remove(self.path) os.remove(self.path)
@@ -44,5 +42,4 @@ class SharedPreferences:
self._data.clear() self._data.clear()
self._save_preferences() self._save_preferences()
sp = SharedPreferences() sp = SharedPreferences()
+26 -32
View File
@@ -1,10 +1,8 @@
import sqlite3 import sqlite3
import time import time
from dataclasses import dataclass
from typing import Any
from astrbot.core.db.po import Platform, Stats from astrbot.core.db.po import Platform, Stats
from typing import Tuple, List, Dict, Any
from dataclasses import dataclass
@dataclass @dataclass
class Conversation: class Conversation:
@@ -78,7 +76,7 @@ PRAGMA encoding = 'UTF-8';
""" """
class SQLiteDatabase: class SQLiteDatabase():
def __init__(self, db_path: str) -> None: def __init__(self, db_path: str) -> None:
super().__init__() super().__init__()
self.db_path = db_path self.db_path = db_path
@@ -95,7 +93,7 @@ class SQLiteDatabase:
c.execute( c.execute(
""" """
PRAGMA table_info(webchat_conversation) PRAGMA table_info(webchat_conversation)
""", """
) )
res = c.fetchall() res = c.fetchall()
has_title = False has_title = False
@@ -109,14 +107,14 @@ class SQLiteDatabase:
c.execute( c.execute(
""" """
ALTER TABLE webchat_conversation ADD COLUMN title TEXT; ALTER TABLE webchat_conversation ADD COLUMN title TEXT;
""", """
) )
self.conn.commit() self.conn.commit()
if not has_persona_id: if not has_persona_id:
c.execute( c.execute(
""" """
ALTER TABLE webchat_conversation ADD COLUMN persona_id TEXT; ALTER TABLE webchat_conversation ADD COLUMN persona_id TEXT;
""", """
) )
self.conn.commit() self.conn.commit()
@@ -127,7 +125,7 @@ class SQLiteDatabase:
conn.text_factory = str conn.text_factory = str
return conn return conn
def _exec_sql(self, sql: str, params: tuple | None = None): def _exec_sql(self, sql: str, params: Tuple = None):
conn = self.conn conn = self.conn
try: try:
c = self.conn.cursor() c = self.conn.cursor()
@@ -175,7 +173,7 @@ class SQLiteDatabase:
""" """
SELECT * FROM platform SELECT * FROM platform
""" """
+ where_clause, + where_clause
) )
platform = [] platform = []
@@ -195,7 +193,7 @@ class SQLiteDatabase:
c.execute( c.execute(
""" """
SELECT SUM(count) FROM platform SELECT SUM(count) FROM platform
""", """
) )
res = c.fetchone() res = c.fetchone()
c.close() c.close()
@@ -215,7 +213,7 @@ class SQLiteDatabase:
SELECT name, SUM(count), timestamp FROM platform SELECT name, SUM(count), timestamp FROM platform
""" """
+ where_clause + where_clause
+ " GROUP BY name", + " GROUP BY name"
) )
platform = [] platform = []
@@ -224,11 +222,9 @@ class SQLiteDatabase:
c.close() c.close()
return Stats(platform) return Stats(platform, [], [])
def get_conversation_by_user_id( def get_conversation_by_user_id(self, user_id: str, cid: str) -> Conversation:
self, user_id: str, cid: str
) -> Conversation | None:
try: try:
c = self.conn.cursor() c = self.conn.cursor()
except sqlite3.ProgrammingError: except sqlite3.ProgrammingError:
@@ -245,7 +241,7 @@ class SQLiteDatabase:
c.close() c.close()
if not res: if not res:
return None return
return Conversation(*res) return Conversation(*res)
@@ -260,7 +256,7 @@ class SQLiteDatabase:
(user_id, cid, history, updated_at, created_at), (user_id, cid, history, updated_at, created_at),
) )
def get_conversations(self, user_id: str) -> list[Conversation]: def get_conversations(self, user_id: str) -> Tuple:
try: try:
c = self.conn.cursor() c = self.conn.cursor()
except sqlite3.ProgrammingError: except sqlite3.ProgrammingError:
@@ -283,7 +279,7 @@ class SQLiteDatabase:
title = row[3] title = row[3]
persona_id = row[4] persona_id = row[4]
conversations.append( conversations.append(
Conversation("", cid, "[]", created_at, updated_at, title, persona_id), Conversation("", cid, "[]", created_at, updated_at, title, persona_id)
) )
return conversations return conversations
@@ -322,10 +318,8 @@ class SQLiteDatabase:
) )
def get_all_conversations( def get_all_conversations(
self, self, page: int = 1, page_size: int = 20
page: int = 1, ) -> Tuple[List[Dict[str, Any]], int]:
page_size: int = 20,
) -> tuple[list[dict[str, Any]], int]:
"""获取所有对话,支持分页,按更新时间降序排序""" """获取所有对话,支持分页,按更新时间降序排序"""
try: try:
c = self.conn.cursor() c = self.conn.cursor()
@@ -371,7 +365,7 @@ class SQLiteDatabase:
"persona_id": persona_id or "", "persona_id": persona_id or "",
"created_at": created_at or 0, "created_at": created_at or 0,
"updated_at": updated_at or 0, "updated_at": updated_at or 0,
}, }
) )
return conversations, total_count return conversations, total_count
@@ -386,12 +380,12 @@ class SQLiteDatabase:
self, self,
page: int = 1, page: int = 1,
page_size: int = 20, page_size: int = 20,
platforms: list[str] | None = None, platforms: List[str] = None,
message_types: list[str] | None = None, message_types: List[str] = None,
search_query: str | None = None, search_query: str = None,
exclude_ids: list[str] | None = None, exclude_ids: List[str] = None,
exclude_platforms: list[str] | None = None, exclude_platforms: List[str] = None,
) -> tuple[list[dict[str, Any]], int]: ) -> Tuple[List[Dict[str, Any]], int]:
"""获取筛选后的对话列表""" """获取筛选后的对话列表"""
try: try:
c = self.conn.cursor() c = self.conn.cursor()
@@ -427,7 +421,7 @@ class SQLiteDatabase:
if search_query: if search_query:
search_query = search_query.encode("unicode_escape").decode("utf-8") search_query = search_query.encode("unicode_escape").decode("utf-8")
where_clauses.append( where_clauses.append(
"(title LIKE ? OR user_id LIKE ? OR cid LIKE ? OR history LIKE ?)", "(title LIKE ? OR user_id LIKE ? OR cid LIKE ? OR history LIKE ?)"
) )
search_param = f"%{search_query}%" search_param = f"%{search_query}%"
params.extend([search_param, search_param, search_param, search_param]) params.extend([search_param, search_param, search_param, search_param])
@@ -487,7 +481,7 @@ class SQLiteDatabase:
"persona_id": persona_id or "", "persona_id": persona_id or "",
"created_at": created_at or 0, "created_at": created_at or 0,
"updated_at": updated_at or 0, "updated_at": updated_at or 0,
}, }
) )
return conversations, total_count return conversations, total_count
+38 -88
View File
@@ -1,9 +1,15 @@
import uuid import uuid
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import TypedDict
from sqlmodel import JSON, Field, SQLModel, Text, UniqueConstraint from datetime import datetime, timezone
from dataclasses import dataclass, field
from sqlmodel import (
SQLModel,
Text,
JSON,
UniqueConstraint,
Field,
)
from typing import Optional, TypedDict
class PlatformStat(SQLModel, table=True): class PlatformStat(SQLModel, table=True):
@@ -12,7 +18,7 @@ class PlatformStat(SQLModel, table=True):
Note: In astrbot v4, we moved `platform` table to here. Note: In astrbot v4, we moved `platform` table to here.
""" """
__tablename__: str = "platform_stats" __tablename__ = "platform_stats"
id: int = Field(primary_key=True, sa_column_kwargs={"autoincrement": True}) id: int = Field(primary_key=True, sa_column_kwargs={"autoincrement": True})
timestamp: datetime = Field(nullable=False) timestamp: datetime = Field(nullable=False)
@@ -31,12 +37,10 @@ class PlatformStat(SQLModel, table=True):
class ConversationV2(SQLModel, table=True): class ConversationV2(SQLModel, table=True):
__tablename__: str = "conversations" __tablename__ = "conversations"
inner_conversation_id: int | None = Field( inner_conversation_id: int = Field(
default=None, primary_key=True, sa_column_kwargs={"autoincrement": True}
primary_key=True,
sa_column_kwargs={"autoincrement": True},
) )
conversation_id: str = Field( conversation_id: str = Field(
max_length=36, max_length=36,
@@ -46,14 +50,14 @@ class ConversationV2(SQLModel, table=True):
) )
platform_id: str = Field(nullable=False) platform_id: str = Field(nullable=False)
user_id: str = Field(nullable=False) user_id: str = Field(nullable=False)
content: list | None = Field(default=None, sa_type=JSON) content: Optional[list] = Field(default=None, sa_type=JSON)
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc)) created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
updated_at: datetime = Field( updated_at: datetime = Field(
default_factory=lambda: datetime.now(timezone.utc), default_factory=lambda: datetime.now(timezone.utc),
sa_column_kwargs={"onupdate": datetime.now(timezone.utc)}, sa_column_kwargs={"onupdate": datetime.now(timezone.utc)},
) )
title: str | None = Field(default=None, max_length=255) title: Optional[str] = Field(default=None, max_length=255)
persona_id: str | None = Field(default=None) persona_id: Optional[str] = Field(default=None)
__table_args__ = ( __table_args__ = (
UniqueConstraint( UniqueConstraint(
@@ -69,18 +73,14 @@ class Persona(SQLModel, table=True):
It can be used to customize the behavior of LLMs. It can be used to customize the behavior of LLMs.
""" """
__tablename__: str = "personas" __tablename__ = "personas"
id: int | None = Field( id: int = Field(primary_key=True, sa_column_kwargs={"autoincrement": True})
primary_key=True,
sa_column_kwargs={"autoincrement": True},
default=None,
)
persona_id: str = Field(max_length=255, nullable=False) persona_id: str = Field(max_length=255, nullable=False)
system_prompt: str = Field(sa_type=Text, nullable=False) system_prompt: str = Field(sa_type=Text, nullable=False)
begin_dialogs: list | None = Field(default=None, sa_type=JSON) begin_dialogs: Optional[list] = Field(default=None, sa_type=JSON)
"""a list of strings, each representing a dialog to start with""" """a list of strings, each representing a dialog to start with"""
tools: list | None = Field(default=None, sa_type=JSON) tools: Optional[list] = Field(default=None, sa_type=JSON)
"""None means use ALL tools for default, empty list means no tools, otherwise a list of tool names.""" """None means use ALL tools for default, empty list means no tools, otherwise a list of tool names."""
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc)) created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
updated_at: datetime = Field( updated_at: datetime = Field(
@@ -99,12 +99,10 @@ class Persona(SQLModel, table=True):
class Preference(SQLModel, table=True): class Preference(SQLModel, table=True):
"""This class represents preferences for bots.""" """This class represents preferences for bots."""
__tablename__: str = "preferences" __tablename__ = "preferences"
id: int | None = Field( id: int | None = Field(
default=None, default=None, primary_key=True, sa_column_kwargs={"autoincrement": True}
primary_key=True,
sa_column_kwargs={"autoincrement": True},
) )
scope: str = Field(nullable=False) scope: str = Field(nullable=False)
"""Scope of the preference, such as 'global', 'umo', 'plugin'.""" """Scope of the preference, such as 'global', 'umo', 'plugin'."""
@@ -135,18 +133,14 @@ class PlatformMessageHistory(SQLModel, table=True):
or platform-specific messages. or platform-specific messages.
""" """
__tablename__: str = "platform_message_history" __tablename__ = "platform_message_history"
id: int | None = Field( id: int = Field(primary_key=True, sa_column_kwargs={"autoincrement": True})
primary_key=True,
sa_column_kwargs={"autoincrement": True},
default=None,
)
platform_id: str = Field(nullable=False) platform_id: str = Field(nullable=False)
user_id: str = Field(nullable=False) # An id of group, user in platform user_id: str = Field(nullable=False) # An id of group, user in platform
sender_id: str | None = Field(default=None) # ID of the sender in the platform sender_id: Optional[str] = Field(default=None) # ID of the sender in the platform
sender_name: str | None = Field( sender_name: Optional[str] = Field(
default=None, default=None
) # Name of the sender in the platform ) # Name of the sender in the platform
content: dict = Field(sa_type=JSON, nullable=False) # a message chain list content: dict = Field(sa_type=JSON, nullable=False) # a message chain list
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc)) created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
@@ -156,60 +150,16 @@ class PlatformMessageHistory(SQLModel, table=True):
) )
class PlatformSession(SQLModel, table=True):
"""Platform session table for managing user sessions across different platforms.
A session represents a chat window for a specific user on a specific platform.
Each session can have multiple conversations (对话) associated with it.
"""
__tablename__: str = "platform_sessions"
inner_id: int | None = Field(
primary_key=True,
sa_column_kwargs={"autoincrement": True},
default=None,
)
session_id: str = Field(
max_length=100,
nullable=False,
unique=True,
default_factory=lambda: str(uuid.uuid4()),
)
platform_id: str = Field(default="webchat", nullable=False)
"""Platform identifier (e.g., 'webchat', 'qq', 'discord')"""
creator: str = Field(nullable=False)
"""Username of the session creator"""
display_name: str | None = Field(default=None, max_length=255)
"""Display name for the session"""
is_group: int = Field(default=0, nullable=False)
"""0 for private chat, 1 for group chat (not implemented yet)"""
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
updated_at: datetime = Field(
default_factory=lambda: datetime.now(timezone.utc),
sa_column_kwargs={"onupdate": datetime.now(timezone.utc)},
)
__table_args__ = (
UniqueConstraint(
"session_id",
name="uix_platform_session_id",
),
)
class Attachment(SQLModel, table=True): class Attachment(SQLModel, table=True):
"""This class represents attachments for messages in AstrBot. """This class represents attachments for messages in AstrBot.
Attachments can be images, files, or other media types. Attachments can be images, files, or other media types.
""" """
__tablename__: str = "attachments" __tablename__ = "attachments"
inner_attachment_id: int | None = Field( inner_attachment_id: int = Field(
primary_key=True, primary_key=True, sa_column_kwargs={"autoincrement": True}
sa_column_kwargs={"autoincrement": True},
default=None,
) )
attachment_id: str = Field( attachment_id: str = Field(
max_length=36, max_length=36,
@@ -262,17 +212,17 @@ class Personality(TypedDict):
v4.0.0 版本及之后推荐使用上面的 Persona 并且 mood_imitation_dialogs 字段已被废弃 v4.0.0 版本及之后推荐使用上面的 Persona 并且 mood_imitation_dialogs 字段已被废弃
""" """
prompt: str prompt: str = ""
name: str name: str = ""
begin_dialogs: list[str] begin_dialogs: list[str] = []
mood_imitation_dialogs: list[str] mood_imitation_dialogs: list[str] = []
"""情感模拟对话预设。在 v4.0.0 版本及之后,已被废弃。""" """情感模拟对话预设。在 v4.0.0 版本及之后,已被废弃。"""
tools: list[str] | None tools: list[str] | None = None
"""工具列表。None 表示使用所有工具,空列表表示不使用任何工具""" """工具列表。None 表示使用所有工具,空列表表示不使用任何工具"""
# cache # cache
_begin_dialogs_processed: list[dict] _begin_dialogs_processed: list[dict] = []
_mood_imitation_dialogs_processed: str _mood_imitation_dialogs_processed: str = ""
# ==== # ====
+57 -380
View File
@@ -1,29 +1,23 @@
import asyncio import asyncio
import threading
import typing as T import typing as T
from datetime import datetime, timedelta, timezone import threading
from datetime import datetime, timedelta
from sqlalchemy import CursorResult
from sqlalchemy.ext.asyncio import AsyncSession
from sqlmodel import col, delete, desc, func, or_, select, text, update
from astrbot.core.db import BaseDatabase from astrbot.core.db import BaseDatabase
from astrbot.core.db.po import ( from astrbot.core.db.po import (
Attachment,
ConversationV2, ConversationV2,
Persona,
PlatformMessageHistory,
PlatformSession,
PlatformStat, PlatformStat,
PlatformMessageHistory,
Attachment,
Persona,
Preference, Preference,
Stats as DeprecatedStats,
Platform as DeprecatedPlatformStat,
SQLModel, SQLModel,
) )
from astrbot.core.db.po import (
Platform as DeprecatedPlatformStat, from sqlalchemy import select, update, delete, text
) from sqlalchemy.ext.asyncio import AsyncSession
from astrbot.core.db.po import ( from sqlalchemy.sql import func
Stats as DeprecatedStats,
)
NOT_GIVEN = T.TypeVar("NOT_GIVEN") NOT_GIVEN = T.TypeVar("NOT_GIVEN")
@@ -39,12 +33,6 @@ class SQLiteDatabase(BaseDatabase):
"""Initialize the database by creating tables if they do not exist.""" """Initialize the database by creating tables if they do not exist."""
async with self.engine.begin() as conn: async with self.engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all) await conn.run_sync(SQLModel.metadata.create_all)
await conn.execute(text("PRAGMA journal_mode=WAL"))
await conn.execute(text("PRAGMA synchronous=NORMAL"))
await conn.execute(text("PRAGMA cache_size=20000"))
await conn.execute(text("PRAGMA temp_store=MEMORY"))
await conn.execute(text("PRAGMA mmap_size=134217728"))
await conn.execute(text("PRAGMA optimize"))
await conn.commit() await conn.commit()
# ==== # ====
@@ -53,10 +41,10 @@ class SQLiteDatabase(BaseDatabase):
async def insert_platform_stats( async def insert_platform_stats(
self, self,
platform_id, platform_id: str,
platform_type, platform_type: str,
count=1, count: int = 1,
timestamp=None, timestamp: datetime = None,
) -> None: ) -> None:
"""Insert a new platform statistic record.""" """Insert a new platform statistic record."""
async with self.get_db() as session: async with self.get_db() as session:
@@ -64,9 +52,7 @@ class SQLiteDatabase(BaseDatabase):
async with session.begin(): async with session.begin():
if timestamp is None: if timestamp is None:
timestamp = datetime.now().replace( timestamp = datetime.now().replace(
minute=0, minute=0, second=0, microsecond=0
second=0,
microsecond=0,
) )
current_hour = timestamp current_hour = timestamp
await session.execute( await session.execute(
@@ -89,14 +75,12 @@ class SQLiteDatabase(BaseDatabase):
async with self.get_db() as session: async with self.get_db() as session:
session: AsyncSession session: AsyncSession
result = await session.execute( result = await session.execute(
select(func.count(col(PlatformStat.platform_id))).select_from( select(func.count(PlatformStat.platform_id)).select_from(PlatformStat)
PlatformStat,
),
) )
count = result.scalar_one_or_none() count = result.scalar_one_or_none()
return count if count is not None else 0 return count if count is not None else 0
async def get_platform_stats(self, offset_sec: int = 86400) -> list[PlatformStat]: async def get_platform_stats(self, offset_sec: int = 86400) -> T.List[PlatformStat]:
"""Get platform statistics within the specified offset in seconds and group by platform_id.""" """Get platform statistics within the specified offset in seconds and group by platform_id."""
async with self.get_db() as session: async with self.get_db() as session:
session: AsyncSession session: AsyncSession
@@ -106,12 +90,12 @@ class SQLiteDatabase(BaseDatabase):
text(""" text("""
SELECT * FROM platform_stats SELECT * FROM platform_stats
WHERE timestamp >= :start_time WHERE timestamp >= :start_time
GROUP BY platform_id
ORDER BY timestamp DESC ORDER BY timestamp DESC
GROUP BY platform_id
"""), """),
{"start_time": start_time}, {"start_time": start_time},
) )
return list(result.scalars().all()) return result.scalars().all()
# ==== # ====
# Conversation Management # Conversation Management
@@ -127,7 +111,7 @@ class SQLiteDatabase(BaseDatabase):
if platform_id: if platform_id:
query = query.where(ConversationV2.platform_id == platform_id) query = query.where(ConversationV2.platform_id == platform_id)
# order by # order by
query = query.order_by(desc(ConversationV2.created_at)) query = query.order_by(ConversationV2.created_at.desc())
result = await session.execute(query) result = await session.execute(query)
return result.scalars().all() return result.scalars().all()
@@ -145,9 +129,9 @@ class SQLiteDatabase(BaseDatabase):
offset = (page - 1) * page_size offset = (page - 1) * page_size
result = await session.execute( result = await session.execute(
select(ConversationV2) select(ConversationV2)
.order_by(desc(ConversationV2.created_at)) .order_by(ConversationV2.created_at.desc())
.offset(offset) .offset(offset)
.limit(page_size), .limit(page_size)
) )
return result.scalars().all() return result.scalars().all()
@@ -166,26 +150,11 @@ class SQLiteDatabase(BaseDatabase):
if platform_ids: if platform_ids:
base_query = base_query.where( base_query = base_query.where(
col(ConversationV2.platform_id).in_(platform_ids), ConversationV2.platform_id.in_(platform_ids)
) )
if search_query: if search_query:
search_query = search_query.encode("unicode_escape").decode("utf-8")
base_query = base_query.where( base_query = base_query.where(
or_( ConversationV2.title.ilike(f"%{search_query}%")
col(ConversationV2.title).ilike(f"%{search_query}%"),
col(ConversationV2.content).ilike(f"%{search_query}%"),
col(ConversationV2.user_id).ilike(f"%{search_query}%"),
col(ConversationV2.conversation_id).ilike(f"%{search_query}%"),
),
)
if "message_types" in kwargs and len(kwargs["message_types"]) > 0:
for msg_type in kwargs["message_types"]:
base_query = base_query.where(
col(ConversationV2.user_id).ilike(f"%:{msg_type}:%"),
)
if "platforms" in kwargs and len(kwargs["platforms"]) > 0:
base_query = base_query.where(
col(ConversationV2.platform_id).in_(kwargs["platforms"]),
) )
# Get total count matching the filters # Get total count matching the filters
@@ -196,7 +165,7 @@ class SQLiteDatabase(BaseDatabase):
# Get paginated results # Get paginated results
offset = (page - 1) * page_size offset = (page - 1) * page_size
result_query = ( result_query = (
base_query.order_by(desc(ConversationV2.created_at)) base_query.order_by(ConversationV2.created_at.desc())
.offset(offset) .offset(offset)
.limit(page_size) .limit(page_size)
) )
@@ -242,7 +211,7 @@ class SQLiteDatabase(BaseDatabase):
session: AsyncSession session: AsyncSession
async with session.begin(): async with session.begin():
query = update(ConversationV2).where( query = update(ConversationV2).where(
col(ConversationV2.conversation_id) == cid, ConversationV2.conversation_id == cid
) )
values = {} values = {}
if title is not None: if title is not None:
@@ -252,7 +221,7 @@ class SQLiteDatabase(BaseDatabase):
if content is not None: if content is not None:
values["content"] = content values["content"] = content
if not values: if not values:
return None return
query = query.values(**values) query = query.values(**values)
await session.execute(query) await session.execute(query)
return await self.get_conversation_by_id(cid) return await self.get_conversation_by_id(cid)
@@ -262,130 +231,9 @@ class SQLiteDatabase(BaseDatabase):
session: AsyncSession session: AsyncSession
async with session.begin(): async with session.begin():
await session.execute( await session.execute(
delete(ConversationV2).where( delete(ConversationV2).where(ConversationV2.conversation_id == cid)
col(ConversationV2.conversation_id) == cid,
),
) )
async def delete_conversations_by_user_id(self, user_id: str) -> None:
async with self.get_db() as session:
session: AsyncSession
async with session.begin():
await session.execute(
delete(ConversationV2).where(
col(ConversationV2.user_id) == user_id
),
)
async def get_session_conversations(
self,
page=1,
page_size=20,
search_query=None,
platform=None,
) -> tuple[list[dict], int]:
"""Get paginated session conversations with joined conversation and persona details."""
async with self.get_db() as session:
session: AsyncSession
offset = (page - 1) * page_size
base_query = (
select(
col(Preference.scope_id).label("session_id"),
func.json_extract(Preference.value, "$.val").label(
"conversation_id",
), # type: ignore
col(ConversationV2.persona_id).label("persona_id"),
col(ConversationV2.title).label("title"),
col(Persona.persona_id).label("persona_name"),
)
.select_from(Preference)
.outerjoin(
ConversationV2,
func.json_extract(Preference.value, "$.val")
== ConversationV2.conversation_id,
)
.outerjoin(
Persona,
col(ConversationV2.persona_id) == Persona.persona_id,
)
.where(Preference.scope == "umo", Preference.key == "sel_conv_id")
)
# 搜索筛选
if search_query:
search_pattern = f"%{search_query}%"
base_query = base_query.where(
or_(
col(Preference.scope_id).ilike(search_pattern),
col(ConversationV2.title).ilike(search_pattern),
col(Persona.persona_id).ilike(search_pattern),
),
)
# 平台筛选
if platform:
platform_pattern = f"{platform}:%"
base_query = base_query.where(
col(Preference.scope_id).like(platform_pattern),
)
# 排序
base_query = base_query.order_by(Preference.scope_id)
# 分页结果
result_query = base_query.offset(offset).limit(page_size)
result = await session.execute(result_query)
rows = result.fetchall()
# 查询总数(应用相同的筛选条件)
count_base_query = (
select(func.count(col(Preference.scope_id)))
.select_from(Preference)
.outerjoin(
ConversationV2,
func.json_extract(Preference.value, "$.val")
== ConversationV2.conversation_id,
)
.outerjoin(
Persona,
col(ConversationV2.persona_id) == Persona.persona_id,
)
.where(Preference.scope == "umo", Preference.key == "sel_conv_id")
)
# 应用相同的搜索和平台筛选条件到计数查询
if search_query:
search_pattern = f"%{search_query}%"
count_base_query = count_base_query.where(
or_(
col(Preference.scope_id).ilike(search_pattern),
col(ConversationV2.title).ilike(search_pattern),
col(Persona.persona_id).ilike(search_pattern),
),
)
if platform:
platform_pattern = f"{platform}:%"
count_base_query = count_base_query.where(
col(Preference.scope_id).like(platform_pattern),
)
total_result = await session.execute(count_base_query)
total = total_result.scalar() or 0
sessions_data = [
{
"session_id": row.session_id,
"conversation_id": row.conversation_id,
"persona_id": row.persona_id,
"title": row.title,
"persona_name": row.persona_name,
}
for row in rows
]
return sessions_data, total
async def insert_platform_message_history( async def insert_platform_message_history(
self, self,
platform_id, platform_id,
@@ -409,12 +257,9 @@ class SQLiteDatabase(BaseDatabase):
return new_history return new_history
async def delete_platform_message_offset( async def delete_platform_message_offset(
self, self, platform_id, user_id, offset_sec=86400
platform_id,
user_id,
offset_sec=86400,
): ):
"""Delete platform message history records newer than the specified offset.""" """Delete platform message history records older than the specified offset."""
async with self.get_db() as session: async with self.get_db() as session:
session: AsyncSession session: AsyncSession
async with session.begin(): async with session.begin():
@@ -422,18 +267,14 @@ class SQLiteDatabase(BaseDatabase):
cutoff_time = now - timedelta(seconds=offset_sec) cutoff_time = now - timedelta(seconds=offset_sec)
await session.execute( await session.execute(
delete(PlatformMessageHistory).where( delete(PlatformMessageHistory).where(
col(PlatformMessageHistory.platform_id) == platform_id, PlatformMessageHistory.platform_id == platform_id,
col(PlatformMessageHistory.user_id) == user_id, PlatformMessageHistory.user_id == user_id,
col(PlatformMessageHistory.created_at) >= cutoff_time, PlatformMessageHistory.created_at < cutoff_time,
), )
) )
async def get_platform_message_history( async def get_platform_message_history(
self, self, platform_id, user_id, page=1, page_size=20
platform_id,
user_id,
page=1,
page_size=20,
): ):
"""Get platform message history records.""" """Get platform message history records."""
async with self.get_db() as session: async with self.get_db() as session:
@@ -445,23 +286,11 @@ class SQLiteDatabase(BaseDatabase):
PlatformMessageHistory.platform_id == platform_id, PlatformMessageHistory.platform_id == platform_id,
PlatformMessageHistory.user_id == user_id, PlatformMessageHistory.user_id == user_id,
) )
.order_by(desc(PlatformMessageHistory.created_at)) .order_by(PlatformMessageHistory.created_at.desc())
) )
result = await session.execute(query.offset(offset).limit(page_size)) result = await session.execute(query.offset(offset).limit(page_size))
return result.scalars().all() return result.scalars().all()
async def get_platform_message_history_by_id(
self, message_id: int
) -> PlatformMessageHistory | None:
"""Get a platform message history record by its ID."""
async with self.get_db() as session:
session: AsyncSession
query = select(PlatformMessageHistory).where(
PlatformMessageHistory.id == message_id
)
result = await session.execute(query)
return result.scalar_one_or_none()
async def insert_attachment(self, path, type, mime_type): async def insert_attachment(self, path, type, mime_type):
"""Insert a new attachment record.""" """Insert a new attachment record."""
async with self.get_db() as session: async with self.get_db() as session:
@@ -479,58 +308,12 @@ class SQLiteDatabase(BaseDatabase):
"""Get an attachment by its ID.""" """Get an attachment by its ID."""
async with self.get_db() as session: async with self.get_db() as session:
session: AsyncSession session: AsyncSession
query = select(Attachment).where(Attachment.attachment_id == attachment_id) query = select(Attachment).where(Attachment.id == attachment_id)
result = await session.execute(query) result = await session.execute(query)
return result.scalar_one_or_none() return result.scalar_one_or_none()
async def get_attachments(self, attachment_ids: list[str]) -> list:
"""Get multiple attachments by their IDs."""
if not attachment_ids:
return []
async with self.get_db() as session:
session: AsyncSession
query = select(Attachment).where(
col(Attachment.attachment_id).in_(attachment_ids)
)
result = await session.execute(query)
return list(result.scalars().all())
async def delete_attachment(self, attachment_id: str) -> bool:
"""Delete an attachment by its ID.
Returns True if the attachment was deleted, False if it was not found.
"""
async with self.get_db() as session:
session: AsyncSession
async with session.begin():
query = delete(Attachment).where(
col(Attachment.attachment_id) == attachment_id
)
result = T.cast(CursorResult, await session.execute(query))
return result.rowcount > 0
async def delete_attachments(self, attachment_ids: list[str]) -> int:
"""Delete multiple attachments by their IDs.
Returns the number of attachments deleted.
"""
if not attachment_ids:
return 0
async with self.get_db() as session:
session: AsyncSession
async with session.begin():
query = delete(Attachment).where(
col(Attachment.attachment_id).in_(attachment_ids)
)
result = T.cast(CursorResult, await session.execute(query))
return result.rowcount
async def insert_persona( async def insert_persona(
self, self, persona_id, system_prompt, begin_dialogs=None, tools=None
persona_id,
system_prompt,
begin_dialogs=None,
tools=None,
): ):
"""Insert a new persona record.""" """Insert a new persona record."""
async with self.get_db() as session: async with self.get_db() as session:
@@ -562,17 +345,13 @@ class SQLiteDatabase(BaseDatabase):
return result.scalars().all() return result.scalars().all()
async def update_persona( async def update_persona(
self, self, persona_id, system_prompt=None, begin_dialogs=None, tools=NOT_GIVEN
persona_id,
system_prompt=None,
begin_dialogs=None,
tools=NOT_GIVEN,
): ):
"""Update a persona's system prompt or begin dialogs.""" """Update a persona's system prompt or begin dialogs."""
async with self.get_db() as session: async with self.get_db() as session:
session: AsyncSession session: AsyncSession
async with session.begin(): async with session.begin():
query = update(Persona).where(col(Persona.persona_id) == persona_id) query = update(Persona).where(Persona.persona_id == persona_id)
values = {} values = {}
if system_prompt is not None: if system_prompt is not None:
values["system_prompt"] = system_prompt values["system_prompt"] = system_prompt
@@ -581,7 +360,7 @@ class SQLiteDatabase(BaseDatabase):
if tools is not NOT_GIVEN: if tools is not NOT_GIVEN:
values["tools"] = tools values["tools"] = tools
if not values: if not values:
return None return
query = query.values(**values) query = query.values(**values)
await session.execute(query) await session.execute(query)
return await self.get_persona_by_id(persona_id) return await self.get_persona_by_id(persona_id)
@@ -592,7 +371,7 @@ class SQLiteDatabase(BaseDatabase):
session: AsyncSession session: AsyncSession
async with session.begin(): async with session.begin():
await session.execute( await session.execute(
delete(Persona).where(col(Persona.persona_id) == persona_id), delete(Persona).where(Persona.persona_id == persona_id)
) )
async def insert_preference_or_update(self, scope, scope_id, key, value): async def insert_preference_or_update(self, scope, scope_id, key, value):
@@ -611,10 +390,7 @@ class SQLiteDatabase(BaseDatabase):
existing_preference.value = value existing_preference.value = value
else: else:
new_preference = Preference( new_preference = Preference(
scope=scope, scope=scope, scope_id=scope_id, key=key, value=value
scope_id=scope_id,
key=key,
value=value,
) )
session.add(new_preference) session.add(new_preference)
return existing_preference or new_preference return existing_preference or new_preference
@@ -650,10 +426,10 @@ class SQLiteDatabase(BaseDatabase):
async with session.begin(): async with session.begin():
await session.execute( await session.execute(
delete(Preference).where( delete(Preference).where(
col(Preference.scope) == scope, Preference.scope == scope,
col(Preference.scope_id) == scope_id, Preference.scope_id == scope_id,
col(Preference.key) == key, Preference.key == key,
), )
) )
await session.commit() await session.commit()
@@ -664,9 +440,8 @@ class SQLiteDatabase(BaseDatabase):
async with session.begin(): async with session.begin():
await session.execute( await session.execute(
delete(Preference).where( delete(Preference).where(
col(Preference.scope) == scope, Preference.scope == scope, Preference.scope_id == scope_id
col(Preference.scope_id) == scope_id, )
),
) )
await session.commit() await session.commit()
@@ -683,7 +458,7 @@ class SQLiteDatabase(BaseDatabase):
now = datetime.now() now = datetime.now()
start_time = now - timedelta(seconds=offset_sec) start_time = now - timedelta(seconds=offset_sec)
result = await session.execute( result = await session.execute(
select(PlatformStat).where(PlatformStat.timestamp >= start_time), select(PlatformStat).where(PlatformStat.timestamp >= start_time)
) )
all_datas = result.scalars().all() all_datas = result.scalars().all()
deprecated_stats = DeprecatedStats() deprecated_stats = DeprecatedStats()
@@ -692,8 +467,8 @@ class SQLiteDatabase(BaseDatabase):
DeprecatedPlatformStat( DeprecatedPlatformStat(
name=data.platform_id, name=data.platform_id,
count=data.count, count=data.count,
timestamp=int(data.timestamp.timestamp()), timestamp=data.timestamp.timestamp(),
), )
) )
return deprecated_stats return deprecated_stats
@@ -715,7 +490,7 @@ class SQLiteDatabase(BaseDatabase):
async with self.get_db() as session: async with self.get_db() as session:
session: AsyncSession session: AsyncSession
result = await session.execute( result = await session.execute(
select(func.sum(PlatformStat.count)).select_from(PlatformStat), select(func.sum(PlatformStat.count)).select_from(PlatformStat)
) )
total_count = result.scalar_one_or_none() total_count = result.scalar_one_or_none()
return total_count if total_count is not None else 0 return total_count if total_count is not None else 0
@@ -741,7 +516,7 @@ class SQLiteDatabase(BaseDatabase):
result = await session.execute( result = await session.execute(
select(PlatformStat.platform_id, func.sum(PlatformStat.count)) select(PlatformStat.platform_id, func.sum(PlatformStat.count))
.where(PlatformStat.timestamp >= start_time) .where(PlatformStat.timestamp >= start_time)
.group_by(PlatformStat.platform_id), .group_by(PlatformStat.platform_id)
) )
grouped_stats = result.all() grouped_stats = result.all()
deprecated_stats = DeprecatedStats() deprecated_stats = DeprecatedStats()
@@ -750,8 +525,8 @@ class SQLiteDatabase(BaseDatabase):
DeprecatedPlatformStat( DeprecatedPlatformStat(
name=platform_id, name=platform_id,
count=count, count=count,
timestamp=int(start_time.timestamp()), timestamp=start_time.timestamp(),
), )
) )
return deprecated_stats return deprecated_stats
@@ -765,101 +540,3 @@ class SQLiteDatabase(BaseDatabase):
t.start() t.start()
t.join() t.join()
return result return result
# ====
# Platform Session Management
# ====
async def create_platform_session(
self,
creator: str,
platform_id: str = "webchat",
session_id: str | None = None,
display_name: str | None = None,
is_group: int = 0,
) -> PlatformSession:
"""Create a new Platform session."""
kwargs = {}
if session_id:
kwargs["session_id"] = session_id
async with self.get_db() as session:
session: AsyncSession
async with session.begin():
new_session = PlatformSession(
creator=creator,
platform_id=platform_id,
display_name=display_name,
is_group=is_group,
**kwargs,
)
session.add(new_session)
await session.flush()
await session.refresh(new_session)
return new_session
async def get_platform_session_by_id(
self, session_id: str
) -> PlatformSession | None:
"""Get a Platform session by its ID."""
async with self.get_db() as session:
session: AsyncSession
query = select(PlatformSession).where(
PlatformSession.session_id == session_id,
)
result = await session.execute(query)
return result.scalar_one_or_none()
async def get_platform_sessions_by_creator(
self,
creator: str,
platform_id: str | None = None,
page: int = 1,
page_size: int = 20,
) -> list[PlatformSession]:
"""Get all Platform sessions for a specific creator (username) and optionally platform."""
async with self.get_db() as session:
session: AsyncSession
offset = (page - 1) * page_size
query = select(PlatformSession).where(PlatformSession.creator == creator)
if platform_id:
query = query.where(PlatformSession.platform_id == platform_id)
query = (
query.order_by(desc(PlatformSession.updated_at))
.offset(offset)
.limit(page_size)
)
result = await session.execute(query)
return list(result.scalars().all())
async def update_platform_session(
self,
session_id: str,
display_name: str | None = None,
) -> None:
"""Update a Platform session's updated_at timestamp and optionally display_name."""
async with self.get_db() as session:
session: AsyncSession
async with session.begin():
values: dict[str, T.Any] = {"updated_at": datetime.now(timezone.utc)}
if display_name is not None:
values["display_name"] = display_name
await session.execute(
update(PlatformSession)
.where(col(PlatformSession.session_id) == session_id)
.values(**values),
)
async def delete_platform_session(self, session_id: str) -> None:
"""Delete a Platform session by its ID."""
async with self.get_db() as session:
session: AsyncSession
async with session.begin():
await session.execute(
delete(PlatformSession).where(
col(PlatformSession.session_id) == session_id,
),
)
+12 -39
View File
@@ -10,47 +10,22 @@ class Result:
class BaseVecDB: class BaseVecDB:
async def initialize(self): async def initialize(self):
"""初始化向量数据库""" """
初始化向量数据库
"""
pass
@abc.abstractmethod @abc.abstractmethod
async def insert( async def insert(self, content: str, metadata: dict = None, id: str = None) -> int:
self, """
content: str, 插入一条文本和其对应向量自动生成 ID 并保持一致性
metadata: dict | None = None,
id: str | None = None,
) -> int:
"""插入一条文本和其对应向量,自动生成 ID 并保持一致性。"""
...
@abc.abstractmethod
async def insert_batch(
self,
contents: list[str],
metadatas: list[dict] | None = None,
ids: list[str] | None = None,
batch_size: int = 32,
tasks_limit: int = 3,
max_retries: int = 3,
progress_callback=None,
) -> int:
"""批量插入文本和其对应向量,自动生成 ID 并保持一致性。
Args:
progress_callback: 进度回调函数接收参数 (current, total)
""" """
... ...
@abc.abstractmethod @abc.abstractmethod
async def retrieve( async def retrieve(self, query: str, top_k: int = 5) -> list[Result]:
self, """
query: str, 搜索最相似的文档
top_k: int = 5,
fetch_k: int = 20,
rerank: bool = False,
metadata_filters: dict | None = None,
) -> list[Result]:
"""搜索最相似的文档。
Args: Args:
query (str): 查询文本 query (str): 查询文本
top_k (int): 返回的最相似文档的数量 top_k (int): 返回的最相似文档的数量
@@ -61,13 +36,11 @@ class BaseVecDB:
@abc.abstractmethod @abc.abstractmethod
async def delete(self, doc_id: str) -> bool: async def delete(self, doc_id: str) -> bool:
"""删除指定文档。 """
删除指定文档
Args: Args:
doc_id (str): 要删除的文档 ID doc_id (str): 要删除的文档 ID
Returns: Returns:
bool: 删除是否成功 bool: 删除是否成功
""" """
... ...
@abc.abstractmethod
async def close(self): ...
@@ -1,3 +1,3 @@
from .vec_db import FaissVecDB from .vec_db import FaissVecDB
__all__ = ["FaissVecDB"] __all__ = ["FaissVecDB"]
@@ -1,232 +1,59 @@
import json import aiosqlite
import os import os
from contextlib import asynccontextmanager
from datetime import datetime
from sqlalchemy import Column, Text
from sqlalchemy.ext.asyncio import AsyncEngine, AsyncSession, create_async_engine
from sqlalchemy.orm import sessionmaker
from sqlmodel import Field, MetaData, SQLModel, col, func, select, text
from astrbot.core import logger
class BaseDocModel(SQLModel, table=False):
metadata = MetaData()
class Document(BaseDocModel, table=True):
"""SQLModel for documents table."""
__tablename__ = "documents" # type: ignore
id: int | None = Field(
default=None,
primary_key=True,
sa_column_kwargs={"autoincrement": True},
)
doc_id: str = Field(nullable=False)
text: str = Field(nullable=False)
metadata_: str | None = Field(default=None, sa_column=Column("metadata", Text))
created_at: datetime | None = Field(default=None)
updated_at: datetime | None = Field(default=None)
class DocumentStorage: class DocumentStorage:
def __init__(self, db_path: str): def __init__(self, db_path: str):
self.db_path = db_path self.db_path = db_path
self.DATABASE_URL = f"sqlite+aiosqlite:///{db_path}" self.connection = None
self.engine: AsyncEngine | None = None
self.async_session_maker: sessionmaker | None = None
self.sqlite_init_path = os.path.join( self.sqlite_init_path = os.path.join(
os.path.dirname(__file__), os.path.dirname(__file__), "sqlite_init.sql"
"sqlite_init.sql",
) )
async def initialize(self): async def initialize(self):
"""Initialize the SQLite database and create the documents table if it doesn't exist.""" """Initialize the SQLite database and create the documents table if it doesn't exist."""
await self.connect() if not os.path.exists(self.db_path):
async with self.engine.begin() as conn: # type: ignore await self.connect()
# Create tables using SQLModel async with self.connection.cursor() as cursor:
await conn.run_sync(BaseDocModel.metadata.create_all) with open(self.sqlite_init_path, "r", encoding="utf-8") as f:
sql_script = f.read()
try: await cursor.executescript(sql_script)
await conn.execute( await self.connection.commit()
text( else:
"ALTER TABLE documents ADD COLUMN kb_doc_id TEXT " await self.connect()
"GENERATED ALWAYS AS (json_extract(metadata, '$.kb_doc_id')) STORED",
),
)
await conn.execute(
text(
"ALTER TABLE documents ADD COLUMN user_id TEXT "
"GENERATED ALWAYS AS (json_extract(metadata, '$.user_id')) STORED",
),
)
# Create indexes
await conn.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_documents_kb_doc_id ON documents(kb_doc_id)",
),
)
await conn.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_documents_user_id ON documents(user_id)",
),
)
except BaseException:
pass
await conn.commit()
async def connect(self): async def connect(self):
"""Connect to the SQLite database.""" """Connect to the SQLite database."""
if self.engine is None: self.connection = await aiosqlite.connect(self.db_path)
self.engine = create_async_engine(
self.DATABASE_URL,
echo=False,
future=True,
)
self.async_session_maker = sessionmaker(
self.engine, # type: ignore
class_=AsyncSession,
expire_on_commit=False,
) # type: ignore
@asynccontextmanager async def get_documents(self, metadata_filters: dict, ids: list = None):
async def get_session(self):
"""Context manager for database sessions."""
async with self.async_session_maker() as session: # type: ignore
yield session
async def get_documents(
self,
metadata_filters: dict,
ids: list | None = None,
offset: int | None = 0,
limit: int | None = 100,
) -> list[dict]:
"""Retrieve documents by metadata filters and ids. """Retrieve documents by metadata filters and ids.
Args: Args:
metadata_filters (dict): The metadata filters to apply. metadata_filters (dict): The metadata filters to apply.
ids (list | None): Optional list of document IDs to filter.
offset (int | None): Offset for pagination.
limit (int | None): Limit for pagination.
Returns: Returns:
list: The list of documents that match the filters. list: The list of document IDs(primary key, not doc_id) that match the filters.
""" """
if self.engine is None: # metadata filter -> SQL WHERE clause
logger.warning( where_clauses = []
"Database connection is not initialized, returning empty result", values = []
) for key, val in metadata_filters.items():
return [] where_clauses.append(f"json_extract(metadata, '$.{key}') = ?")
values.append(val)
if ids is not None and len(ids) > 0:
ids = [str(i) for i in ids if i != -1]
where_clauses.append("id IN ({})".format(",".join("?" * len(ids))))
values.extend(ids)
where_sql = " AND ".join(where_clauses) or "1=1"
async with self.get_session() as session: result = []
query = select(Document) async with self.connection.cursor() as cursor:
sql = "SELECT * FROM documents WHERE " + where_sql
for key, val in metadata_filters.items(): await cursor.execute(sql, values)
query = query.where( for row in await cursor.fetchall():
text(f"json_extract(metadata, '$.{key}') = :filter_{key}"), result.append(await self.tuple_to_dict(row))
).params(**{f"filter_{key}": val}) return result
if ids is not None and len(ids) > 0:
valid_ids = [int(i) for i in ids if i != -1]
if valid_ids:
query = query.where(col(Document.id).in_(valid_ids))
if limit is not None:
query = query.limit(limit)
if offset is not None:
query = query.offset(offset)
result = await session.execute(query)
documents = result.scalars().all()
return [self._document_to_dict(doc) for doc in documents]
async def insert_document(self, doc_id: str, text: str, metadata: dict) -> int:
"""Insert a single document and return its integer ID.
Args:
doc_id (str): The document ID (UUID string).
text (str): The document text.
metadata (dict): The document metadata.
Returns:
int: The integer ID of the inserted document.
"""
assert self.engine is not None, "Database connection is not initialized."
async with self.get_session() as session, session.begin():
document = Document(
doc_id=doc_id,
text=text,
metadata_=json.dumps(metadata),
created_at=datetime.now(),
updated_at=datetime.now(),
)
session.add(document)
await session.flush() # Flush to get the ID
return document.id # type: ignore
async def insert_documents_batch(
self,
doc_ids: list[str],
texts: list[str],
metadatas: list[dict],
) -> list[int]:
"""Batch insert documents and return their integer IDs.
Args:
doc_ids (list[str]): List of document IDs (UUID strings).
texts (list[str]): List of document texts.
metadatas (list[dict]): List of document metadata.
Returns:
list[int]: List of integer IDs of the inserted documents.
"""
assert self.engine is not None, "Database connection is not initialized."
async with self.get_session() as session, session.begin():
import json
documents = []
for doc_id, text, metadata in zip(doc_ids, texts, metadatas):
document = Document(
doc_id=doc_id,
text=text,
metadata_=json.dumps(metadata),
created_at=datetime.now(),
updated_at=datetime.now(),
)
documents.append(document)
session.add(document)
await session.flush() # Flush to get all IDs
return [doc.id for doc in documents] # type: ignore
async def delete_document_by_doc_id(self, doc_id: str):
"""Delete a document by its doc_id.
Args:
doc_id (str): The doc_id of the document to delete.
"""
assert self.engine is not None, "Database connection is not initialized."
async with self.get_session() as session, session.begin():
query = select(Document).where(col(Document.doc_id) == doc_id)
result = await session.execute(query)
document = result.scalar_one_or_none()
if document:
await session.delete(document)
async def get_document_by_doc_id(self, doc_id: str): async def get_document_by_doc_id(self, doc_id: str):
"""Retrieve a document by its doc_id. """Retrieve a document by its doc_id.
@@ -235,134 +62,40 @@ class DocumentStorage:
doc_id (str): The doc_id of the document to retrieve. doc_id (str): The doc_id of the document to retrieve.
Returns: Returns:
dict: The document data or None if not found. dict: The document data.
""" """
assert self.engine is not None, "Database connection is not initialized." async with self.connection.cursor() as cursor:
await cursor.execute("SELECT * FROM documents WHERE doc_id = ?", (doc_id,))
async with self.get_session() as session: row = await cursor.fetchone()
query = select(Document).where(col(Document.doc_id) == doc_id) if row:
result = await session.execute(query) return await self.tuple_to_dict(row)
document = result.scalar_one_or_none() else:
return None
if document:
return self._document_to_dict(document)
return None
async def update_document_by_doc_id(self, doc_id: str, new_text: str): async def update_document_by_doc_id(self, doc_id: str, new_text: str):
"""Update a document by its doc_id. """Retrieve a document by its doc_id.
Args: Args:
doc_id (str): The doc_id. doc_id (str): The doc_id.
new_text (str): The new text to update the document with. new_text (str): The new text to update the document with.
""" """
assert self.engine is not None, "Database connection is not initialized." async with self.connection.cursor() as cursor:
await cursor.execute(
async with self.get_session() as session, session.begin(): "UPDATE documents SET text = ? WHERE doc_id = ?", (new_text, doc_id)
query = select(Document).where(col(Document.doc_id) == doc_id)
result = await session.execute(query)
document = result.scalar_one_or_none()
if document:
document.text = new_text
document.updated_at = datetime.now()
session.add(document)
async def delete_documents(self, metadata_filters: dict):
"""Delete documents by their metadata filters.
Args:
metadata_filters (dict): The metadata filters to apply.
"""
if self.engine is None:
logger.warning(
"Database connection is not initialized, skipping delete operation",
) )
return await self.connection.commit()
async with self.get_session() as session, session.begin():
query = select(Document)
for key, val in metadata_filters.items():
query = query.where(
text(f"json_extract(metadata, '$.{key}') = :filter_{key}"),
).params(**{f"filter_{key}": val})
result = await session.execute(query)
documents = result.scalars().all()
for doc in documents:
await session.delete(doc)
async def count_documents(self, metadata_filters: dict | None = None) -> int:
"""Count documents in the database.
Args:
metadata_filters (dict | None): Metadata filters to apply.
Returns:
int: The count of documents.
"""
if self.engine is None:
logger.warning("Database connection is not initialized, returning 0")
return 0
async with self.get_session() as session:
query = select(func.count(col(Document.id)))
if metadata_filters:
for key, val in metadata_filters.items():
query = query.where(
text(f"json_extract(metadata, '$.{key}') = :filter_{key}"),
).params(**{f"filter_{key}": val})
result = await session.execute(query)
count = result.scalar_one_or_none()
return count if count is not None else 0
async def get_user_ids(self) -> list[str]: async def get_user_ids(self) -> list[str]:
"""Retrieve all user IDs from the documents table. """Retrieve all user IDs from the documents table.
Returns: Returns:
list: A list of user IDs. list: A list of user IDs.
""" """
assert self.engine is not None, "Database connection is not initialized." async with self.connection.cursor() as cursor:
await cursor.execute("SELECT DISTINCT user_id FROM documents")
async with self.get_session() as session: rows = await cursor.fetchall()
query = text(
"SELECT DISTINCT user_id FROM documents WHERE user_id IS NOT NULL",
)
result = await session.execute(query)
rows = result.fetchall()
return [row[0] for row in rows] return [row[0] for row in rows]
def _document_to_dict(self, document: Document) -> dict:
"""Convert a Document model to a dictionary.
Args:
document (Document): The document to convert.
Returns:
dict: The converted dictionary.
"""
return {
"id": document.id,
"doc_id": document.doc_id,
"text": document.text,
"metadata": document.metadata_,
"created_at": document.created_at.isoformat()
if isinstance(document.created_at, datetime)
else document.created_at,
"updated_at": document.updated_at.isoformat()
if isinstance(document.updated_at, datetime)
else document.updated_at,
}
async def tuple_to_dict(self, row): async def tuple_to_dict(self, row):
"""Convert a tuple to a dictionary. """Convert a tuple to a dictionary.
@@ -371,9 +104,6 @@ class DocumentStorage:
Returns: Returns:
dict: The converted dictionary. dict: The converted dictionary.
Note: This method is kept for backward compatibility but is no longer used internally.
""" """
return { return {
"id": row[0], "id": row[0],
@@ -386,7 +116,6 @@ class DocumentStorage:
async def close(self): async def close(self):
"""Close the connection to the SQLite database.""" """Close the connection to the SQLite database."""
if self.engine: if self.connection:
await self.engine.dispose() await self.connection.close()
self.engine = None self.connection = None
self.async_session_maker = None
@@ -2,15 +2,14 @@ try:
import faiss import faiss
except ModuleNotFoundError: except ModuleNotFoundError:
raise ImportError( raise ImportError(
"faiss 未安装。请使用 'pip install faiss-cpu''pip install faiss-gpu' 安装。", "faiss 未安装。请使用 'pip install faiss-cpu''pip install faiss-gpu' 安装。"
) )
import os import os
import numpy as np import numpy as np
class EmbeddingStorage: class EmbeddingStorage:
def __init__(self, dimension: int, path: str | None = None): def __init__(self, dimension: int, path: str = None):
self.dimension = dimension self.dimension = dimension
self.path = path self.path = path
self.index = None self.index = None
@@ -19,6 +18,7 @@ class EmbeddingStorage:
else: else:
base_index = faiss.IndexFlatL2(dimension) base_index = faiss.IndexFlatL2(dimension)
self.index = faiss.IndexIDMap(base_index) self.index = faiss.IndexIDMap(base_index)
self.storage = {}
async def insert(self, vector: np.ndarray, id: int): async def insert(self, vector: np.ndarray, id: int):
"""插入向量 """插入向量
@@ -28,32 +28,13 @@ class EmbeddingStorage:
id (int): 向量的ID id (int): 向量的ID
Raises: Raises:
ValueError: 如果向量的维度与存储的维度不匹配 ValueError: 如果向量的维度与存储的维度不匹配
""" """
assert self.index is not None, "FAISS index is not initialized."
if vector.shape[0] != self.dimension: if vector.shape[0] != self.dimension:
raise ValueError( raise ValueError(
f"向量维度不匹配, 期望: {self.dimension}, 实际: {vector.shape[0]}", f"向量维度不匹配, 期望: {self.dimension}, 实际: {vector.shape[0]}"
) )
self.index.add_with_ids(vector.reshape(1, -1), np.array([id])) self.index.add_with_ids(vector.reshape(1, -1), np.array([id]))
await self.save_index() self.storage[id] = vector
async def insert_batch(self, vectors: np.ndarray, ids: list[int]):
"""批量插入向量
Args:
vectors (np.ndarray): 要插入的向量数组
ids (list[int]): 向量的ID列表
Raises:
ValueError: 如果向量的维度与存储的维度不匹配
"""
assert self.index is not None, "FAISS index is not initialized."
if vectors.shape[1] != self.dimension:
raise ValueError(
f"向量维度不匹配, 期望: {self.dimension}, 实际: {vectors.shape[1]}",
)
self.index.add_with_ids(vectors, np.array(ids))
await self.save_index() await self.save_index()
async def search(self, vector: np.ndarray, k: int) -> tuple: async def search(self, vector: np.ndarray, k: int) -> tuple:
@@ -64,32 +45,15 @@ class EmbeddingStorage:
k (int): 返回的最相似向量的数量 k (int): 返回的最相似向量的数量
Returns: Returns:
tuple: (距离, 索引) tuple: (距离, 索引)
""" """
assert self.index is not None, "FAISS index is not initialized."
faiss.normalize_L2(vector) faiss.normalize_L2(vector)
distances, indices = self.index.search(vector, k) distances, indices = self.index.search(vector, k)
return distances, indices return distances, indices
async def delete(self, ids: list[int]):
"""删除向量
Args:
ids (list[int]): 要删除的向量ID列表
"""
assert self.index is not None, "FAISS index is not initialized."
id_array = np.array(ids, dtype=np.int64)
self.index.remove_ids(id_array)
await self.save_index()
async def save_index(self): async def save_index(self):
"""保存索引 """保存索引
Args: Args:
path (str): 保存索引的路径 path (str): 保存索引的路径
""" """
if self.index is None:
return
faiss.write_index(self.index, self.path) faiss.write_index(self.index, self.path)
+43 -107
View File
@@ -1,18 +1,17 @@
import time
import uuid import uuid
import json
import numpy as np import numpy as np
from astrbot import logger
from astrbot.core.provider.provider import EmbeddingProvider, RerankProvider
from ..base import BaseVecDB, Result
from .document_storage import DocumentStorage from .document_storage import DocumentStorage
from .embedding_storage import EmbeddingStorage from .embedding_storage import EmbeddingStorage
from ..base import Result, BaseVecDB
from astrbot.core.provider.provider import EmbeddingProvider
from astrbot.core.provider.provider import RerankProvider
class FaissVecDB(BaseVecDB): class FaissVecDB(BaseVecDB):
"""A class to represent a vector database.""" """
A class to represent a vector database.
"""
def __init__( def __init__(
self, self,
@@ -26,8 +25,7 @@ class FaissVecDB(BaseVecDB):
self.embedding_provider = embedding_provider self.embedding_provider = embedding_provider
self.document_storage = DocumentStorage(doc_store_path) self.document_storage = DocumentStorage(doc_store_path)
self.embedding_storage = EmbeddingStorage( self.embedding_storage = EmbeddingStorage(
embedding_provider.get_dim(), embedding_provider.get_dim(), index_store_path
index_store_path,
) )
self.embedding_provider = embedding_provider self.embedding_provider = embedding_provider
self.rerank_provider = rerank_provider self.rerank_provider = rerank_provider
@@ -36,69 +34,28 @@ class FaissVecDB(BaseVecDB):
await self.document_storage.initialize() await self.document_storage.initialize()
async def insert( async def insert(
self, self, content: str, metadata: dict | None = None, id: str | None = None
content: str,
metadata: dict | None = None,
id: str | None = None,
) -> int: ) -> int:
"""插入一条文本和其对应向量,自动生成 ID 并保持一致性。""" """
插入一条文本和其对应向量自动生成 ID 并保持一致性
"""
metadata = metadata or {} metadata = metadata or {}
str_id = id or str(uuid.uuid4()) # 使用 UUID 作为原始 ID str_id = id or str(uuid.uuid4()) # 使用 UUID 作为原始 ID
vector = await self.embedding_provider.get_embedding(content) vector = await self.embedding_provider.get_embedding(content)
vector = np.array(vector, dtype=np.float32) vector = np.array(vector, dtype=np.float32)
async with self.document_storage.connection.cursor() as cursor:
await cursor.execute(
"INSERT INTO documents (doc_id, text, metadata) VALUES (?, ?, ?)",
(str_id, content, json.dumps(metadata)),
)
await self.document_storage.connection.commit()
result = await self.document_storage.get_document_by_doc_id(str_id)
int_id = result["id"]
# 使用 DocumentStorage 的方法插入文档 # 插入向量到 FAISS
int_id = await self.document_storage.insert_document(str_id, content, metadata) await self.embedding_storage.insert(vector, int_id)
return int_id
# 插入向量到 FAISS
await self.embedding_storage.insert(vector, int_id)
return int_id
async def insert_batch(
self,
contents: list[str],
metadatas: list[dict] | None = None,
ids: list[str] | None = None,
batch_size: int = 32,
tasks_limit: int = 3,
max_retries: int = 3,
progress_callback=None,
) -> list[int]:
"""批量插入文本和其对应向量,自动生成 ID 并保持一致性。
Args:
progress_callback: 进度回调函数接收参数 (current, total)
"""
metadatas = metadatas or [{} for _ in contents]
ids = ids or [str(uuid.uuid4()) for _ in contents]
start = time.time()
logger.debug(f"Generating embeddings for {len(contents)} contents...")
vectors = await self.embedding_provider.get_embeddings_batch(
contents,
batch_size=batch_size,
tasks_limit=tasks_limit,
max_retries=max_retries,
progress_callback=progress_callback,
)
end = time.time()
logger.debug(
f"Generated embeddings for {len(contents)} contents in {end - start:.2f} seconds.",
)
# 使用 DocumentStorage 的批量插入方法
int_ids = await self.document_storage.insert_documents_batch(
ids,
contents,
metadatas,
)
# 批量插入向量到 FAISS
vectors_array = np.array(vectors).astype("float32")
await self.embedding_storage.insert_batch(vectors_array, int_ids)
return int_ids
async def retrieve( async def retrieve(
self, self,
@@ -108,7 +65,8 @@ class FaissVecDB(BaseVecDB):
rerank: bool = False, rerank: bool = False,
metadata_filters: dict | None = None, metadata_filters: dict | None = None,
) -> list[Result]: ) -> list[Result]:
"""搜索最相似的文档。 """
搜索最相似的文档
Args: Args:
query (str): 查询文本 query (str): 查询文本
@@ -119,7 +77,6 @@ class FaissVecDB(BaseVecDB):
Returns: Returns:
List[Result]: 查询结果 List[Result]: 查询结果
""" """
embedding = await self.embedding_provider.get_embedding(query) embedding = await self.embedding_provider.get_embedding(query)
scores, indices = await self.embedding_storage.search( scores, indices = await self.embedding_storage.search(
@@ -132,8 +89,7 @@ class FaissVecDB(BaseVecDB):
scores[0] = 1.0 - (scores[0] / 2.0) scores[0] = 1.0 - (scores[0] / 2.0)
# NOTE: maybe the size is less than k. # NOTE: maybe the size is less than k.
fetched_docs = await self.document_storage.get_documents( fetched_docs = await self.document_storage.get_documents(
metadata_filters=metadata_filters or {}, metadata_filters=metadata_filters or {}, ids=indices[0]
ids=indices[0],
) )
if not fetched_docs: if not fetched_docs:
return [] return []
@@ -154,51 +110,31 @@ class FaissVecDB(BaseVecDB):
documents = [doc.data["text"] for doc in top_k_results] documents = [doc.data["text"] for doc in top_k_results]
reranked_results = await self.rerank_provider.rerank(query, documents) reranked_results = await self.rerank_provider.rerank(query, documents)
reranked_results = sorted( reranked_results = sorted(
reranked_results, reranked_results, key=lambda x: x.relevance_score, reverse=True
key=lambda x: x.relevance_score,
reverse=True,
) )
top_k_results = [ top_k_results = [
top_k_results[reranked_result.index] top_k_results[reranked_result.index] for reranked_result in reranked_results
for reranked_result in reranked_results
] ]
return top_k_results return top_k_results
async def delete(self, doc_id: str): async def delete(self, doc_id: int):
"""删除一条文档块(chunk""" """
# 获得对应的 int id 删除一条文档
result = await self.document_storage.get_document_by_doc_id(doc_id) """
int_id = result["id"] if result else None await self.document_storage.connection.execute(
if int_id is None: "DELETE FROM documents WHERE doc_id = ?", (doc_id,)
return )
await self.document_storage.connection.commit()
# 使用 DocumentStorage 的删除方法
await self.document_storage.delete_document_by_doc_id(doc_id)
await self.embedding_storage.delete([int_id])
async def close(self): async def close(self):
await self.document_storage.close() await self.document_storage.close()
async def count_documents(self, metadata_filter: dict | None = None) -> int: async def count_documents(self) -> int:
"""计算文档数量
Args:
metadata_filter (dict | None): 元数据过滤器
""" """
count = await self.document_storage.count_documents( 计算文档数量
metadata_filters=metadata_filter or {}, """
) async with self.document_storage.connection.cursor() as cursor:
return count await cursor.execute("SELECT COUNT(*) FROM documents")
count = await cursor.fetchone()
async def delete_documents(self, metadata_filters: dict): return count[0] if count else 0
"""根据元数据过滤器删除文档"""
docs = await self.document_storage.get_documents(
metadata_filters=metadata_filters,
offset=None,
limit=None,
)
doc_ids: list[int] = [doc["id"] for doc in docs]
await self.embedding_storage.delete(doc_ids)
await self.document_storage.delete_documents(metadata_filters=metadata_filters)
+7 -14
View File
@@ -1,4 +1,5 @@
"""事件总线, 用于处理事件的分发和处理 """
事件总线, 用于处理事件的分发和处理
事件总线是一个异步队列, 用于接收各种消息事件, 并将其发送到Scheduler调度器进行处理 事件总线是一个异步队列, 用于接收各种消息事件, 并将其发送到Scheduler调度器进行处理
其中包含了一个无限循环的调度函数, 用于从事件队列中获取新的事件, 并创建一个新的异步任务来执行管道调度器的处理逻辑 其中包含了一个无限循环的调度函数, 用于从事件队列中获取新的事件, 并创建一个新的异步任务来执行管道调度器的处理逻辑
@@ -12,12 +13,10 @@ class:
import asyncio import asyncio
from asyncio import Queue from asyncio import Queue
from astrbot.core import logger
from astrbot.core.astrbot_config_mgr import AstrBotConfigManager
from astrbot.core.pipeline.scheduler import PipelineScheduler from astrbot.core.pipeline.scheduler import PipelineScheduler
from astrbot.core import logger
from .platform import AstrMessageEvent from .platform import AstrMessageEvent
from astrbot.core.astrbot_config_mgr import AstrBotConfigManager
class EventBus: class EventBus:
@@ -27,7 +26,7 @@ class EventBus:
self, self,
event_queue: Queue, event_queue: Queue,
pipeline_scheduler_mapping: dict[str, PipelineScheduler], pipeline_scheduler_mapping: dict[str, PipelineScheduler],
astrbot_config_mgr: AstrBotConfigManager, astrbot_config_mgr: AstrBotConfigManager = None,
): ):
self.event_queue = event_queue # 事件队列 self.event_queue = event_queue # 事件队列
# abconf uuid -> scheduler # abconf uuid -> scheduler
@@ -40,11 +39,6 @@ class EventBus:
conf_info = self.astrbot_config_mgr.get_conf_info(event.unified_msg_origin) conf_info = self.astrbot_config_mgr.get_conf_info(event.unified_msg_origin)
self._print_event(event, conf_info["name"]) self._print_event(event, conf_info["name"])
scheduler = self.pipeline_scheduler_mapping.get(conf_info["id"]) scheduler = self.pipeline_scheduler_mapping.get(conf_info["id"])
if not scheduler:
logger.error(
f"PipelineScheduler not found for id: {conf_info['id']}, event ignored."
)
continue
asyncio.create_task(scheduler.execute(event)) asyncio.create_task(scheduler.execute(event))
def _print_event(self, event: AstrMessageEvent, conf_name: str): def _print_event(self, event: AstrMessageEvent, conf_name: str):
@@ -52,15 +46,14 @@ class EventBus:
Args: Args:
event (AstrMessageEvent): 事件对象 event (AstrMessageEvent): 事件对象
""" """
# 如果有发送者名称: [平台名] 发送者名称/发送者ID: 消息概要 # 如果有发送者名称: [平台名] 发送者名称/发送者ID: 消息概要
if event.get_sender_name(): if event.get_sender_name():
logger.info( logger.info(
f"[{conf_name}] [{event.get_platform_id()}({event.get_platform_name()})] {event.get_sender_name()}/{event.get_sender_id()}: {event.get_message_outline()}", f"[{conf_name}] [{event.get_platform_id()}({event.get_platform_name()})] {event.get_sender_name()}/{event.get_sender_id()}: {event.get_message_outline()}"
) )
# 没有发送者名称: [平台名] 发送者ID: 消息概要 # 没有发送者名称: [平台名] 发送者ID: 消息概要
else: else:
logger.info( logger.info(
f"[{conf_name}] [{event.get_platform_id()}({event.get_platform_name()})] {event.get_sender_id()}: {event.get_message_outline()}", f"[{conf_name}] [{event.get_platform_id()}({event.get_platform_name()})] {event.get_sender_id()}: {event.get_message_outline()}"
) )
-9
View File
@@ -1,9 +0,0 @@
from __future__ import annotations
class AstrBotError(Exception):
"""Base exception for all AstrBot errors."""
class ProviderNotFoundError(AstrBotError):
"""Raised when a specified provider is not found."""
+6 -12
View File
@@ -1,9 +1,9 @@
import asyncio import asyncio
import os import os
import platform
import time
import uuid import uuid
from urllib.parse import unquote, urlparse import time
from urllib.parse import urlparse, unquote
import platform
class FileTokenService: class FileTokenService:
@@ -23,12 +23,7 @@ class FileTokenService:
for token in expired_tokens: for token in expired_tokens:
self.staged_files.pop(token, None) self.staged_files.pop(token, None)
async def check_token_expired(self, file_token: str) -> bool: async def register_file(self, file_path: str, timeout: float = None) -> str:
async with self.lock:
await self._cleanup_expired_tokens()
return file_token not in self.staged_files
async def register_file(self, file_path: str, timeout: float | None = None) -> str:
"""向令牌服务注册一个文件。 """向令牌服务注册一个文件。
Args: Args:
@@ -40,8 +35,8 @@ class FileTokenService:
Raises: Raises:
FileNotFoundError: 当路径不存在时抛出 FileNotFoundError: 当路径不存在时抛出
""" """
# 处理 file:/// # 处理 file:///
try: try:
parsed_uri = urlparse(file_path) parsed_uri = urlparse(file_path)
@@ -61,7 +56,7 @@ class FileTokenService:
if not os.path.exists(local_path): if not os.path.exists(local_path):
raise FileNotFoundError( raise FileNotFoundError(
f"文件不存在: {local_path} (原始输入: {file_path})", f"文件不存在: {local_path} (原始输入: {file_path})"
) )
file_token = str(uuid.uuid4()) file_token = str(uuid.uuid4())
@@ -84,7 +79,6 @@ class FileTokenService:
Raises: Raises:
KeyError: 当令牌不存在或已过期时抛出 KeyError: 当令牌不存在或已过期时抛出
FileNotFoundError: 当文件本身已被删除时抛出 FileNotFoundError: 当文件本身已被删除时抛出
""" """
async with self.lock: async with self.lock:
await self._cleanup_expired_tokens() await self._cleanup_expired_tokens()
+8 -16
View File
@@ -1,4 +1,5 @@
"""AstrBot 启动器,负责初始化和启动核心组件和仪表板服务器。 """
AstrBot 启动器负责初始化和启动核心组件和仪表板服务器
工作流程: 工作流程:
1. 初始化核心生命周期, 传递数据库和日志代理实例到核心生命周期 1. 初始化核心生命周期, 传递数据库和日志代理实例到核心生命周期
@@ -7,10 +8,10 @@
import asyncio import asyncio
import traceback import traceback
from astrbot.core import logger
from astrbot.core import LogBroker, logger
from astrbot.core.core_lifecycle import AstrBotCoreLifecycle from astrbot.core.core_lifecycle import AstrBotCoreLifecycle
from astrbot.core.db import BaseDatabase from astrbot.core.db import BaseDatabase
from astrbot.core import LogBroker
from astrbot.dashboard.server import AstrBotDashboard from astrbot.dashboard.server import AstrBotDashboard
@@ -21,7 +22,6 @@ class InitialLoader:
self.db = db self.db = db
self.logger = logger self.logger = logger
self.log_broker = log_broker self.log_broker = log_broker
self.webui_dir: str | None = None
async def start(self): async def start(self):
core_lifecycle = AstrBotCoreLifecycle(self.log_broker, self.db) core_lifecycle = AstrBotCoreLifecycle(self.log_broker, self.db)
@@ -35,21 +35,13 @@ class InitialLoader:
core_task = core_lifecycle.start() core_task = core_lifecycle.start()
webui_dir = self.webui_dir
self.dashboard_server = AstrBotDashboard( self.dashboard_server = AstrBotDashboard(
core_lifecycle, core_lifecycle, self.db, core_lifecycle.dashboard_shutdown_event
self.db,
core_lifecycle.dashboard_shutdown_event,
webui_dir,
) )
task = asyncio.gather(
core_task, self.dashboard_server.run()
) # 启动核心任务和仪表板服务器
coro = self.dashboard_server.run()
if coro:
# 启动核心任务和仪表板服务器
task = asyncio.gather(core_task, coro)
else:
task = core_task
try: try:
await task # 整个AstrBot在这里运行 await task # 整个AstrBot在这里运行
except asyncio.CancelledError: except asyncio.CancelledError:
@@ -1,9 +0,0 @@
"""文档分块模块"""
from .base import BaseChunker
from .fixed_size import FixedSizeChunker
__all__ = [
"BaseChunker",
"FixedSizeChunker",
]
@@ -1,25 +0,0 @@
"""文档分块器基类
定义了文档分块处理的抽象接口
"""
from abc import ABC, abstractmethod
class BaseChunker(ABC):
"""分块器基类
所有分块器都应该继承此类并实现 chunk 方法
"""
@abstractmethod
async def chunk(self, text: str, **kwargs) -> list[str]:
"""将文本分块
Args:
text: 输入文本
Returns:
list[str]: 分块后的文本列表
"""
@@ -1,59 +0,0 @@
"""固定大小分块器
按照固定的字符数将文本分块,支持重叠区域
"""
from .base import BaseChunker
class FixedSizeChunker(BaseChunker):
"""固定大小分块器
按照固定的字符数分块,并支持块之间的重叠
"""
def __init__(self, chunk_size: int = 512, chunk_overlap: int = 50):
"""初始化分块器
Args:
chunk_size: 块的大小(字符数)
chunk_overlap: 块之间的重叠字符数
"""
self.chunk_size = chunk_size
self.chunk_overlap = chunk_overlap
async def chunk(self, text: str, **kwargs) -> list[str]:
"""固定大小分块
Args:
text: 输入文本
chunk_size: 每个文本块的最大大小
chunk_overlap: 每个文本块之间的重叠部分大小
Returns:
list[str]: 分块后的文本列表
"""
chunk_size = kwargs.get("chunk_size", self.chunk_size)
chunk_overlap = kwargs.get("chunk_overlap", self.chunk_overlap)
chunks = []
start = 0
text_len = len(text)
while start < text_len:
end = start + chunk_size
chunk = text[start:end]
if chunk:
chunks.append(chunk)
# 移动窗口,保留重叠部分
start = end - chunk_overlap
# 防止无限循环: 如果重叠过大,直接移到end
if start >= end or chunk_overlap >= chunk_size:
start = end
return chunks
@@ -1,161 +0,0 @@
from collections.abc import Callable
from .base import BaseChunker
class RecursiveCharacterChunker(BaseChunker):
def __init__(
self,
chunk_size: int = 500,
chunk_overlap: int = 100,
length_function: Callable[[str], int] = len,
is_separator_regex: bool = False,
separators: list[str] | None = None,
):
"""初始化递归字符文本分割器
Args:
chunk_size: 每个文本块的最大大小
chunk_overlap: 每个文本块之间的重叠部分大小
length_function: 计算文本长度的函数
is_separator_regex: 分隔符是否为正则表达式
separators: 用于分割文本的分隔符列表按优先级排序
"""
self.chunk_size = chunk_size
self.chunk_overlap = chunk_overlap
self.length_function = length_function
self.is_separator_regex = is_separator_regex
# 默认分隔符列表,按优先级从高到低
self.separators = separators or [
"\n\n", # 段落
"\n", # 换行
"", # 中文句子
"", # 中文逗号
". ", # 句子
", ", # 逗号分隔
" ", # 单词
"", # 字符
]
async def chunk(self, text: str, **kwargs) -> list[str]:
"""递归地将文本分割成块
Args:
text: 要分割的文本
chunk_size: 每个文本块的最大大小
chunk_overlap: 每个文本块之间的重叠部分大小
Returns:
分割后的文本块列表
"""
if not text:
return []
overlap = kwargs.get("chunk_overlap", self.chunk_overlap)
chunk_size = kwargs.get("chunk_size", self.chunk_size)
text_length = self.length_function(text)
if text_length <= chunk_size:
return [text]
for separator in self.separators:
if separator == "":
return self._split_by_character(text, chunk_size, overlap)
if separator in text:
splits = text.split(separator)
# 重新添加分隔符(除了最后一个片段)
splits = [s + separator for s in splits[:-1]] + [splits[-1]]
splits = [s for s in splits if s]
if len(splits) == 1:
continue
# 递归合并分割后的文本块
final_chunks = []
current_chunk = []
current_chunk_length = 0
for split in splits:
split_length = self.length_function(split)
# 如果单个分割部分已经超过了chunk_size,需要递归分割
if split_length > chunk_size:
# 先处理当前积累的块
if current_chunk:
combined_text = "".join(current_chunk)
final_chunks.extend(
await self.chunk(
combined_text,
chunk_size=chunk_size,
chunk_overlap=overlap,
),
)
current_chunk = []
current_chunk_length = 0
# 递归分割过大的部分
final_chunks.extend(
await self.chunk(
split,
chunk_size=chunk_size,
chunk_overlap=overlap,
),
)
# 如果添加这部分会使当前块超过chunk_size
elif current_chunk_length + split_length > chunk_size:
# 合并当前块并添加到结果中
combined_text = "".join(current_chunk)
final_chunks.append(combined_text)
# 处理重叠部分
overlap_start = max(0, len(combined_text) - overlap)
if overlap_start > 0:
overlap_text = combined_text[overlap_start:]
current_chunk = [overlap_text, split]
current_chunk_length = (
self.length_function(overlap_text) + split_length
)
else:
current_chunk = [split]
current_chunk_length = split_length
else:
# 添加到当前块
current_chunk.append(split)
current_chunk_length += split_length
# 处理剩余的块
if current_chunk:
final_chunks.append("".join(current_chunk))
return final_chunks
return [text]
def _split_by_character(
self,
text: str,
chunk_size: int | None = None,
overlap: int | None = None,
) -> list[str]:
"""按字符级别分割文本
Args:
text: 要分割的文本
Returns:
分割后的文本块列表
"""
chunk_size = chunk_size or self.chunk_size
overlap = overlap or self.chunk_overlap
result = []
for i in range(0, len(text), chunk_size - overlap):
end = min(i + chunk_size, len(text))
result.append(text[i:end])
if end == len(text):
break
return result
-301
View File
@@ -1,301 +0,0 @@
from contextlib import asynccontextmanager
from pathlib import Path
from sqlalchemy import delete, func, select, text, update
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from sqlmodel import col, desc
from astrbot.core import logger
from astrbot.core.db.vec_db.faiss_impl import FaissVecDB
from astrbot.core.knowledge_base.models import (
BaseKBModel,
KBDocument,
KBMedia,
KnowledgeBase,
)
class KBSQLiteDatabase:
def __init__(self, db_path: str = "data/knowledge_base/kb.db") -> None:
"""初始化知识库数据库
Args:
db_path: 数据库文件路径, 默认为 data/knowledge_base/kb.db
"""
self.db_path = db_path
self.DATABASE_URL = f"sqlite+aiosqlite:///{db_path}"
self.inited = False
# 确保目录存在
Path(db_path).parent.mkdir(parents=True, exist_ok=True)
# 创建异步引擎
self.engine = create_async_engine(
self.DATABASE_URL,
echo=False,
pool_pre_ping=True,
pool_recycle=3600,
)
# 创建会话工厂
self.async_session = async_sessionmaker(
self.engine,
class_=AsyncSession,
expire_on_commit=False,
)
@asynccontextmanager
async def get_db(self):
"""获取数据库会话
用法:
async with kb_db.get_db() as session:
# 执行数据库操作
result = await session.execute(stmt)
"""
async with self.async_session() as session:
yield session
async def initialize(self) -> None:
"""初始化数据库,创建表并配置 SQLite 参数"""
async with self.engine.begin() as conn:
# 创建所有知识库相关表
await conn.run_sync(BaseKBModel.metadata.create_all)
# 配置 SQLite 性能优化参数
await conn.execute(text("PRAGMA journal_mode=WAL"))
await conn.execute(text("PRAGMA synchronous=NORMAL"))
await conn.execute(text("PRAGMA cache_size=20000"))
await conn.execute(text("PRAGMA temp_store=MEMORY"))
await conn.execute(text("PRAGMA mmap_size=134217728"))
await conn.execute(text("PRAGMA optimize"))
await conn.commit()
self.inited = True
async def migrate_to_v1(self) -> None:
"""执行知识库数据库 v1 迁移
创建所有必要的索引以优化查询性能
"""
async with self.get_db() as session:
session: AsyncSession
async with session.begin():
# 创建知识库表索引
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_kb_kb_id "
"ON knowledge_bases(kb_id)",
),
)
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_kb_name "
"ON knowledge_bases(kb_name)",
),
)
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_kb_created_at "
"ON knowledge_bases(created_at)",
),
)
# 创建文档表索引
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_doc_doc_id "
"ON kb_documents(doc_id)",
),
)
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_doc_kb_id "
"ON kb_documents(kb_id)",
),
)
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_doc_name "
"ON kb_documents(doc_name)",
),
)
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_doc_type "
"ON kb_documents(file_type)",
),
)
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_doc_created_at "
"ON kb_documents(created_at)",
),
)
# 创建多媒体表索引
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_media_media_id "
"ON kb_media(media_id)",
),
)
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_media_doc_id "
"ON kb_media(doc_id)",
),
)
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_media_kb_id ON kb_media(kb_id)",
),
)
await session.execute(
text(
"CREATE INDEX IF NOT EXISTS idx_media_type "
"ON kb_media(media_type)",
),
)
await session.commit()
async def close(self) -> None:
"""关闭数据库连接"""
await self.engine.dispose()
logger.info(f"知识库数据库已关闭: {self.db_path}")
async def get_kb_by_id(self, kb_id: str) -> KnowledgeBase | None:
"""根据 ID 获取知识库"""
async with self.get_db() as session:
stmt = select(KnowledgeBase).where(col(KnowledgeBase.kb_id) == kb_id)
result = await session.execute(stmt)
return result.scalar_one_or_none()
async def get_kb_by_name(self, kb_name: str) -> KnowledgeBase | None:
"""根据名称获取知识库"""
async with self.get_db() as session:
stmt = select(KnowledgeBase).where(col(KnowledgeBase.kb_name) == kb_name)
result = await session.execute(stmt)
return result.scalar_one_or_none()
async def list_kbs(self, offset: int = 0, limit: int = 100) -> list[KnowledgeBase]:
"""列出所有知识库"""
async with self.get_db() as session:
stmt = (
select(KnowledgeBase)
.offset(offset)
.limit(limit)
.order_by(desc(KnowledgeBase.created_at))
)
result = await session.execute(stmt)
return list(result.scalars().all())
async def count_kbs(self) -> int:
"""统计知识库数量"""
async with self.get_db() as session:
stmt = select(func.count(col(KnowledgeBase.id)))
result = await session.execute(stmt)
return result.scalar() or 0
# ===== 文档查询 =====
async def get_document_by_id(self, doc_id: str) -> KBDocument | None:
"""根据 ID 获取文档"""
async with self.get_db() as session:
stmt = select(KBDocument).where(col(KBDocument.doc_id) == doc_id)
result = await session.execute(stmt)
return result.scalar_one_or_none()
async def list_documents_by_kb(
self,
kb_id: str,
offset: int = 0,
limit: int = 100,
) -> list[KBDocument]:
"""列出知识库的所有文档"""
async with self.get_db() as session:
stmt = (
select(KBDocument)
.where(col(KBDocument.kb_id) == kb_id)
.offset(offset)
.limit(limit)
.order_by(desc(KBDocument.created_at))
)
result = await session.execute(stmt)
return list(result.scalars().all())
async def count_documents_by_kb(self, kb_id: str) -> int:
"""统计知识库的文档数量"""
async with self.get_db() as session:
stmt = select(func.count(col(KBDocument.id))).where(
col(KBDocument.kb_id) == kb_id,
)
result = await session.execute(stmt)
return result.scalar() or 0
async def get_document_with_metadata(self, doc_id: str) -> dict | None:
async with self.get_db() as session:
stmt = (
select(KBDocument, KnowledgeBase)
.join(KnowledgeBase, col(KBDocument.kb_id) == col(KnowledgeBase.kb_id))
.where(col(KBDocument.doc_id) == doc_id)
)
result = await session.execute(stmt)
row = result.first()
if not row:
return None
return {
"document": row[0],
"knowledge_base": row[1],
}
async def delete_document_by_id(self, doc_id: str, vec_db: FaissVecDB):
"""删除单个文档及其相关数据"""
# 在知识库表中删除
async with self.get_db() as session, session.begin():
# 删除文档记录
delete_stmt = delete(KBDocument).where(col(KBDocument.doc_id) == doc_id)
await session.execute(delete_stmt)
await session.commit()
# 在 vec db 中删除相关向量
await vec_db.delete_documents(metadata_filters={"kb_doc_id": doc_id})
# ===== 多媒体查询 =====
async def list_media_by_doc(self, doc_id: str) -> list[KBMedia]:
"""列出文档的所有多媒体资源"""
async with self.get_db() as session:
stmt = select(KBMedia).where(col(KBMedia.doc_id) == doc_id)
result = await session.execute(stmt)
return list(result.scalars().all())
async def get_media_by_id(self, media_id: str) -> KBMedia | None:
"""根据 ID 获取多媒体资源"""
async with self.get_db() as session:
stmt = select(KBMedia).where(col(KBMedia.media_id) == media_id)
result = await session.execute(stmt)
return result.scalar_one_or_none()
async def update_kb_stats(self, kb_id: str, vec_db: FaissVecDB) -> None:
"""更新知识库统计信息"""
chunk_cnt = await vec_db.count_documents()
async with self.get_db() as session, session.begin():
update_stmt = (
update(KnowledgeBase)
.where(col(KnowledgeBase.kb_id) == kb_id)
.values(
doc_count=select(func.count(col(KBDocument.id)))
.where(col(KBDocument.kb_id) == kb_id)
.scalar_subquery(),
chunk_count=chunk_cnt,
)
)
await session.execute(update_stmt)
await session.commit()
-642
View File
@@ -1,642 +0,0 @@
import asyncio
import json
import re
import time
import uuid
from pathlib import Path
import aiofiles
from astrbot.core import logger
from astrbot.core.db.vec_db.base import BaseVecDB
from astrbot.core.db.vec_db.faiss_impl.vec_db import FaissVecDB
from astrbot.core.provider.manager import ProviderManager
from astrbot.core.provider.provider import (
EmbeddingProvider,
RerankProvider,
)
from astrbot.core.provider.provider import (
Provider as LLMProvider,
)
from .chunking.base import BaseChunker
from .chunking.recursive import RecursiveCharacterChunker
from .kb_db_sqlite import KBSQLiteDatabase
from .models import KBDocument, KBMedia, KnowledgeBase
from .parsers.url_parser import extract_text_from_url
from .parsers.util import select_parser
from .prompts import TEXT_REPAIR_SYSTEM_PROMPT
class RateLimiter:
"""一个简单的速率限制器"""
def __init__(self, max_rpm: int):
self.max_per_minute = max_rpm
self.interval = 60.0 / max_rpm if max_rpm > 0 else 0
self.last_call_time = 0
async def __aenter__(self):
if self.interval == 0:
return
now = time.monotonic()
elapsed = now - self.last_call_time
if elapsed < self.interval:
await asyncio.sleep(self.interval - elapsed)
self.last_call_time = time.monotonic()
async def __aexit__(self, exc_type, exc_val, exc_tb):
pass
async def _repair_and_translate_chunk_with_retry(
chunk: str,
repair_llm_service: LLMProvider,
rate_limiter: RateLimiter,
max_retries: int = 2,
) -> list[str]:
"""
Repairs, translates, and optionally re-chunks a single text chunk using the small LLM, with rate limiting.
"""
# 为了防止 LLM 上下文污染,在 user_prompt 中也加入明确的指令
user_prompt = f"""IGNORE ALL PREVIOUS INSTRUCTIONS. Your ONLY task is to process the following text chunk according to the system prompt provided.
Text chunk to process:
---
{chunk}
---
"""
for attempt in range(max_retries + 1):
try:
async with rate_limiter:
response = await repair_llm_service.text_chat(
prompt=user_prompt, system_prompt=TEXT_REPAIR_SYSTEM_PROMPT
)
llm_output = response.completion_text
if "<discard_chunk />" in llm_output:
return [] # Signal to discard this chunk
# More robust regex to handle potential LLM formatting errors (spaces, newlines in tags)
matches = re.findall(
r"<\s*repaired_text\s*>\s*(.*?)\s*<\s*/\s*repaired_text\s*>",
llm_output,
re.DOTALL,
)
if matches:
# Further cleaning to ensure no empty strings are returned
return [m.strip() for m in matches if m.strip()]
else:
# If no valid tags and not explicitly discarded, discard it to be safe.
return []
except Exception as e:
logger.warning(
f" - LLM call failed on attempt {attempt + 1}/{max_retries + 1}. Error: {str(e)}"
)
logger.error(
f" - Failed to process chunk after {max_retries + 1} attempts. Using original text."
)
return [chunk]
class KBHelper:
vec_db: BaseVecDB
kb: KnowledgeBase
def __init__(
self,
kb_db: KBSQLiteDatabase,
kb: KnowledgeBase,
provider_manager: ProviderManager,
kb_root_dir: str,
chunker: BaseChunker,
):
self.kb_db = kb_db
self.kb = kb
self.prov_mgr = provider_manager
self.kb_root_dir = kb_root_dir
self.chunker = chunker
self.kb_dir = Path(self.kb_root_dir) / self.kb.kb_id
self.kb_medias_dir = Path(self.kb_dir) / "medias" / self.kb.kb_id
self.kb_files_dir = Path(self.kb_dir) / "files" / self.kb.kb_id
self.kb_medias_dir.mkdir(parents=True, exist_ok=True)
self.kb_files_dir.mkdir(parents=True, exist_ok=True)
async def initialize(self):
await self._ensure_vec_db()
async def get_ep(self) -> EmbeddingProvider:
if not self.kb.embedding_provider_id:
raise ValueError(f"知识库 {self.kb.kb_name} 未配置 Embedding Provider")
ep: EmbeddingProvider = await self.prov_mgr.get_provider_by_id(
self.kb.embedding_provider_id,
) # type: ignore
if not ep:
raise ValueError(
f"无法找到 ID 为 {self.kb.embedding_provider_id} 的 Embedding Provider",
)
return ep
async def get_rp(self) -> RerankProvider | None:
if not self.kb.rerank_provider_id:
return None
rp: RerankProvider = await self.prov_mgr.get_provider_by_id(
self.kb.rerank_provider_id,
) # type: ignore
if not rp:
raise ValueError(
f"无法找到 ID 为 {self.kb.rerank_provider_id} 的 Rerank Provider",
)
return rp
async def _ensure_vec_db(self) -> FaissVecDB:
if not self.kb.embedding_provider_id:
raise ValueError(f"知识库 {self.kb.kb_name} 未配置 Embedding Provider")
ep = await self.get_ep()
rp = await self.get_rp()
vec_db = FaissVecDB(
doc_store_path=str(self.kb_dir / "doc.db"),
index_store_path=str(self.kb_dir / "index.faiss"),
embedding_provider=ep,
rerank_provider=rp,
)
await vec_db.initialize()
self.vec_db = vec_db
return vec_db
async def delete_vec_db(self):
"""删除知识库的向量数据库和所有相关文件"""
import shutil
await self.terminate()
if self.kb_dir.exists():
shutil.rmtree(self.kb_dir)
async def terminate(self):
if self.vec_db:
await self.vec_db.close()
async def upload_document(
self,
file_name: str,
file_content: bytes | None,
file_type: str,
chunk_size: int = 512,
chunk_overlap: int = 50,
batch_size: int = 32,
tasks_limit: int = 3,
max_retries: int = 3,
progress_callback=None,
pre_chunked_text: list[str] | None = None,
) -> KBDocument:
"""上传并处理文档(带原子性保证和失败清理)
流程:
1. 保存原始文件
2. 解析文档内容
3. 提取多媒体资源
4. 分块处理
5. 生成向量并存储
6. 保存元数据事务
7. 更新统计
Args:
progress_callback: 进度回调函数接收参数 (stage, current, total)
- stage: 当前阶段 ('parsing', 'chunking', 'embedding')
- current: 当前进度
- total: 总数
"""
await self._ensure_vec_db()
doc_id = str(uuid.uuid4())
media_paths: list[Path] = []
file_size = 0
# file_path = self.kb_files_dir / f"{doc_id}.{file_type}"
# async with aiofiles.open(file_path, "wb") as f:
# await f.write(file_content)
try:
chunks_text = []
saved_media = []
if pre_chunked_text is not None:
# 如果提供了预分块文本,直接使用
chunks_text = pre_chunked_text
file_size = sum(len(chunk) for chunk in chunks_text)
logger.info(f"使用预分块文本进行上传,共 {len(chunks_text)} 个块。")
else:
# 否则,执行标准的文件解析和分块流程
if file_content is None:
raise ValueError(
"当未提供 pre_chunked_text 时,file_content 不能为空。"
)
file_size = len(file_content)
# 阶段1: 解析文档
if progress_callback:
await progress_callback("parsing", 0, 100)
parser = await select_parser(f".{file_type}")
parse_result = await parser.parse(file_content, file_name)
text_content = parse_result.text
media_items = parse_result.media
if progress_callback:
await progress_callback("parsing", 100, 100)
# 保存媒体文件
for media_item in media_items:
media = await self._save_media(
doc_id=doc_id,
media_type=media_item.media_type,
file_name=media_item.file_name,
content=media_item.content,
mime_type=media_item.mime_type,
)
saved_media.append(media)
media_paths.append(Path(media.file_path))
# 阶段2: 分块
if progress_callback:
await progress_callback("chunking", 0, 100)
chunks_text = await self.chunker.chunk(
text_content,
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
)
contents = []
metadatas = []
for idx, chunk_text in enumerate(chunks_text):
contents.append(chunk_text)
metadatas.append(
{
"kb_id": self.kb.kb_id,
"kb_doc_id": doc_id,
"chunk_index": idx,
},
)
if progress_callback:
await progress_callback("chunking", 100, 100)
# 阶段3: 生成向量(带进度回调)
async def embedding_progress_callback(current, total):
if progress_callback:
await progress_callback("embedding", current, total)
await self.vec_db.insert_batch(
contents=contents,
metadatas=metadatas,
batch_size=batch_size,
tasks_limit=tasks_limit,
max_retries=max_retries,
progress_callback=embedding_progress_callback,
)
# 保存文档的元数据
doc = KBDocument(
doc_id=doc_id,
kb_id=self.kb.kb_id,
doc_name=file_name,
file_type=file_type,
file_size=file_size,
# file_path=str(file_path),
file_path="",
chunk_count=len(chunks_text),
media_count=0,
)
async with self.kb_db.get_db() as session:
async with session.begin():
session.add(doc)
for media in saved_media:
session.add(media)
await session.commit()
await session.refresh(doc)
vec_db: FaissVecDB = self.vec_db # type: ignore
await self.kb_db.update_kb_stats(kb_id=self.kb.kb_id, vec_db=vec_db)
await self.refresh_kb()
await self.refresh_document(doc_id)
return doc
except Exception as e:
logger.error(f"上传文档失败: {e}")
# if file_path.exists():
# file_path.unlink()
for media_path in media_paths:
try:
if media_path.exists():
media_path.unlink()
except Exception as me:
logger.warning(f"清理多媒体文件失败 {media_path}: {me}")
raise e
async def list_documents(
self,
offset: int = 0,
limit: int = 100,
) -> list[KBDocument]:
"""列出知识库的所有文档"""
docs = await self.kb_db.list_documents_by_kb(self.kb.kb_id, offset, limit)
return docs
async def get_document(self, doc_id: str) -> KBDocument | None:
"""获取单个文档"""
doc = await self.kb_db.get_document_by_id(doc_id)
return doc
async def delete_document(self, doc_id: str):
"""删除单个文档及其相关数据"""
await self.kb_db.delete_document_by_id(
doc_id=doc_id,
vec_db=self.vec_db, # type: ignore
)
await self.kb_db.update_kb_stats(
kb_id=self.kb.kb_id,
vec_db=self.vec_db, # type: ignore
)
await self.refresh_kb()
async def delete_chunk(self, chunk_id: str, doc_id: str):
"""删除单个文本块及其相关数据"""
vec_db: FaissVecDB = self.vec_db # type: ignore
await vec_db.delete(chunk_id)
await self.kb_db.update_kb_stats(
kb_id=self.kb.kb_id,
vec_db=self.vec_db, # type: ignore
)
await self.refresh_kb()
await self.refresh_document(doc_id)
async def refresh_kb(self):
if self.kb:
kb = await self.kb_db.get_kb_by_id(self.kb.kb_id)
if kb:
self.kb = kb
async def refresh_document(self, doc_id: str) -> None:
"""更新文档的元数据"""
doc = await self.get_document(doc_id)
if not doc:
raise ValueError(f"无法找到 ID 为 {doc_id} 的文档")
chunk_count = await self.get_chunk_count_by_doc_id(doc_id)
doc.chunk_count = chunk_count
async with self.kb_db.get_db() as session:
async with session.begin():
session.add(doc)
await session.commit()
await session.refresh(doc)
async def get_chunks_by_doc_id(
self,
doc_id: str,
offset: int = 0,
limit: int = 100,
) -> list[dict]:
"""获取文档的所有块及其元数据"""
vec_db: FaissVecDB = self.vec_db # type: ignore
chunks = await vec_db.document_storage.get_documents(
metadata_filters={"kb_doc_id": doc_id},
offset=offset,
limit=limit,
)
result = []
for chunk in chunks:
chunk_md = json.loads(chunk["metadata"])
result.append(
{
"chunk_id": chunk["doc_id"],
"doc_id": chunk_md["kb_doc_id"],
"kb_id": chunk_md["kb_id"],
"chunk_index": chunk_md["chunk_index"],
"content": chunk["text"],
"char_count": len(chunk["text"]),
},
)
return result
async def get_chunk_count_by_doc_id(self, doc_id: str) -> int:
"""获取文档的块数量"""
vec_db: FaissVecDB = self.vec_db # type: ignore
count = await vec_db.count_documents(metadata_filter={"kb_doc_id": doc_id})
return count
async def _save_media(
self,
doc_id: str,
media_type: str,
file_name: str,
content: bytes,
mime_type: str,
) -> KBMedia:
"""保存多媒体资源"""
media_id = str(uuid.uuid4())
ext = Path(file_name).suffix
# 保存文件
file_path = self.kb_medias_dir / doc_id / f"{media_id}{ext}"
file_path.parent.mkdir(parents=True, exist_ok=True)
async with aiofiles.open(file_path, "wb") as f:
await f.write(content)
media = KBMedia(
media_id=media_id,
doc_id=doc_id,
kb_id=self.kb.kb_id,
media_type=media_type,
file_name=file_name,
file_path=str(file_path),
file_size=len(content),
mime_type=mime_type,
)
return media
async def upload_from_url(
self,
url: str,
chunk_size: int = 512,
chunk_overlap: int = 50,
batch_size: int = 32,
tasks_limit: int = 3,
max_retries: int = 3,
progress_callback=None,
enable_cleaning: bool = False,
cleaning_provider_id: str | None = None,
) -> KBDocument:
"""从 URL 上传并处理文档(带原子性保证和失败清理)
Args:
url: 要提取内容的网页 URL
chunk_size: 文本块大小
chunk_overlap: 文本块重叠大小
batch_size: 批处理大小
tasks_limit: 并发任务限制
max_retries: 最大重试次数
progress_callback: 进度回调函数接收参数 (stage, current, total)
- stage: 当前阶段 ('extracting', 'cleaning', 'parsing', 'chunking', 'embedding')
- current: 当前进度
- total: 总数
Returns:
KBDocument: 上传的文档对象
Raises:
ValueError: 如果 URL 为空或无法提取内容
IOError: 如果网络请求失败
"""
# 获取 Tavily API 密钥
config = self.prov_mgr.acm.default_conf
tavily_keys = config.get("provider_settings", {}).get(
"websearch_tavily_key", []
)
if not tavily_keys:
raise ValueError(
"Error: Tavily API key is not configured in provider_settings."
)
# 阶段1: 从 URL 提取内容
if progress_callback:
await progress_callback("extracting", 0, 100)
try:
text_content = await extract_text_from_url(url, tavily_keys)
except Exception as e:
logger.error(f"Failed to extract content from URL {url}: {e}")
raise OSError(f"Failed to extract content from URL {url}: {e}") from e
if not text_content:
raise ValueError(f"No content extracted from URL: {url}")
if progress_callback:
await progress_callback("extracting", 100, 100)
# 阶段2: (可选)清洗内容并分块
final_chunks = await self._clean_and_rechunk_content(
content=text_content,
url=url,
progress_callback=progress_callback,
enable_cleaning=enable_cleaning,
cleaning_provider_id=cleaning_provider_id,
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
)
if enable_cleaning and not final_chunks:
raise ValueError(
"内容清洗后未提取到有效文本。请尝试关闭内容清洗功能,或更换更高性能的LLM模型后重试。"
)
# 创建一个虚拟文件名
file_name = url.split("/")[-1] or f"document_from_{url}"
if not Path(file_name).suffix:
file_name += ".url"
# 复用现有的 upload_document 方法,但传入预分块文本
return await self.upload_document(
file_name=file_name,
file_content=None,
file_type="url", # 使用 'url' 作为特殊文件类型
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
batch_size=batch_size,
tasks_limit=tasks_limit,
max_retries=max_retries,
progress_callback=progress_callback,
pre_chunked_text=final_chunks,
)
async def _clean_and_rechunk_content(
self,
content: str,
url: str,
progress_callback=None,
enable_cleaning: bool = False,
cleaning_provider_id: str | None = None,
repair_max_rpm: int = 60,
chunk_size: int = 512,
chunk_overlap: int = 50,
) -> list[str]:
"""
对从 URL 获取的内容进行清洗修复翻译和重新分块
"""
if not enable_cleaning:
# 如果不启用清洗,则使用从前端传递的参数进行分块
logger.info(
f"内容清洗未启用,使用指定参数进行分块: chunk_size={chunk_size}, chunk_overlap={chunk_overlap}"
)
return await self.chunker.chunk(
content, chunk_size=chunk_size, chunk_overlap=chunk_overlap
)
if not cleaning_provider_id:
logger.warning(
"启用了内容清洗,但未提供 cleaning_provider_id,跳过清洗并使用默认分块。"
)
return await self.chunker.chunk(content)
if progress_callback:
await progress_callback("cleaning", 0, 100)
try:
# 获取指定的 LLM Provider
llm_provider = await self.prov_mgr.get_provider_by_id(cleaning_provider_id)
if not llm_provider or not isinstance(llm_provider, LLMProvider):
raise ValueError(
f"无法找到 ID 为 {cleaning_provider_id} 的 LLM Provider 或类型不正确"
)
# 初步分块
# 优化分隔符,优先按段落分割,以获得更高质量的文本块
text_splitter = RecursiveCharacterChunker(
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
separators=["\n\n", "\n", " "], # 优先使用段落分隔符
)
initial_chunks = await text_splitter.chunk(content)
logger.info(f"初步分块完成,生成 {len(initial_chunks)} 个块用于修复。")
# 并发处理所有块
rate_limiter = RateLimiter(repair_max_rpm)
tasks = [
_repair_and_translate_chunk_with_retry(
chunk, llm_provider, rate_limiter
)
for chunk in initial_chunks
]
repaired_results = await asyncio.gather(*tasks, return_exceptions=True)
final_chunks = []
for i, result in enumerate(repaired_results):
if isinstance(result, Exception):
logger.warning(f"{i} 处理异常: {str(result)}. 回退到原始块。")
final_chunks.append(initial_chunks[i])
elif isinstance(result, list):
final_chunks.extend(result)
logger.info(
f"文本修复完成: {len(initial_chunks)} 个原始块 -> {len(final_chunks)} 个最终块。"
)
if progress_callback:
await progress_callback("cleaning", 100, 100)
return final_chunks
except Exception as e:
logger.error(f"使用 Provider '{cleaning_provider_id}' 清洗内容失败: {e}")
# 清洗失败,返回默认分块结果,保证流程不中断
return await self.chunker.chunk(content)
-330
View File
@@ -1,330 +0,0 @@
import traceback
from pathlib import Path
from astrbot.core import logger
from astrbot.core.provider.manager import ProviderManager
# from .chunking.fixed_size import FixedSizeChunker
from .chunking.recursive import RecursiveCharacterChunker
from .kb_db_sqlite import KBSQLiteDatabase
from .kb_helper import KBHelper
from .models import KBDocument, KnowledgeBase
from .retrieval.manager import RetrievalManager, RetrievalResult
from .retrieval.rank_fusion import RankFusion
from .retrieval.sparse_retriever import SparseRetriever
FILES_PATH = "data/knowledge_base"
DB_PATH = Path(FILES_PATH) / "kb.db"
"""Knowledge Base storage root directory"""
CHUNKER = RecursiveCharacterChunker()
class KnowledgeBaseManager:
kb_db: KBSQLiteDatabase
retrieval_manager: RetrievalManager
def __init__(
self,
provider_manager: ProviderManager,
):
Path(DB_PATH).parent.mkdir(parents=True, exist_ok=True)
self.provider_manager = provider_manager
self._session_deleted_callback_registered = False
self.kb_insts: dict[str, KBHelper] = {}
async def initialize(self):
"""初始化知识库模块"""
try:
logger.info("正在初始化知识库模块...")
# 初始化数据库
await self._init_kb_database()
# 初始化检索管理器
sparse_retriever = SparseRetriever(self.kb_db)
rank_fusion = RankFusion(self.kb_db)
self.retrieval_manager = RetrievalManager(
sparse_retriever=sparse_retriever,
rank_fusion=rank_fusion,
kb_db=self.kb_db,
)
await self.load_kbs()
except ImportError as e:
logger.error(f"知识库模块导入失败: {e}")
logger.warning("请确保已安装所需依赖: pypdf, aiofiles, Pillow, rank-bm25")
except Exception as e:
logger.error(f"知识库模块初始化失败: {e}")
logger.error(traceback.format_exc())
async def _init_kb_database(self):
self.kb_db = KBSQLiteDatabase(DB_PATH.as_posix())
await self.kb_db.initialize()
await self.kb_db.migrate_to_v1()
logger.info(f"KnowledgeBase database initialized: {DB_PATH}")
async def load_kbs(self):
"""加载所有知识库实例"""
kb_records = await self.kb_db.list_kbs()
for record in kb_records:
kb_helper = KBHelper(
kb_db=self.kb_db,
kb=record,
provider_manager=self.provider_manager,
kb_root_dir=FILES_PATH,
chunker=CHUNKER,
)
await kb_helper.initialize()
self.kb_insts[record.kb_id] = kb_helper
async def create_kb(
self,
kb_name: str,
description: str | None = None,
emoji: str | None = None,
embedding_provider_id: str | None = None,
rerank_provider_id: str | None = None,
chunk_size: int | None = None,
chunk_overlap: int | None = None,
top_k_dense: int | None = None,
top_k_sparse: int | None = None,
top_m_final: int | None = None,
) -> KBHelper:
"""创建新的知识库实例"""
kb = KnowledgeBase(
kb_name=kb_name,
description=description,
emoji=emoji or "📚",
embedding_provider_id=embedding_provider_id,
rerank_provider_id=rerank_provider_id,
chunk_size=chunk_size if chunk_size is not None else 512,
chunk_overlap=chunk_overlap if chunk_overlap is not None else 50,
top_k_dense=top_k_dense if top_k_dense is not None else 50,
top_k_sparse=top_k_sparse if top_k_sparse is not None else 50,
top_m_final=top_m_final if top_m_final is not None else 5,
)
async with self.kb_db.get_db() as session:
session.add(kb)
await session.commit()
await session.refresh(kb)
kb_helper = KBHelper(
kb_db=self.kb_db,
kb=kb,
provider_manager=self.provider_manager,
kb_root_dir=FILES_PATH,
chunker=CHUNKER,
)
await kb_helper.initialize()
self.kb_insts[kb.kb_id] = kb_helper
return kb_helper
async def get_kb(self, kb_id: str) -> KBHelper | None:
"""获取知识库实例"""
if kb_id in self.kb_insts:
return self.kb_insts[kb_id]
async def get_kb_by_name(self, kb_name: str) -> KBHelper | None:
"""通过名称获取知识库实例"""
for kb_helper in self.kb_insts.values():
if kb_helper.kb.kb_name == kb_name:
return kb_helper
return None
async def delete_kb(self, kb_id: str) -> bool:
"""删除知识库实例"""
kb_helper = await self.get_kb(kb_id)
if not kb_helper:
return False
await kb_helper.delete_vec_db()
async with self.kb_db.get_db() as session:
await session.delete(kb_helper.kb)
await session.commit()
self.kb_insts.pop(kb_id, None)
return True
async def list_kbs(self) -> list[KnowledgeBase]:
"""列出所有知识库实例"""
kbs = [kb_helper.kb for kb_helper in self.kb_insts.values()]
return kbs
async def update_kb(
self,
kb_id: str,
kb_name: str,
description: str | None = None,
emoji: str | None = None,
embedding_provider_id: str | None = None,
rerank_provider_id: str | None = None,
chunk_size: int | None = None,
chunk_overlap: int | None = None,
top_k_dense: int | None = None,
top_k_sparse: int | None = None,
top_m_final: int | None = None,
) -> KBHelper | None:
"""更新知识库实例"""
kb_helper = await self.get_kb(kb_id)
if not kb_helper:
return None
kb = kb_helper.kb
if kb_name is not None:
kb.kb_name = kb_name
if description is not None:
kb.description = description
if emoji is not None:
kb.emoji = emoji
if embedding_provider_id is not None:
kb.embedding_provider_id = embedding_provider_id
kb.rerank_provider_id = rerank_provider_id # 允许设置为 None
if chunk_size is not None:
kb.chunk_size = chunk_size
if chunk_overlap is not None:
kb.chunk_overlap = chunk_overlap
if top_k_dense is not None:
kb.top_k_dense = top_k_dense
if top_k_sparse is not None:
kb.top_k_sparse = top_k_sparse
if top_m_final is not None:
kb.top_m_final = top_m_final
async with self.kb_db.get_db() as session:
session.add(kb)
await session.commit()
await session.refresh(kb)
return kb_helper
async def retrieve(
self,
query: str,
kb_names: list[str],
top_k_fusion: int = 20,
top_m_final: int = 5,
) -> dict | None:
"""从指定知识库中检索相关内容"""
kb_ids = []
kb_id_helper_map = {}
for kb_name in kb_names:
if kb_helper := await self.get_kb_by_name(kb_name):
kb_ids.append(kb_helper.kb.kb_id)
kb_id_helper_map[kb_helper.kb.kb_id] = kb_helper
if not kb_ids:
return {}
results = await self.retrieval_manager.retrieve(
query=query,
kb_ids=kb_ids,
kb_id_helper_map=kb_id_helper_map,
top_k_fusion=top_k_fusion,
top_m_final=top_m_final,
)
if not results:
return None
context_text = self._format_context(results)
results_dict = [
{
"chunk_id": r.chunk_id,
"doc_id": r.doc_id,
"kb_id": r.kb_id,
"kb_name": r.kb_name,
"doc_name": r.doc_name,
"chunk_index": r.metadata.get("chunk_index", 0),
"content": r.content,
"score": r.score,
"char_count": r.metadata.get("char_count", 0),
}
for r in results
]
return {
"context_text": context_text,
"results": results_dict,
}
def _format_context(self, results: list[RetrievalResult]) -> str:
"""格式化知识上下文
Args:
results: 检索结果列表
Returns:
str: 格式化的上下文文本
"""
lines = ["以下是相关的知识库内容,请参考这些信息回答用户的问题:\n"]
for i, result in enumerate(results, 1):
lines.append(f"【知识 {i}")
lines.append(f"来源: {result.kb_name} / {result.doc_name}")
lines.append(f"内容: {result.content}")
lines.append(f"相关度: {result.score:.2f}")
lines.append("")
return "\n".join(lines)
async def terminate(self):
"""终止所有知识库实例,关闭数据库连接"""
for kb_id, kb_helper in self.kb_insts.items():
try:
await kb_helper.terminate()
except Exception as e:
logger.error(f"关闭知识库 {kb_id} 失败: {e}")
self.kb_insts.clear()
# 关闭元数据数据库
if hasattr(self, "kb_db") and self.kb_db:
try:
await self.kb_db.close()
except Exception as e:
logger.error(f"关闭知识库元数据数据库失败: {e}")
async def upload_from_url(
self,
kb_id: str,
url: str,
chunk_size: int = 512,
chunk_overlap: int = 50,
batch_size: int = 32,
tasks_limit: int = 3,
max_retries: int = 3,
progress_callback=None,
) -> KBDocument:
"""从 URL 上传文档到指定的知识库
Args:
kb_id: 知识库 ID
url: 要提取内容的网页 URL
chunk_size: 文本块大小
chunk_overlap: 文本块重叠大小
batch_size: 批处理大小
tasks_limit: 并发任务限制
max_retries: 最大重试次数
progress_callback: 进度回调函数
Returns:
KBDocument: 上传的文档对象
Raises:
ValueError: 如果知识库不存在或 URL 为空
IOError: 如果网络请求失败
"""
kb_helper = await self.get_kb(kb_id)
if not kb_helper:
raise ValueError(f"Knowledge base with id {kb_id} not found.")
return await kb_helper.upload_from_url(
url=url,
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
batch_size=batch_size,
tasks_limit=tasks_limit,
max_retries=max_retries,
progress_callback=progress_callback,
)
-120
View File
@@ -1,120 +0,0 @@
import uuid
from datetime import datetime, timezone
from sqlmodel import Field, MetaData, SQLModel, Text, UniqueConstraint
class BaseKBModel(SQLModel, table=False):
metadata = MetaData()
class KnowledgeBase(BaseKBModel, table=True):
"""知识库表
存储知识库的基本信息和统计数据
"""
__tablename__ = "knowledge_bases" # type: ignore
id: int | None = Field(
primary_key=True,
sa_column_kwargs={"autoincrement": True},
default=None,
)
kb_id: str = Field(
max_length=36,
nullable=False,
unique=True,
default_factory=lambda: str(uuid.uuid4()),
index=True,
)
kb_name: str = Field(max_length=100, nullable=False)
description: str | None = Field(default=None, sa_type=Text)
emoji: str | None = Field(default="📚", max_length=10)
embedding_provider_id: str | None = Field(default=None, max_length=100)
rerank_provider_id: str | None = Field(default=None, max_length=100)
# 分块配置参数
chunk_size: int | None = Field(default=512, nullable=True)
chunk_overlap: int | None = Field(default=50, nullable=True)
# 检索配置参数
top_k_dense: int | None = Field(default=50, nullable=True)
top_k_sparse: int | None = Field(default=50, nullable=True)
top_m_final: int | None = Field(default=5, nullable=True)
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
updated_at: datetime = Field(
default_factory=lambda: datetime.now(timezone.utc),
sa_column_kwargs={"onupdate": datetime.now(timezone.utc)},
)
doc_count: int = Field(default=0, nullable=False)
chunk_count: int = Field(default=0, nullable=False)
__table_args__ = (
UniqueConstraint(
"kb_name",
name="uix_kb_name",
),
)
class KBDocument(BaseKBModel, table=True):
"""文档表
存储上传到知识库的文档元数据
"""
__tablename__ = "kb_documents" # type: ignore
id: int | None = Field(
primary_key=True,
sa_column_kwargs={"autoincrement": True},
default=None,
)
doc_id: str = Field(
max_length=36,
nullable=False,
unique=True,
default_factory=lambda: str(uuid.uuid4()),
index=True,
)
kb_id: str = Field(max_length=36, nullable=False, index=True)
doc_name: str = Field(max_length=255, nullable=False)
file_type: str = Field(max_length=20, nullable=False)
file_size: int = Field(nullable=False)
file_path: str = Field(max_length=512, nullable=False)
chunk_count: int = Field(default=0, nullable=False)
media_count: int = Field(default=0, nullable=False)
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
updated_at: datetime = Field(
default_factory=lambda: datetime.now(timezone.utc),
sa_column_kwargs={"onupdate": datetime.now(timezone.utc)},
)
class KBMedia(BaseKBModel, table=True):
"""多媒体资源表
存储从文档中提取的图片视频等多媒体资源
"""
__tablename__ = "kb_media" # type: ignore
id: int | None = Field(
primary_key=True,
sa_column_kwargs={"autoincrement": True},
default=None,
)
media_id: str = Field(
max_length=36,
nullable=False,
unique=True,
default_factory=lambda: str(uuid.uuid4()),
index=True,
)
doc_id: str = Field(max_length=36, nullable=False, index=True)
kb_id: str = Field(max_length=36, nullable=False, index=True)
media_type: str = Field(max_length=20, nullable=False)
file_name: str = Field(max_length=255, nullable=False)
file_path: str = Field(max_length=512, nullable=False)
file_size: int = Field(nullable=False)
mime_type: str = Field(max_length=100, nullable=False)
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
@@ -1,13 +0,0 @@
"""文档解析器模块"""
from .base import BaseParser, MediaItem, ParseResult
from .pdf_parser import PDFParser
from .text_parser import TextParser
__all__ = [
"BaseParser",
"MediaItem",
"PDFParser",
"ParseResult",
"TextParser",
]
@@ -1,51 +0,0 @@
"""文档解析器基类和数据结构
定义了文档解析器的抽象接口和相关数据类
"""
from abc import ABC, abstractmethod
from dataclasses import dataclass
@dataclass
class MediaItem:
"""多媒体项
表示从文档中提取的多媒体资源
"""
media_type: str # image, video
file_name: str
content: bytes
mime_type: str
@dataclass
class ParseResult:
"""解析结果
包含解析后的文本内容和提取的多媒体资源
"""
text: str
media: list[MediaItem]
class BaseParser(ABC):
"""文档解析器基类
所有文档解析器都应该继承此类并实现 parse 方法
"""
@abstractmethod
async def parse(self, file_content: bytes, file_name: str) -> ParseResult:
"""解析文档
Args:
file_content: 文件内容
file_name: 文件名
Returns:
ParseResult: 解析结果
"""
@@ -1,26 +0,0 @@
import io
import os
from markitdown_no_magika import MarkItDown, StreamInfo
from astrbot.core.knowledge_base.parsers.base import (
BaseParser,
ParseResult,
)
class MarkitdownParser(BaseParser):
"""解析 docx, xls, xlsx 格式"""
async def parse(self, file_content: bytes, file_name: str) -> ParseResult:
md = MarkItDown(enable_plugins=False)
bio = io.BytesIO(file_content)
stream_info = StreamInfo(
extension=os.path.splitext(file_name)[1].lower(),
filename=file_name,
)
result = md.convert(bio, stream_info=stream_info)
return ParseResult(
text=result.markdown,
media=[],
)

Some files were not shown because too many files have changed in this diff Show More