Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| a2fe0ec5a1 | |||
| 6957ec713d | |||
| d97c8b5b2b | |||
| d07a1ad5c9 | |||
| d8e6dfbd6b |
@@ -15,6 +15,7 @@ Always reference these instructions first and fallback to search or bash command
|
||||
### Running the Application
|
||||
- Run main application: `uv run main.py` -- starts in ~3 seconds
|
||||
- Application creates WebUI on http://localhost:6185 (default credentials: `astrbot`/`astrbot`)
|
||||
- Application loads plugins automatically from `packages/` and `data/plugins/` directories
|
||||
|
||||
### Dashboard Build (Vue.js/Node.js)
|
||||
- **Prerequisites**: Node.js 20+ and npm 10+ required
|
||||
@@ -34,7 +35,7 @@ Always reference these instructions first and fallback to search or bash command
|
||||
- **ALWAYS** run `uv run ruff check .` and `uv run ruff format .` before committing changes
|
||||
|
||||
### Plugin Development
|
||||
- Plugins load from `astrbot/builtin_stars/` (built-in) and `data/plugins/` (user-installed)
|
||||
- Plugins load from `packages/` (built-in) and `data/plugins/` (user-installed)
|
||||
- Plugin system supports function tools and message handlers
|
||||
- Key plugins: python_interpreter, web_searcher, astrbot, reminder, session_controller
|
||||
|
||||
|
||||
@@ -36,7 +36,7 @@ jobs:
|
||||
zip -r dist.zip dist
|
||||
|
||||
- name: Archive production artifacts
|
||||
uses: actions/upload-artifact@v6
|
||||
uses: actions/upload-artifact@v5
|
||||
with:
|
||||
name: dist-without-markdown
|
||||
path: |
|
||||
|
||||
+15
-52
@@ -1,64 +1,27 @@
|
||||
# 本工作流用于标记并关闭长期不活跃的 Issue。
|
||||
# 目前仅针对带 `bug` 标签的 Issue 生效,不会处理 PR。
|
||||
# This workflow warns and then closes issues and PRs that have had no activity for a specified amount of time.
|
||||
#
|
||||
# 文档: https://github.com/actions/stale
|
||||
name: Mark stale bug issues
|
||||
# You can adjust the behavior by modifying this file.
|
||||
# For more information, see:
|
||||
# https://github.com/actions/stale
|
||||
name: Mark stale issues and pull requests
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# 每天 UTC 08:30 执行 (北京时间 16:30)
|
||||
- cron: '30 8 * * *'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
dry-run:
|
||||
description: '仅预览, 不实际执行 (Dry run mode)'
|
||||
required: false
|
||||
default: true
|
||||
type: boolean
|
||||
- cron: '21 23 * * *'
|
||||
|
||||
jobs:
|
||||
stale:
|
||||
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
issues: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- uses: actions/stale@v10
|
||||
with:
|
||||
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
operations-per-run: 200
|
||||
|
||||
# 只处理带 bug 标签的 Issue
|
||||
any-of-labels: 'bug'
|
||||
|
||||
# 不处理 PR
|
||||
days-before-pr-stale: -1
|
||||
days-before-pr-close: -1
|
||||
|
||||
# 不活跃判定与关闭策略: 先标记 stale, 再延迟关闭
|
||||
days-before-issue-stale: 60
|
||||
days-before-issue-close: 30
|
||||
|
||||
stale-issue-label: 'stale'
|
||||
stale-issue-message: |
|
||||
This issue has been automatically marked as **stale** because it has not had any activity.
|
||||
It will be closed in a certain period of time if no further activity occurs.
|
||||
If this issue is still relevant, please leave a comment.
|
||||
|
||||
---
|
||||
|
||||
该 Issue 已较长时间无活动, 已被标记为 `stale`。
|
||||
如无后续活动, 将在一段时间后自动关闭。
|
||||
如仍需跟进, 请回复评论。
|
||||
close-issue-message: |
|
||||
This issue has been automatically closed due to inactivity.
|
||||
If the problem still exists, feel free to reopen or create a new issue with updated information.
|
||||
|
||||
---
|
||||
|
||||
该 Issue 因长期无活动已自动关闭。
|
||||
如问题仍存在, 欢迎补充复现信息并重新打开或新建 Issue。
|
||||
|
||||
remove-stale-when-updated: true
|
||||
|
||||
debug-only: ${{ github.event_name == 'workflow_dispatch' && inputs.dry-run }}
|
||||
- uses: actions/stale@v10
|
||||
with:
|
||||
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
stale-issue-message: 'Stale issue message'
|
||||
stale-pr-message: 'Stale pull request message'
|
||||
stale-issue-label: 'no-issue-activity'
|
||||
stale-pr-label: 'no-pr-activity'
|
||||
|
||||
+2
-2
@@ -24,9 +24,9 @@ configs/session
|
||||
configs/config.yaml
|
||||
cmd_config.json
|
||||
|
||||
# Plugins
|
||||
# Plugins and packages
|
||||
addons/plugins
|
||||
astrbot/builtin_stars/python_interpreter/workplace
|
||||
packages/python_interpreter/workplace
|
||||
tests/astrbot_plugin_openai
|
||||
|
||||
# Dashboard
|
||||
|
||||
+1
-26
@@ -33,20 +33,6 @@
|
||||
- 请使用英文描述您的 PR。
|
||||
- 标题请使用 `fix: `, `feat: `, `docs: `, `style: `, `refactor: `, `test: `, `chore: ` 等语义化前缀,并简要描述更改内容。如:`fix: correct login page typo`。
|
||||
|
||||
#### 代码规范
|
||||
|
||||
##### Core
|
||||
|
||||
我们使用 Ruff 作为代码格式化和静态分析工具。在提交代码之前,请运行以下命令以确保代码符合规范:
|
||||
|
||||
```bash
|
||||
ruff format .
|
||||
ruff check .
|
||||
```
|
||||
|
||||
如果您使用 VSCode,可以安装 `Ruff` 插件。
|
||||
|
||||
|
||||
## Contributing Guide
|
||||
|
||||
First off, thanks for taking the time to contribute! ❤️
|
||||
@@ -76,15 +62,4 @@ We use the `fix/` prefix for bug fixes and the `feat/` prefix for new features.
|
||||
|
||||
#### PR Description
|
||||
- Please use English to describe your PR.
|
||||
- Use semantic prefixes like `fix: `, `feat: `, `docs: `, `style: `, `refactor: `, `test: `, `chore: ` in the title, followed by a brief description of the changes, e.g., `fix: correct login page typo`.
|
||||
|
||||
#### Code Style
|
||||
|
||||
##### Core
|
||||
|
||||
We use Ruff as our code formatter and static analysis tool. Before submitting your code, please run the following commands to ensure your code adheres to the style guidelines:
|
||||
|
||||
```bash
|
||||
ruff format .
|
||||
ruff check .
|
||||
```
|
||||
- Use semantic prefixes like `fix: `, `feat: `, `docs: `, `style: `, `refactor: `, `test: `, `chore: ` in the title, followed by a brief description of the changes, e.g., `fix: correct login page typo`.
|
||||
@@ -132,7 +132,6 @@ uv run main.py
|
||||
|
||||
**社区维护**
|
||||
|
||||
- [Matrix](https://github.com/stevessr/astrbot_plugin_matrix_adapter)
|
||||
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
|
||||
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
|
||||
- [Bilibili 私信](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
|
||||
@@ -209,7 +208,6 @@ pre-commit install
|
||||
- 5 群:822130018
|
||||
- 6 群:753075035
|
||||
- 7 群:743746109
|
||||
- 8 群:1030353265
|
||||
- 开发者群:975206796
|
||||
|
||||
### Telegram 群组
|
||||
@@ -245,10 +243,4 @@ pre-commit install
|
||||
|
||||
</details>
|
||||
|
||||
<div align="center">
|
||||
|
||||
_私は、高性能ですから!_
|
||||
|
||||
<img src="https://files.astrbot.app/watashiwa-koseino-desukara.gif" width="100"/>
|
||||
</div
|
||||
|
||||
|
||||
@@ -134,7 +134,6 @@ Or refer to the official documentation: [Deploy AstrBot from Source](https://ast
|
||||
|
||||
**Community Maintained**
|
||||
|
||||
- [Matrix](https://github.com/stevessr/astrbot_plugin_matrix_adapter)
|
||||
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
|
||||
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
|
||||
- [Bilibili Direct Messages](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
|
||||
|
||||
@@ -134,7 +134,6 @@ Ou consultez la documentation officielle : [Déployer AstrBot depuis les sources
|
||||
|
||||
**Maintenues par la communauté**
|
||||
|
||||
- [Matrix](https://github.com/stevessr/astrbot_plugin_matrix_adapter)
|
||||
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
|
||||
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
|
||||
- [Messages directs Bilibili](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
|
||||
|
||||
@@ -134,7 +134,6 @@ uv run main.py
|
||||
|
||||
**コミュニティメンテナンス**
|
||||
|
||||
- [Matrix](https://github.com/stevessr/astrbot_plugin_matrix_adapter)
|
||||
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
|
||||
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
|
||||
- [Bilibili ダイレクトメッセージ](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
|
||||
|
||||
@@ -134,7 +134,6 @@ uv run main.py
|
||||
|
||||
**Поддерживаемые сообществом**
|
||||
|
||||
- [Matrix](https://github.com/stevessr/astrbot_plugin_matrix_adapter)
|
||||
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
|
||||
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
|
||||
- [Личные сообщения Bilibili](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
|
||||
|
||||
@@ -134,7 +134,6 @@ uv run main.py
|
||||
|
||||
**社群維護**
|
||||
|
||||
- [Matrix](https://github.com/stevessr/astrbot_plugin_matrix_adapter)
|
||||
- [KOOK](https://github.com/wuyan1003/astrbot_plugin_kook_adapter)
|
||||
- [VoceChat](https://github.com/HikariFroya/astrbot_plugin_vocechat)
|
||||
- [Bilibili 私訊](https://github.com/Hina-Chat/astrbot_plugin_bilibili_adapter)
|
||||
|
||||
@@ -21,9 +21,6 @@ from astrbot.core.star.register import (
|
||||
from astrbot.core.star.register import register_on_llm_request as on_llm_request
|
||||
from astrbot.core.star.register import register_on_llm_response as on_llm_response
|
||||
from astrbot.core.star.register import register_on_platform_loaded as on_platform_loaded
|
||||
from astrbot.core.star.register import (
|
||||
register_on_waiting_llm_request as on_waiting_llm_request,
|
||||
)
|
||||
from astrbot.core.star.register import register_permission_type as permission_type
|
||||
from astrbot.core.star.register import (
|
||||
register_platform_adapter_type as platform_adapter_type,
|
||||
@@ -49,7 +46,6 @@ __all__ = [
|
||||
"on_llm_request",
|
||||
"on_llm_response",
|
||||
"on_platform_loaded",
|
||||
"on_waiting_llm_request",
|
||||
"permission_type",
|
||||
"platform_adapter_type",
|
||||
"regex",
|
||||
|
||||
@@ -1,88 +0,0 @@
|
||||
import aiohttp
|
||||
|
||||
from astrbot.api import star
|
||||
from astrbot.api.event import AstrMessageEvent, MessageEventResult
|
||||
from astrbot.core.config.default import VERSION
|
||||
from astrbot.core.star import command_management
|
||||
from astrbot.core.utils.io import get_dashboard_version
|
||||
|
||||
|
||||
class HelpCommand:
|
||||
def __init__(self, context: star.Context):
|
||||
self.context = context
|
||||
|
||||
async def _query_astrbot_notice(self):
|
||||
try:
|
||||
async with aiohttp.ClientSession(trust_env=True) as session:
|
||||
async with session.get(
|
||||
"https://astrbot.app/notice.json",
|
||||
timeout=2,
|
||||
) as resp:
|
||||
return (await resp.json())["notice"]
|
||||
except BaseException:
|
||||
return ""
|
||||
|
||||
async def _build_reserved_command_lines(self) -> list[str]:
|
||||
"""
|
||||
使用实时指令配置生成内置指令清单,确保重命名/禁用后与实际生效状态保持一致。
|
||||
"""
|
||||
try:
|
||||
commands = await command_management.list_commands()
|
||||
except BaseException:
|
||||
return []
|
||||
|
||||
lines: list[str] = []
|
||||
hidden_commands = {"set", "unset", "websearch"}
|
||||
|
||||
def walk(items: list[dict], indent: int = 0):
|
||||
for item in items:
|
||||
if not item.get("reserved") or not item.get("enabled"):
|
||||
continue
|
||||
# 仅展示顶级指令或指令组
|
||||
if item.get("type") == "sub_command":
|
||||
continue
|
||||
if item.get("parent_signature"):
|
||||
continue
|
||||
|
||||
effective = (
|
||||
item.get("effective_command")
|
||||
or item.get("original_command")
|
||||
or item.get("handler_name")
|
||||
)
|
||||
if not effective:
|
||||
continue
|
||||
if effective in hidden_commands:
|
||||
continue
|
||||
|
||||
description = item.get("description") or ""
|
||||
desc_text = f" - {description}" if description else ""
|
||||
indent_prefix = " " * indent
|
||||
lines.append(f"{indent_prefix}/{effective}{desc_text}")
|
||||
|
||||
walk(commands)
|
||||
return lines
|
||||
|
||||
async def help(self, event: AstrMessageEvent):
|
||||
"""查看帮助"""
|
||||
notice = ""
|
||||
try:
|
||||
notice = await self._query_astrbot_notice()
|
||||
except BaseException:
|
||||
pass
|
||||
|
||||
dashboard_version = await get_dashboard_version()
|
||||
command_lines = await self._build_reserved_command_lines()
|
||||
commands_section = (
|
||||
"\n".join(command_lines) if command_lines else "暂无启用的内置指令"
|
||||
)
|
||||
|
||||
msg_parts = [
|
||||
f"AstrBot v{VERSION}(WebUI: {dashboard_version})",
|
||||
"内置指令:",
|
||||
commands_section,
|
||||
]
|
||||
if notice:
|
||||
msg_parts.append(notice)
|
||||
msg = "\n".join(msg_parts)
|
||||
|
||||
event.set_result(MessageEventResult().message(msg).use_t2i(False))
|
||||
@@ -1 +1 @@
|
||||
__version__ = "4.11.0"
|
||||
__version__ = "4.8.0"
|
||||
|
||||
@@ -1,243 +0,0 @@
|
||||
from typing import TYPE_CHECKING, Protocol, runtime_checkable
|
||||
|
||||
from ..message import Message
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from astrbot import logger
|
||||
else:
|
||||
try:
|
||||
from astrbot import logger
|
||||
except ImportError:
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger("astrbot")
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from astrbot.core.provider.provider import Provider
|
||||
|
||||
from ..context.truncator import ContextTruncator
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class ContextCompressor(Protocol):
|
||||
"""
|
||||
Protocol for context compressors.
|
||||
Provides an interface for compressing message lists.
|
||||
"""
|
||||
|
||||
def should_compress(
|
||||
self, messages: list[Message], current_tokens: int, max_tokens: int
|
||||
) -> bool:
|
||||
"""Check if compression is needed.
|
||||
|
||||
Args:
|
||||
messages: The message list to evaluate.
|
||||
current_tokens: The current token count.
|
||||
max_tokens: The maximum allowed tokens for the model.
|
||||
|
||||
Returns:
|
||||
True if compression is needed, False otherwise.
|
||||
"""
|
||||
...
|
||||
|
||||
async def __call__(self, messages: list[Message]) -> list[Message]:
|
||||
"""Compress the message list.
|
||||
|
||||
Args:
|
||||
messages: The original message list.
|
||||
|
||||
Returns:
|
||||
The compressed message list.
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
class TruncateByTurnsCompressor:
|
||||
"""Truncate by turns compressor implementation.
|
||||
Truncates the message list by removing older turns.
|
||||
"""
|
||||
|
||||
def __init__(self, truncate_turns: int = 1, compression_threshold: float = 0.82):
|
||||
"""Initialize the truncate by turns compressor.
|
||||
|
||||
Args:
|
||||
truncate_turns: The number of turns to remove when truncating (default: 1).
|
||||
compression_threshold: The compression trigger threshold (default: 0.82).
|
||||
"""
|
||||
self.truncate_turns = truncate_turns
|
||||
self.compression_threshold = compression_threshold
|
||||
|
||||
def should_compress(
|
||||
self, messages: list[Message], current_tokens: int, max_tokens: int
|
||||
) -> bool:
|
||||
"""Check if compression is needed.
|
||||
|
||||
Args:
|
||||
messages: The message list to evaluate.
|
||||
current_tokens: The current token count.
|
||||
max_tokens: The maximum allowed tokens.
|
||||
|
||||
Returns:
|
||||
True if compression is needed, False otherwise.
|
||||
"""
|
||||
if max_tokens <= 0 or current_tokens <= 0:
|
||||
return False
|
||||
usage_rate = current_tokens / max_tokens
|
||||
return usage_rate > self.compression_threshold
|
||||
|
||||
async def __call__(self, messages: list[Message]) -> list[Message]:
|
||||
truncator = ContextTruncator()
|
||||
truncated_messages = truncator.truncate_by_dropping_oldest_turns(
|
||||
messages,
|
||||
drop_turns=self.truncate_turns,
|
||||
)
|
||||
return truncated_messages
|
||||
|
||||
|
||||
def split_history(
|
||||
messages: list[Message], keep_recent: int
|
||||
) -> tuple[list[Message], list[Message], list[Message]]:
|
||||
"""Split the message list into system messages, messages to summarize, and recent messages.
|
||||
|
||||
Ensures that the split point is between complete user-assistant pairs to maintain conversation flow.
|
||||
|
||||
Args:
|
||||
messages: The original message list.
|
||||
keep_recent: The number of latest messages to keep.
|
||||
|
||||
Returns:
|
||||
tuple: (system_messages, messages_to_summarize, recent_messages)
|
||||
"""
|
||||
# keep the system messages
|
||||
first_non_system = 0
|
||||
for i, msg in enumerate(messages):
|
||||
if msg.role != "system":
|
||||
first_non_system = i
|
||||
break
|
||||
|
||||
system_messages = messages[:first_non_system]
|
||||
non_system_messages = messages[first_non_system:]
|
||||
|
||||
if len(non_system_messages) <= keep_recent:
|
||||
return system_messages, [], non_system_messages
|
||||
|
||||
# Find the split point, ensuring recent_messages starts with a user message
|
||||
# This maintains complete conversation turns
|
||||
split_index = len(non_system_messages) - keep_recent
|
||||
|
||||
# Search backward from split_index to find the first user message
|
||||
# This ensures recent_messages starts with a user message (complete turn)
|
||||
while split_index > 0 and non_system_messages[split_index].role != "user":
|
||||
# TODO: +=1 or -=1 ? calculate by tokens
|
||||
split_index -= 1
|
||||
|
||||
# If we couldn't find a user message, keep all messages as recent
|
||||
if split_index == 0:
|
||||
return system_messages, [], non_system_messages
|
||||
|
||||
messages_to_summarize = non_system_messages[:split_index]
|
||||
recent_messages = non_system_messages[split_index:]
|
||||
|
||||
return system_messages, messages_to_summarize, recent_messages
|
||||
|
||||
|
||||
class LLMSummaryCompressor:
|
||||
"""LLM-based summary compressor.
|
||||
Uses LLM to summarize the old conversation history, keeping the latest messages.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
provider: "Provider",
|
||||
keep_recent: int = 4,
|
||||
instruction_text: str | None = None,
|
||||
compression_threshold: float = 0.82,
|
||||
):
|
||||
"""Initialize the LLM summary compressor.
|
||||
|
||||
Args:
|
||||
provider: The LLM provider instance.
|
||||
keep_recent: The number of latest messages to keep (default: 4).
|
||||
instruction_text: Custom instruction for summary generation.
|
||||
compression_threshold: The compression trigger threshold (default: 0.82).
|
||||
"""
|
||||
self.provider = provider
|
||||
self.keep_recent = keep_recent
|
||||
self.compression_threshold = compression_threshold
|
||||
|
||||
self.instruction_text = instruction_text or (
|
||||
"Based on our full conversation history, produce a concise summary of key takeaways and/or project progress.\n"
|
||||
"1. Systematically cover all core topics discussed and the final conclusion/outcome for each; clearly highlight the latest primary focus.\n"
|
||||
"2. If any tools were used, summarize tool usage (total call count) and extract the most valuable insights from tool outputs.\n"
|
||||
"3. If there was an initial user goal, state it first and describe the current progress/status.\n"
|
||||
"4. Write the summary in the user's language.\n"
|
||||
)
|
||||
|
||||
def should_compress(
|
||||
self, messages: list[Message], current_tokens: int, max_tokens: int
|
||||
) -> bool:
|
||||
"""Check if compression is needed.
|
||||
|
||||
Args:
|
||||
messages: The message list to evaluate.
|
||||
current_tokens: The current token count.
|
||||
max_tokens: The maximum allowed tokens.
|
||||
|
||||
Returns:
|
||||
True if compression is needed, False otherwise.
|
||||
"""
|
||||
if max_tokens <= 0 or current_tokens <= 0:
|
||||
return False
|
||||
usage_rate = current_tokens / max_tokens
|
||||
return usage_rate > self.compression_threshold
|
||||
|
||||
async def __call__(self, messages: list[Message]) -> list[Message]:
|
||||
"""Use LLM to generate a summary of the conversation history.
|
||||
|
||||
Process:
|
||||
1. Divide messages: keep the system message and the latest N messages.
|
||||
2. Send the old messages + the instruction message to the LLM.
|
||||
3. Reconstruct the message list: [system message, summary message, latest messages].
|
||||
"""
|
||||
if len(messages) <= self.keep_recent + 1:
|
||||
return messages
|
||||
|
||||
system_messages, messages_to_summarize, recent_messages = split_history(
|
||||
messages, self.keep_recent
|
||||
)
|
||||
|
||||
if not messages_to_summarize:
|
||||
return messages
|
||||
|
||||
# build payload
|
||||
instruction_message = Message(role="user", content=self.instruction_text)
|
||||
llm_payload = messages_to_summarize + [instruction_message]
|
||||
|
||||
# generate summary
|
||||
try:
|
||||
response = await self.provider.text_chat(contexts=llm_payload)
|
||||
summary_content = response.completion_text
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to generate summary: {e}")
|
||||
return messages
|
||||
|
||||
# build result
|
||||
result = []
|
||||
result.extend(system_messages)
|
||||
|
||||
result.append(
|
||||
Message(
|
||||
role="user",
|
||||
content=f"Our previous history conversation summary: {summary_content}",
|
||||
)
|
||||
)
|
||||
result.append(
|
||||
Message(
|
||||
role="assistant",
|
||||
content="Acknowledged the summary of our previous conversation history.",
|
||||
)
|
||||
)
|
||||
|
||||
result.extend(recent_messages)
|
||||
|
||||
return result
|
||||
@@ -1,35 +0,0 @@
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from .compressor import ContextCompressor
|
||||
from .token_counter import TokenCounter
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from astrbot.core.provider.provider import Provider
|
||||
|
||||
|
||||
@dataclass
|
||||
class ContextConfig:
|
||||
"""Context configuration class."""
|
||||
|
||||
max_context_tokens: int = 0
|
||||
"""Maximum number of context tokens. <= 0 means no limit."""
|
||||
enforce_max_turns: int = -1 # -1 means no limit
|
||||
"""Maximum number of conversation turns to keep. -1 means no limit. Executed before compression."""
|
||||
truncate_turns: int = 1
|
||||
"""Number of conversation turns to discard at once when truncation is triggered.
|
||||
Two processes will use this value:
|
||||
|
||||
1. Enforce max turns truncation.
|
||||
2. Truncation by turns compression strategy.
|
||||
"""
|
||||
llm_compress_instruction: str | None = None
|
||||
"""Instruction prompt for LLM-based compression."""
|
||||
llm_compress_keep_recent: int = 0
|
||||
"""Number of recent messages to keep during LLM-based compression."""
|
||||
llm_compress_provider: "Provider | None" = None
|
||||
"""LLM provider used for compression tasks. If None, truncation strategy is used."""
|
||||
custom_token_counter: TokenCounter | None = None
|
||||
"""Custom token counting method. If None, the default method is used."""
|
||||
custom_compressor: ContextCompressor | None = None
|
||||
"""Custom context compression method. If None, the default method is used."""
|
||||
@@ -1,120 +0,0 @@
|
||||
from astrbot import logger
|
||||
|
||||
from ..message import Message
|
||||
from .compressor import LLMSummaryCompressor, TruncateByTurnsCompressor
|
||||
from .config import ContextConfig
|
||||
from .token_counter import EstimateTokenCounter
|
||||
from .truncator import ContextTruncator
|
||||
|
||||
|
||||
class ContextManager:
|
||||
"""Context compression manager."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
config: ContextConfig,
|
||||
):
|
||||
"""Initialize the context manager.
|
||||
|
||||
There are two strategies to handle context limit reached:
|
||||
1. Truncate by turns: remove older messages by turns.
|
||||
2. LLM-based compression: use LLM to summarize old messages.
|
||||
|
||||
Args:
|
||||
config: The context configuration.
|
||||
"""
|
||||
self.config = config
|
||||
|
||||
self.token_counter = config.custom_token_counter or EstimateTokenCounter()
|
||||
self.truncator = ContextTruncator()
|
||||
|
||||
if config.custom_compressor:
|
||||
self.compressor = config.custom_compressor
|
||||
elif config.llm_compress_provider:
|
||||
self.compressor = LLMSummaryCompressor(
|
||||
provider=config.llm_compress_provider,
|
||||
keep_recent=config.llm_compress_keep_recent,
|
||||
instruction_text=config.llm_compress_instruction,
|
||||
)
|
||||
else:
|
||||
self.compressor = TruncateByTurnsCompressor(
|
||||
truncate_turns=config.truncate_turns
|
||||
)
|
||||
|
||||
async def process(
|
||||
self, messages: list[Message], trusted_token_usage: int = 0
|
||||
) -> list[Message]:
|
||||
"""Process the messages.
|
||||
|
||||
Args:
|
||||
messages: The original message list.
|
||||
|
||||
Returns:
|
||||
The processed message list.
|
||||
"""
|
||||
try:
|
||||
result = messages
|
||||
|
||||
# 1. 基于轮次的截断 (Enforce max turns)
|
||||
if self.config.enforce_max_turns != -1:
|
||||
result = self.truncator.truncate_by_turns(
|
||||
result,
|
||||
keep_most_recent_turns=self.config.enforce_max_turns,
|
||||
drop_turns=self.config.truncate_turns,
|
||||
)
|
||||
|
||||
# 2. 基于 token 的压缩
|
||||
if self.config.max_context_tokens > 0:
|
||||
total_tokens = self.token_counter.count_tokens(
|
||||
result, trusted_token_usage
|
||||
)
|
||||
|
||||
if self.compressor.should_compress(
|
||||
result, total_tokens, self.config.max_context_tokens
|
||||
):
|
||||
result = await self._run_compression(result, total_tokens)
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
logger.error(f"Error during context processing: {e}", exc_info=True)
|
||||
return messages
|
||||
|
||||
async def _run_compression(
|
||||
self, messages: list[Message], prev_tokens: int
|
||||
) -> list[Message]:
|
||||
"""
|
||||
Compress/truncate the messages.
|
||||
|
||||
Args:
|
||||
messages: The original message list.
|
||||
prev_tokens: The token count before compression.
|
||||
|
||||
Returns:
|
||||
The compressed/truncated message list.
|
||||
"""
|
||||
logger.debug("Compress triggered, starting compression...")
|
||||
|
||||
messages = await self.compressor(messages)
|
||||
|
||||
# double check
|
||||
tokens_after_summary = self.token_counter.count_tokens(messages)
|
||||
|
||||
# calculate compress rate
|
||||
compress_rate = (tokens_after_summary / self.config.max_context_tokens) * 100
|
||||
logger.info(
|
||||
f"Compress completed."
|
||||
f" {prev_tokens} -> {tokens_after_summary} tokens,"
|
||||
f" compression rate: {compress_rate:.2f}%.",
|
||||
)
|
||||
|
||||
# last check
|
||||
if self.compressor.should_compress(
|
||||
messages, tokens_after_summary, self.config.max_context_tokens
|
||||
):
|
||||
logger.info(
|
||||
"Context still exceeds max tokens after compression, applying halving truncation..."
|
||||
)
|
||||
# still need compress, truncate by half
|
||||
messages = self.truncator.truncate_by_halving(messages)
|
||||
|
||||
return messages
|
||||
@@ -1,64 +0,0 @@
|
||||
import json
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
from ..message import Message, TextPart
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class TokenCounter(Protocol):
|
||||
"""
|
||||
Protocol for token counters.
|
||||
Provides an interface for counting tokens in message lists.
|
||||
"""
|
||||
|
||||
def count_tokens(
|
||||
self, messages: list[Message], trusted_token_usage: int = 0
|
||||
) -> int:
|
||||
"""Count the total tokens in the message list.
|
||||
|
||||
Args:
|
||||
messages: The message list.
|
||||
trusted_token_usage: The total token usage that LLM API returned.
|
||||
For some cases, this value is more accurate.
|
||||
But some API does not return it, so the value defaults to 0.
|
||||
|
||||
Returns:
|
||||
The total token count.
|
||||
"""
|
||||
...
|
||||
|
||||
|
||||
class EstimateTokenCounter:
|
||||
"""Estimate token counter implementation.
|
||||
Provides a simple estimation of token count based on character types.
|
||||
"""
|
||||
|
||||
def count_tokens(
|
||||
self, messages: list[Message], trusted_token_usage: int = 0
|
||||
) -> int:
|
||||
if trusted_token_usage > 0:
|
||||
return trusted_token_usage
|
||||
|
||||
total = 0
|
||||
for msg in messages:
|
||||
content = msg.content
|
||||
if isinstance(content, str):
|
||||
total += self._estimate_tokens(content)
|
||||
elif isinstance(content, list):
|
||||
# 处理多模态内容
|
||||
for part in content:
|
||||
if isinstance(part, TextPart):
|
||||
total += self._estimate_tokens(part.text)
|
||||
|
||||
# 处理 Tool Calls
|
||||
if msg.tool_calls:
|
||||
for tc in msg.tool_calls:
|
||||
tc_str = json.dumps(tc if isinstance(tc, dict) else tc.model_dump())
|
||||
total += self._estimate_tokens(tc_str)
|
||||
|
||||
return total
|
||||
|
||||
def _estimate_tokens(self, text: str) -> int:
|
||||
chinese_count = len([c for c in text if "\u4e00" <= c <= "\u9fff"])
|
||||
other_count = len(text) - chinese_count
|
||||
return int(chinese_count * 0.6 + other_count * 0.3)
|
||||
@@ -1,141 +0,0 @@
|
||||
from ..message import Message
|
||||
|
||||
|
||||
class ContextTruncator:
|
||||
"""Context truncator."""
|
||||
|
||||
def fix_messages(self, messages: list[Message]) -> list[Message]:
|
||||
fixed_messages = []
|
||||
for message in messages:
|
||||
if message.role == "tool":
|
||||
# tool block 前面必须要有 user 和 assistant block
|
||||
if len(fixed_messages) < 2:
|
||||
# 这种情况可能是上下文被截断导致的
|
||||
# 我们直接将之前的上下文都清空
|
||||
fixed_messages = []
|
||||
else:
|
||||
fixed_messages.append(message)
|
||||
else:
|
||||
fixed_messages.append(message)
|
||||
return fixed_messages
|
||||
|
||||
def truncate_by_turns(
|
||||
self,
|
||||
messages: list[Message],
|
||||
keep_most_recent_turns: int,
|
||||
drop_turns: int = 1,
|
||||
) -> list[Message]:
|
||||
"""截断上下文列表,确保不超过最大长度。
|
||||
一个 turn 包含一个 user 消息和一个 assistant 消息。
|
||||
这个方法会保证截断后的上下文列表符合 OpenAI 的上下文格式。
|
||||
|
||||
Args:
|
||||
messages: 上下文列表
|
||||
keep_most_recent_turns: 保留最近的对话轮数
|
||||
drop_turns: 一次性丢弃的对话轮数
|
||||
|
||||
Returns:
|
||||
截断后的上下文列表
|
||||
"""
|
||||
if keep_most_recent_turns == -1:
|
||||
return messages
|
||||
|
||||
first_non_system = 0
|
||||
for i, msg in enumerate(messages):
|
||||
if msg.role != "system":
|
||||
first_non_system = i
|
||||
break
|
||||
|
||||
system_messages = messages[:first_non_system]
|
||||
non_system_messages = messages[first_non_system:]
|
||||
|
||||
if len(non_system_messages) // 2 <= keep_most_recent_turns:
|
||||
return messages
|
||||
|
||||
num_to_keep = keep_most_recent_turns - drop_turns + 1
|
||||
if num_to_keep <= 0:
|
||||
truncated_contexts = []
|
||||
else:
|
||||
truncated_contexts = non_system_messages[-num_to_keep * 2 :]
|
||||
|
||||
# 找到第一个 role 为 user 的索引,确保上下文格式正确
|
||||
index = next(
|
||||
(i for i, item in enumerate(truncated_contexts) if item.role == "user"),
|
||||
None,
|
||||
)
|
||||
if index is not None and index > 0:
|
||||
truncated_contexts = truncated_contexts[index:]
|
||||
|
||||
result = system_messages + truncated_contexts
|
||||
|
||||
return self.fix_messages(result)
|
||||
|
||||
def truncate_by_dropping_oldest_turns(
|
||||
self,
|
||||
messages: list[Message],
|
||||
drop_turns: int = 1,
|
||||
) -> list[Message]:
|
||||
"""丢弃最旧的 N 个对话轮次。"""
|
||||
if drop_turns <= 0:
|
||||
return messages
|
||||
|
||||
first_non_system = 0
|
||||
for i, msg in enumerate(messages):
|
||||
if msg.role != "system":
|
||||
first_non_system = i
|
||||
break
|
||||
|
||||
system_messages = messages[:first_non_system]
|
||||
non_system_messages = messages[first_non_system:]
|
||||
|
||||
if len(non_system_messages) // 2 <= drop_turns:
|
||||
truncated_non_system = []
|
||||
else:
|
||||
truncated_non_system = non_system_messages[drop_turns * 2 :]
|
||||
|
||||
index = next(
|
||||
(i for i, item in enumerate(truncated_non_system) if item.role == "user"),
|
||||
None,
|
||||
)
|
||||
if index is not None:
|
||||
truncated_non_system = truncated_non_system[index:]
|
||||
elif truncated_non_system:
|
||||
truncated_non_system = []
|
||||
|
||||
result = system_messages + truncated_non_system
|
||||
|
||||
return self.fix_messages(result)
|
||||
|
||||
def truncate_by_halving(
|
||||
self,
|
||||
messages: list[Message],
|
||||
) -> list[Message]:
|
||||
"""对半砍策略,删除 50% 的消息"""
|
||||
if len(messages) <= 2:
|
||||
return messages
|
||||
|
||||
first_non_system = 0
|
||||
for i, msg in enumerate(messages):
|
||||
if msg.role != "system":
|
||||
first_non_system = i
|
||||
break
|
||||
|
||||
system_messages = messages[:first_non_system]
|
||||
non_system_messages = messages[first_non_system:]
|
||||
|
||||
messages_to_delete = len(non_system_messages) // 2
|
||||
if messages_to_delete == 0:
|
||||
return messages
|
||||
|
||||
truncated_non_system = non_system_messages[messages_to_delete:]
|
||||
|
||||
index = next(
|
||||
(i for i, item in enumerate(truncated_non_system) if item.role == "user"),
|
||||
None,
|
||||
)
|
||||
if index is not None:
|
||||
truncated_non_system = truncated_non_system[index:]
|
||||
|
||||
result = system_messages + truncated_non_system
|
||||
|
||||
return self.fix_messages(result)
|
||||
@@ -3,7 +3,7 @@
|
||||
|
||||
from typing import Any, ClassVar, Literal, cast
|
||||
|
||||
from pydantic import BaseModel, GetCoreSchemaHandler, model_serializer, model_validator
|
||||
from pydantic import BaseModel, GetCoreSchemaHandler, model_validator
|
||||
from pydantic_core import core_schema
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ class ContentPart(BaseModel):
|
||||
|
||||
__content_part_registry: ClassVar[dict[str, type["ContentPart"]]] = {}
|
||||
|
||||
type: Literal["text", "think", "image_url", "audio_url"]
|
||||
type: str
|
||||
|
||||
def __init_subclass__(cls, **kwargs: Any) -> None:
|
||||
super().__init_subclass__(**kwargs)
|
||||
@@ -63,28 +63,6 @@ class TextPart(ContentPart):
|
||||
text: str
|
||||
|
||||
|
||||
class ThinkPart(ContentPart):
|
||||
"""
|
||||
>>> ThinkPart(think="I think I need to think about this.").model_dump()
|
||||
{'type': 'think', 'think': 'I think I need to think about this.', 'encrypted': None}
|
||||
"""
|
||||
|
||||
type: str = "think"
|
||||
think: str
|
||||
encrypted: str | None = None
|
||||
"""Encrypted thinking content, or signature."""
|
||||
|
||||
def merge_in_place(self, other: Any) -> bool:
|
||||
if not isinstance(other, ThinkPart):
|
||||
return False
|
||||
if self.encrypted:
|
||||
return False
|
||||
self.think += other.think
|
||||
if other.encrypted:
|
||||
self.encrypted = other.encrypted
|
||||
return True
|
||||
|
||||
|
||||
class ImageURLPart(ContentPart):
|
||||
"""
|
||||
>>> ImageURLPart(image_url="http://example.com/image.jpg").model_dump()
|
||||
@@ -144,12 +122,10 @@ class ToolCall(BaseModel):
|
||||
extra_content: dict[str, Any] | None = None
|
||||
"""Extra metadata for the tool call."""
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def serialize(self, handler):
|
||||
data = handler(self)
|
||||
def model_dump(self, **kwargs: Any) -> dict[str, Any]:
|
||||
if self.extra_content is None:
|
||||
data.pop("extra_content", None)
|
||||
return data
|
||||
kwargs.setdefault("exclude", set()).add("extra_content")
|
||||
return super().model_dump(**kwargs)
|
||||
|
||||
|
||||
class ToolCallPart(BaseModel):
|
||||
@@ -191,15 +167,6 @@ class Message(BaseModel):
|
||||
)
|
||||
return self
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def serialize(self, handler):
|
||||
data = handler(self)
|
||||
if self.tool_calls is None:
|
||||
data.pop("tool_calls", None)
|
||||
if self.tool_call_id is None:
|
||||
data.pop("tool_call_id", None)
|
||||
return data
|
||||
|
||||
|
||||
class AssistantMessageSegment(Message):
|
||||
"""A message segment from the assistant."""
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
import typing as T
|
||||
from dataclasses import dataclass, field
|
||||
from dataclasses import dataclass
|
||||
|
||||
from astrbot.core.message.message_event_result import MessageChain
|
||||
from astrbot.core.provider.entities import TokenUsage
|
||||
|
||||
|
||||
class AgentResponseData(T.TypedDict):
|
||||
@@ -13,23 +12,3 @@ class AgentResponseData(T.TypedDict):
|
||||
class AgentResponse:
|
||||
type: str
|
||||
data: AgentResponseData
|
||||
|
||||
|
||||
@dataclass
|
||||
class AgentStats:
|
||||
token_usage: TokenUsage = field(default_factory=TokenUsage)
|
||||
start_time: float = 0.0
|
||||
end_time: float = 0.0
|
||||
time_to_first_token: float = 0.0
|
||||
|
||||
@property
|
||||
def duration(self) -> float:
|
||||
return self.end_time - self.start_time
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"token_usage": self.token_usage.__dict__,
|
||||
"start_time": self.start_time,
|
||||
"end_time": self.end_time,
|
||||
"time_to_first_token": self.time_to_first_token,
|
||||
}
|
||||
|
||||
@@ -9,7 +9,7 @@ from .message import Message
|
||||
TContext = TypeVar("TContext", default=Any)
|
||||
|
||||
|
||||
@dataclass
|
||||
@dataclass(config={"arbitrary_types_allowed": True})
|
||||
class ContextWrapper(Generic[TContext]):
|
||||
"""A context for running an agent, which can be used to pass additional data or state."""
|
||||
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import sys
|
||||
import time
|
||||
import traceback
|
||||
import typing as T
|
||||
|
||||
@@ -13,8 +12,6 @@ from mcp.types import (
|
||||
)
|
||||
|
||||
from astrbot import logger
|
||||
from astrbot.core.agent.message import TextPart, ThinkPart
|
||||
from astrbot.core.message.components import Json
|
||||
from astrbot.core.message.message_event_result import (
|
||||
MessageChain,
|
||||
)
|
||||
@@ -25,13 +22,9 @@ from astrbot.core.provider.entities import (
|
||||
)
|
||||
from astrbot.core.provider.provider import Provider
|
||||
|
||||
from ..context.compressor import ContextCompressor
|
||||
from ..context.config import ContextConfig
|
||||
from ..context.manager import ContextManager
|
||||
from ..context.token_counter import TokenCounter
|
||||
from ..hooks import BaseAgentRunHooks
|
||||
from ..message import AssistantMessageSegment, Message, ToolCallMessageSegment
|
||||
from ..response import AgentResponseData, AgentStats
|
||||
from ..response import AgentResponseData
|
||||
from ..run_context import ContextWrapper, TContext
|
||||
from ..tool_executor import BaseFunctionToolExecutor
|
||||
from .base import AgentResponse, AgentState, BaseAgentRunner
|
||||
@@ -51,47 +44,10 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
run_context: ContextWrapper[TContext],
|
||||
tool_executor: BaseFunctionToolExecutor[TContext],
|
||||
agent_hooks: BaseAgentRunHooks[TContext],
|
||||
streaming: bool = False,
|
||||
# enforce max turns, will discard older turns when exceeded BEFORE compression
|
||||
# -1 means no limit
|
||||
enforce_max_turns: int = -1,
|
||||
# llm compressor
|
||||
llm_compress_instruction: str | None = None,
|
||||
llm_compress_keep_recent: int = 0,
|
||||
llm_compress_provider: Provider | None = None,
|
||||
# truncate by turns compressor
|
||||
truncate_turns: int = 1,
|
||||
# customize
|
||||
custom_token_counter: TokenCounter | None = None,
|
||||
custom_compressor: ContextCompressor | None = None,
|
||||
**kwargs: T.Any,
|
||||
) -> None:
|
||||
self.req = request
|
||||
self.streaming = streaming
|
||||
self.enforce_max_turns = enforce_max_turns
|
||||
self.llm_compress_instruction = llm_compress_instruction
|
||||
self.llm_compress_keep_recent = llm_compress_keep_recent
|
||||
self.llm_compress_provider = llm_compress_provider
|
||||
self.truncate_turns = truncate_turns
|
||||
self.custom_token_counter = custom_token_counter
|
||||
self.custom_compressor = custom_compressor
|
||||
# we will do compress when:
|
||||
# 1. before requesting LLM
|
||||
# TODO: 2. after LLM output a tool call
|
||||
self.context_config = ContextConfig(
|
||||
# <=0 will never do compress
|
||||
max_context_tokens=provider.provider_config.get("max_context_tokens", 0),
|
||||
# enforce max turns before compression
|
||||
enforce_max_turns=self.enforce_max_turns,
|
||||
truncate_turns=self.truncate_turns,
|
||||
llm_compress_instruction=self.llm_compress_instruction,
|
||||
llm_compress_keep_recent=self.llm_compress_keep_recent,
|
||||
llm_compress_provider=self.llm_compress_provider,
|
||||
custom_token_counter=self.custom_token_counter,
|
||||
custom_compressor=self.custom_compressor,
|
||||
)
|
||||
self.context_manager = ContextManager(self.context_config)
|
||||
|
||||
self.streaming = kwargs.get("streaming", False)
|
||||
self.provider = provider
|
||||
self.final_llm_resp = None
|
||||
self._state = AgentState.IDLE
|
||||
@@ -113,25 +69,14 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
)
|
||||
self.run_context.messages = messages
|
||||
|
||||
self.stats = AgentStats()
|
||||
self.stats.start_time = time.time()
|
||||
|
||||
async def _iter_llm_responses(self) -> T.AsyncGenerator[LLMResponse, None]:
|
||||
"""Yields chunks *and* a final LLMResponse."""
|
||||
payload = {
|
||||
"contexts": self.run_context.messages, # list[Message]
|
||||
"func_tool": self.req.func_tool,
|
||||
"model": self.req.model, # NOTE: in fact, this arg is None in most cases
|
||||
"session_id": self.req.session_id,
|
||||
"extra_user_content_parts": self.req.extra_user_content_parts, # list[ContentPart]
|
||||
}
|
||||
|
||||
if self.streaming:
|
||||
stream = self.provider.text_chat_stream(**payload)
|
||||
stream = self.provider.text_chat_stream(**self.req.__dict__)
|
||||
async for resp in stream: # type: ignore
|
||||
yield resp
|
||||
else:
|
||||
yield await self.provider.text_chat(**payload)
|
||||
yield await self.provider.text_chat(**self.req.__dict__)
|
||||
|
||||
@override
|
||||
async def step(self):
|
||||
@@ -151,18 +96,8 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
self._transition_state(AgentState.RUNNING)
|
||||
llm_resp_result = None
|
||||
|
||||
# do truncate and compress
|
||||
token_usage = self.req.conversation.token_usage if self.req.conversation else 0
|
||||
self.run_context.messages = await self.context_manager.process(
|
||||
self.run_context.messages, trusted_token_usage=token_usage
|
||||
)
|
||||
|
||||
async for llm_response in self._iter_llm_responses():
|
||||
if llm_response.is_chunk:
|
||||
# update ttft
|
||||
if self.stats.time_to_first_token == 0:
|
||||
self.stats.time_to_first_token = time.time() - self.stats.start_time
|
||||
|
||||
if llm_response.result_chain:
|
||||
yield AgentResponse(
|
||||
type="streaming_delta",
|
||||
@@ -186,10 +121,6 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
)
|
||||
continue
|
||||
llm_resp_result = llm_response
|
||||
|
||||
if not llm_response.is_chunk and llm_response.usage:
|
||||
# only count the token usage of the final response for computation purpose
|
||||
self.stats.token_usage += llm_response.usage
|
||||
break # got final response
|
||||
|
||||
if not llm_resp_result:
|
||||
@@ -201,7 +132,6 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
if llm_resp.role == "err":
|
||||
# 如果 LLM 响应错误,转换到错误状态
|
||||
self.final_llm_resp = llm_resp
|
||||
self.stats.end_time = time.time()
|
||||
self._transition_state(AgentState.ERROR)
|
||||
yield AgentResponse(
|
||||
type="err",
|
||||
@@ -216,21 +146,13 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
# 如果没有工具调用,转换到完成状态
|
||||
self.final_llm_resp = llm_resp
|
||||
self._transition_state(AgentState.DONE)
|
||||
self.stats.end_time = time.time()
|
||||
|
||||
# record the final assistant message
|
||||
parts = []
|
||||
if llm_resp.reasoning_content or llm_resp.reasoning_signature:
|
||||
parts.append(
|
||||
ThinkPart(
|
||||
think=llm_resp.reasoning_content,
|
||||
encrypted=llm_resp.reasoning_signature,
|
||||
)
|
||||
)
|
||||
parts.append(TextPart(text=llm_resp.completion_text or "*No response*"))
|
||||
self.run_context.messages.append(Message(role="assistant", content=parts))
|
||||
|
||||
# call the on_agent_done hook
|
||||
self.run_context.messages.append(
|
||||
Message(
|
||||
role="assistant",
|
||||
content=llm_resp.completion_text or "",
|
||||
),
|
||||
)
|
||||
try:
|
||||
await self.agent_hooks.on_agent_done(self.run_context, llm_resp)
|
||||
except Exception as e:
|
||||
@@ -253,35 +175,29 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
# 如果有工具调用,还需处理工具调用
|
||||
if llm_resp.tools_call_name:
|
||||
tool_call_result_blocks = []
|
||||
for tool_call_name in llm_resp.tools_call_name:
|
||||
yield AgentResponse(
|
||||
type="tool_call",
|
||||
data=AgentResponseData(
|
||||
chain=MessageChain(type="tool_call").message(
|
||||
f"🔨 调用工具: {tool_call_name}"
|
||||
),
|
||||
),
|
||||
)
|
||||
async for result in self._handle_function_tools(self.req, llm_resp):
|
||||
if isinstance(result, list):
|
||||
tool_call_result_blocks = result
|
||||
elif isinstance(result, MessageChain):
|
||||
if result.type is None:
|
||||
# should not happen
|
||||
continue
|
||||
if result.type == "tool_direct_result":
|
||||
ar_type = "tool_call_result"
|
||||
else:
|
||||
ar_type = result.type
|
||||
result.type = "tool_call_result"
|
||||
yield AgentResponse(
|
||||
type=ar_type,
|
||||
type="tool_call_result",
|
||||
data=AgentResponseData(chain=result),
|
||||
)
|
||||
# 将结果添加到上下文中
|
||||
parts = []
|
||||
if llm_resp.reasoning_content or llm_resp.reasoning_signature:
|
||||
parts.append(
|
||||
ThinkPart(
|
||||
think=llm_resp.reasoning_content,
|
||||
encrypted=llm_resp.reasoning_signature,
|
||||
)
|
||||
)
|
||||
parts.append(TextPart(text=llm_resp.completion_text or "*No response*"))
|
||||
tool_calls_result = ToolCallsResult(
|
||||
tool_calls_info=AssistantMessageSegment(
|
||||
tool_calls=llm_resp.to_openai_to_calls_model(),
|
||||
content=parts,
|
||||
content=llm_resp.completion_text,
|
||||
),
|
||||
tool_calls_result=tool_call_result_blocks,
|
||||
)
|
||||
@@ -302,25 +218,6 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
async for resp in self.step():
|
||||
yield resp
|
||||
|
||||
# 如果循环结束了但是 agent 还没有完成,说明是达到了 max_step
|
||||
if not self.done():
|
||||
logger.warning(
|
||||
f"Agent reached max steps ({max_step}), forcing a final response."
|
||||
)
|
||||
# 拔掉所有工具
|
||||
if self.req:
|
||||
self.req.func_tool = None
|
||||
# 注入提示词
|
||||
self.run_context.messages.append(
|
||||
Message(
|
||||
role="user",
|
||||
content="工具调用次数已达到上限,请停止使用工具,并根据已经收集到的信息,对你的任务和发现进行总结,然后直接回复用户。",
|
||||
)
|
||||
)
|
||||
# 再执行最后一步
|
||||
async for resp in self.step():
|
||||
yield resp
|
||||
|
||||
async def _handle_function_tools(
|
||||
self,
|
||||
req: ProviderRequest,
|
||||
@@ -336,19 +233,6 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
llm_response.tools_call_args,
|
||||
llm_response.tools_call_ids,
|
||||
):
|
||||
yield MessageChain(
|
||||
type="tool_call",
|
||||
chain=[
|
||||
Json(
|
||||
data={
|
||||
"id": func_tool_id,
|
||||
"name": func_tool_name,
|
||||
"args": func_tool_args,
|
||||
"ts": time.time(),
|
||||
}
|
||||
)
|
||||
],
|
||||
)
|
||||
try:
|
||||
if not req.func_tool:
|
||||
return
|
||||
@@ -422,6 +306,7 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
content=res.content[0].text,
|
||||
),
|
||||
)
|
||||
yield MessageChain().message(res.content[0].text)
|
||||
elif isinstance(res.content[0], ImageContent):
|
||||
tool_call_result_blocks.append(
|
||||
ToolCallMessageSegment(
|
||||
@@ -443,6 +328,7 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
content=resource.text,
|
||||
),
|
||||
)
|
||||
yield MessageChain().message(resource.text)
|
||||
elif (
|
||||
isinstance(resource, BlobResourceContents)
|
||||
and resource.mimeType
|
||||
@@ -466,34 +352,20 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
content="返回的数据类型不受支持",
|
||||
),
|
||||
)
|
||||
yield MessageChain().message("返回的数据类型不受支持。")
|
||||
|
||||
elif resp is None:
|
||||
# Tool 直接请求发送消息给用户
|
||||
# 这里我们将直接结束 Agent Loop。
|
||||
# 发送消息逻辑在 ToolExecutor 中处理了。
|
||||
logger.warning(
|
||||
f"{func_tool_name} 没有没有返回值或者将结果直接发送给用户。"
|
||||
f"{func_tool_name} 没有没有返回值或者将结果直接发送给用户,此工具调用不会被记录到历史中。"
|
||||
)
|
||||
self._transition_state(AgentState.DONE)
|
||||
self.stats.end_time = time.time()
|
||||
tool_call_result_blocks.append(
|
||||
ToolCallMessageSegment(
|
||||
role="tool",
|
||||
tool_call_id=func_tool_id,
|
||||
content="*工具没有返回值或者将结果直接发送给了用户*",
|
||||
),
|
||||
)
|
||||
else:
|
||||
# 不应该出现其他类型
|
||||
logger.warning(
|
||||
f"Tool 返回了不支持的类型: {type(resp)}。",
|
||||
)
|
||||
tool_call_result_blocks.append(
|
||||
ToolCallMessageSegment(
|
||||
role="tool",
|
||||
tool_call_id=func_tool_id,
|
||||
content="*工具返回了不支持的类型,请告诉用户检查这个工具的定义和实现。*",
|
||||
),
|
||||
f"Tool 返回了不支持的类型: {type(resp)},将忽略。",
|
||||
)
|
||||
|
||||
try:
|
||||
@@ -515,22 +387,6 @@ class ToolLoopAgentRunner(BaseAgentRunner[TContext]):
|
||||
),
|
||||
)
|
||||
|
||||
# yield the last tool call result
|
||||
if tool_call_result_blocks:
|
||||
last_tcr_content = str(tool_call_result_blocks[-1].content)
|
||||
yield MessageChain(
|
||||
type="tool_call_result",
|
||||
chain=[
|
||||
Json(
|
||||
data={
|
||||
"id": func_tool_id,
|
||||
"ts": time.time(),
|
||||
"result": last_tcr_content,
|
||||
}
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
# 处理函数调用响应
|
||||
if tool_call_result_blocks:
|
||||
yield tool_call_result_blocks
|
||||
|
||||
@@ -6,10 +6,8 @@ from astrbot.core.platform.astr_message_event import AstrMessageEvent
|
||||
from astrbot.core.star.context import Context
|
||||
|
||||
|
||||
@dataclass
|
||||
@dataclass(config={"arbitrary_types_allowed": True})
|
||||
class AstrAgentContext:
|
||||
__pydantic_config__ = {"arbitrary_types_allowed": True}
|
||||
|
||||
context: Context
|
||||
"""The star context instance"""
|
||||
event: AstrMessageEvent
|
||||
|
||||
@@ -13,12 +13,6 @@ from astrbot.core.star.star_handler import EventType
|
||||
class MainAgentHooks(BaseAgentRunHooks[AstrAgentContext]):
|
||||
async def on_agent_done(self, run_context, llm_response):
|
||||
# 执行事件钩子
|
||||
if llm_response and llm_response.reasoning_content:
|
||||
# we will use this in result_decorate stage to inject reasoning content to chain
|
||||
run_context.context.event.set_extra(
|
||||
"_llm_reasoning_content", llm_response.reasoning_content
|
||||
)
|
||||
|
||||
await call_event_hook(
|
||||
run_context.context.event,
|
||||
EventType.OnLLMResponseEvent,
|
||||
|
||||
@@ -2,10 +2,8 @@ import traceback
|
||||
from collections.abc import AsyncGenerator
|
||||
|
||||
from astrbot.core import logger
|
||||
from astrbot.core.agent.message import Message
|
||||
from astrbot.core.agent.runners.tool_loop_agent_runner import ToolLoopAgentRunner
|
||||
from astrbot.core.astr_agent_context import AstrAgentContext
|
||||
from astrbot.core.message.components import Json
|
||||
from astrbot.core.message.message_event_result import (
|
||||
MessageChain,
|
||||
MessageEventResult,
|
||||
@@ -25,25 +23,8 @@ async def run_agent(
|
||||
) -> AsyncGenerator[MessageChain | None, None]:
|
||||
step_idx = 0
|
||||
astr_event = agent_runner.run_context.context.event
|
||||
while step_idx < max_step + 1:
|
||||
while step_idx < max_step:
|
||||
step_idx += 1
|
||||
|
||||
if step_idx == max_step + 1:
|
||||
logger.warning(
|
||||
f"Agent reached max steps ({max_step}), forcing a final response."
|
||||
)
|
||||
if not agent_runner.done():
|
||||
# 拔掉所有工具
|
||||
if agent_runner.req:
|
||||
agent_runner.req.func_tool = None
|
||||
# 注入提示词
|
||||
agent_runner.run_context.messages.append(
|
||||
Message(
|
||||
role="user",
|
||||
content="工具调用次数已达到上限,请停止使用工具,并根据已经收集到的信息,对你的任务和发现进行总结,然后直接回复用户。",
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
async for resp in agent_runner.step():
|
||||
if astr_event.is_stopped():
|
||||
@@ -52,27 +33,16 @@ async def run_agent(
|
||||
msg_chain = resp.data["chain"]
|
||||
if msg_chain.type == "tool_direct_result":
|
||||
# tool_direct_result 用于标记 llm tool 需要直接发送给用户的内容
|
||||
await astr_event.send(msg_chain)
|
||||
await astr_event.send(resp.data["chain"])
|
||||
continue
|
||||
if astr_event.get_platform_id() == "webchat":
|
||||
await astr_event.send(msg_chain)
|
||||
# 对于其他情况,暂时先不处理
|
||||
continue
|
||||
elif resp.type == "tool_call":
|
||||
if agent_runner.streaming:
|
||||
# 用来标记流式响应需要分节
|
||||
yield MessageChain(chain=[], type="break")
|
||||
|
||||
if astr_event.get_platform_name() == "webchat":
|
||||
if show_tool_use:
|
||||
await astr_event.send(resp.data["chain"])
|
||||
elif show_tool_use:
|
||||
json_comp = resp.data["chain"].chain[0]
|
||||
if isinstance(json_comp, Json):
|
||||
m = f"🔨 调用工具: {json_comp.data.get('name')}"
|
||||
else:
|
||||
m = "🔨 调用工具..."
|
||||
chain = MessageChain(type="tool_call").message(m)
|
||||
await astr_event.send(chain)
|
||||
continue
|
||||
|
||||
if stream_to_general and resp.type == "streaming_delta":
|
||||
@@ -99,15 +69,6 @@ async def run_agent(
|
||||
continue
|
||||
yield resp.data["chain"] # MessageChain
|
||||
if agent_runner.done():
|
||||
# send agent stats to webchat
|
||||
if astr_event.get_platform_name() == "webchat":
|
||||
await astr_event.send(
|
||||
MessageChain(
|
||||
type="agent_stats",
|
||||
chain=[Json(data=agent_runner.stats.to_dict())],
|
||||
)
|
||||
)
|
||||
|
||||
break
|
||||
|
||||
except Exception as e:
|
||||
|
||||
@@ -209,42 +209,12 @@ async def call_local_llm_tool(
|
||||
else:
|
||||
raise ValueError(f"未知的方法名: {method_name}")
|
||||
except ValueError as e:
|
||||
raise Exception(f"Tool execution ValueError: {e}") from e
|
||||
except TypeError as e:
|
||||
# 获取函数的签名(包括类型),除了第一个 event/context 参数。
|
||||
try:
|
||||
sig = inspect.signature(handler)
|
||||
params = list(sig.parameters.values())
|
||||
# 跳过第一个参数(event 或 context)
|
||||
if params:
|
||||
params = params[1:]
|
||||
|
||||
param_strs = []
|
||||
for param in params:
|
||||
param_str = param.name
|
||||
if param.annotation != inspect.Parameter.empty:
|
||||
# 获取类型注解的字符串表示
|
||||
if isinstance(param.annotation, type):
|
||||
type_str = param.annotation.__name__
|
||||
else:
|
||||
type_str = str(param.annotation)
|
||||
param_str += f": {type_str}"
|
||||
if param.default != inspect.Parameter.empty:
|
||||
param_str += f" = {param.default!r}"
|
||||
param_strs.append(param_str)
|
||||
|
||||
handler_param_str = (
|
||||
", ".join(param_strs) if param_strs else "(no additional parameters)"
|
||||
)
|
||||
except Exception:
|
||||
handler_param_str = "(unable to inspect signature)"
|
||||
|
||||
raise Exception(
|
||||
f"Tool handler parameter mismatch, please check the handler definition. Handler parameters: {handler_param_str}"
|
||||
) from e
|
||||
logger.error(f"调用本地 LLM 工具时出错: {e}", exc_info=True)
|
||||
except TypeError:
|
||||
logger.error("处理函数参数不匹配,请检查 handler 的定义。", exc_info=True)
|
||||
except Exception as e:
|
||||
trace_ = traceback.format_exc()
|
||||
raise Exception(f"Tool execution error: {e}. Traceback: {trace_}") from e
|
||||
logger.error(f"调用本地 LLM 工具时出错: {e}\n{trace_}")
|
||||
|
||||
if not ready_to_call:
|
||||
return
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
"""AstrBot 备份与恢复模块
|
||||
|
||||
提供数据导出和导入功能,支持用户在服务器迁移时一键备份和恢复所有数据。
|
||||
"""
|
||||
|
||||
# 从 constants 模块导入共享常量
|
||||
from .constants import (
|
||||
BACKUP_MANIFEST_VERSION,
|
||||
KB_METADATA_MODELS,
|
||||
MAIN_DB_MODELS,
|
||||
get_backup_directories,
|
||||
)
|
||||
|
||||
# 导入导出器和导入器
|
||||
from .exporter import AstrBotExporter
|
||||
from .importer import AstrBotImporter, ImportPreCheckResult
|
||||
|
||||
__all__ = [
|
||||
"AstrBotExporter",
|
||||
"AstrBotImporter",
|
||||
"ImportPreCheckResult",
|
||||
"MAIN_DB_MODELS",
|
||||
"KB_METADATA_MODELS",
|
||||
"get_backup_directories",
|
||||
"BACKUP_MANIFEST_VERSION",
|
||||
]
|
||||
@@ -1,77 +0,0 @@
|
||||
"""AstrBot 备份模块共享常量
|
||||
|
||||
此文件定义了导出器和导入器共享的常量,确保两端配置一致。
|
||||
"""
|
||||
|
||||
from sqlmodel import SQLModel
|
||||
|
||||
from astrbot.core.db.po import (
|
||||
Attachment,
|
||||
CommandConfig,
|
||||
CommandConflict,
|
||||
ConversationV2,
|
||||
Persona,
|
||||
PlatformMessageHistory,
|
||||
PlatformSession,
|
||||
PlatformStat,
|
||||
Preference,
|
||||
)
|
||||
from astrbot.core.knowledge_base.models import (
|
||||
KBDocument,
|
||||
KBMedia,
|
||||
KnowledgeBase,
|
||||
)
|
||||
from astrbot.core.utils.astrbot_path import (
|
||||
get_astrbot_config_path,
|
||||
get_astrbot_plugin_data_path,
|
||||
get_astrbot_plugin_path,
|
||||
get_astrbot_t2i_templates_path,
|
||||
get_astrbot_temp_path,
|
||||
get_astrbot_webchat_path,
|
||||
)
|
||||
|
||||
# ============================================================
|
||||
# 共享常量 - 确保导出和导入端配置一致
|
||||
# ============================================================
|
||||
|
||||
# 主数据库模型类映射
|
||||
MAIN_DB_MODELS: dict[str, type[SQLModel]] = {
|
||||
"platform_stats": PlatformStat,
|
||||
"conversations": ConversationV2,
|
||||
"personas": Persona,
|
||||
"preferences": Preference,
|
||||
"platform_message_history": PlatformMessageHistory,
|
||||
"platform_sessions": PlatformSession,
|
||||
"attachments": Attachment,
|
||||
"command_configs": CommandConfig,
|
||||
"command_conflicts": CommandConflict,
|
||||
}
|
||||
|
||||
# 知识库元数据模型类映射
|
||||
KB_METADATA_MODELS: dict[str, type[SQLModel]] = {
|
||||
"knowledge_bases": KnowledgeBase,
|
||||
"kb_documents": KBDocument,
|
||||
"kb_media": KBMedia,
|
||||
}
|
||||
|
||||
|
||||
def get_backup_directories() -> dict[str, str]:
|
||||
"""获取需要备份的目录列表
|
||||
|
||||
使用 astrbot_path 模块动态获取路径,支持通过环境变量 ASTRBOT_ROOT 自定义根目录。
|
||||
|
||||
Returns:
|
||||
dict: 键为备份文件中的目录名称,值为目录的绝对路径
|
||||
"""
|
||||
return {
|
||||
"plugins": get_astrbot_plugin_path(), # 插件本体
|
||||
"plugin_data": get_astrbot_plugin_data_path(), # 插件数据
|
||||
"config": get_astrbot_config_path(), # 配置目录
|
||||
"t2i_templates": get_astrbot_t2i_templates_path(), # T2I 模板
|
||||
"webchat": get_astrbot_webchat_path(), # WebChat 数据
|
||||
"temp": get_astrbot_temp_path(), # 临时文件
|
||||
}
|
||||
|
||||
|
||||
# 备份清单版本号
|
||||
BACKUP_MANIFEST_VERSION = "1.1"
|
||||
@@ -1,477 +0,0 @@
|
||||
"""AstrBot 数据导出器
|
||||
|
||||
负责将所有数据导出为 ZIP 备份文件。
|
||||
导出格式为 JSON,这是数据库无关的方案,支持未来向 MySQL/PostgreSQL 迁移。
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import zipfile
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from astrbot.core import logger
|
||||
from astrbot.core.config.default import VERSION
|
||||
from astrbot.core.db import BaseDatabase
|
||||
from astrbot.core.utils.astrbot_path import (
|
||||
get_astrbot_backups_path,
|
||||
get_astrbot_data_path,
|
||||
)
|
||||
|
||||
# 从共享常量模块导入
|
||||
from .constants import (
|
||||
BACKUP_MANIFEST_VERSION,
|
||||
KB_METADATA_MODELS,
|
||||
MAIN_DB_MODELS,
|
||||
get_backup_directories,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from astrbot.core.knowledge_base.kb_mgr import KnowledgeBaseManager
|
||||
|
||||
CMD_CONFIG_FILE_PATH = os.path.join(get_astrbot_data_path(), "cmd_config.json")
|
||||
|
||||
|
||||
class AstrBotExporter:
|
||||
"""AstrBot 数据导出器
|
||||
|
||||
导出内容:
|
||||
- 主数据库所有表(data/data_v4.db)
|
||||
- 知识库元数据(data/knowledge_base/kb.db)
|
||||
- 每个知识库的向量文档数据
|
||||
- 配置文件(data/cmd_config.json)
|
||||
- 附件文件
|
||||
- 知识库多媒体文件
|
||||
- 插件目录(data/plugins)
|
||||
- 插件数据目录(data/plugin_data)
|
||||
- 配置目录(data/config)
|
||||
- T2I 模板目录(data/t2i_templates)
|
||||
- WebChat 数据目录(data/webchat)
|
||||
- 临时文件目录(data/temp)
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
main_db: BaseDatabase,
|
||||
kb_manager: "KnowledgeBaseManager | None" = None,
|
||||
config_path: str = CMD_CONFIG_FILE_PATH,
|
||||
):
|
||||
self.main_db = main_db
|
||||
self.kb_manager = kb_manager
|
||||
self.config_path = config_path
|
||||
self._checksums: dict[str, str] = {}
|
||||
|
||||
async def export_all(
|
||||
self,
|
||||
output_dir: str | None = None,
|
||||
progress_callback: Any | None = None,
|
||||
) -> str:
|
||||
"""导出所有数据到 ZIP 文件
|
||||
|
||||
Args:
|
||||
output_dir: 输出目录
|
||||
progress_callback: 进度回调函数,接收参数 (stage, current, total, message)
|
||||
|
||||
Returns:
|
||||
str: 生成的 ZIP 文件路径
|
||||
"""
|
||||
if output_dir is None:
|
||||
output_dir = get_astrbot_backups_path()
|
||||
|
||||
# 确保输出目录存在
|
||||
Path(output_dir).mkdir(parents=True, exist_ok=True)
|
||||
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
zip_filename = f"astrbot_backup_{timestamp}.zip"
|
||||
zip_path = os.path.join(output_dir, zip_filename)
|
||||
|
||||
logger.info(f"开始导出备份到 {zip_path}")
|
||||
|
||||
try:
|
||||
with zipfile.ZipFile(zip_path, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
# 1. 导出主数据库
|
||||
if progress_callback:
|
||||
await progress_callback("main_db", 0, 100, "正在导出主数据库...")
|
||||
main_data = await self._export_main_database()
|
||||
main_db_json = json.dumps(
|
||||
main_data, ensure_ascii=False, indent=2, default=str
|
||||
)
|
||||
zf.writestr("databases/main_db.json", main_db_json)
|
||||
self._add_checksum("databases/main_db.json", main_db_json)
|
||||
if progress_callback:
|
||||
await progress_callback("main_db", 100, 100, "主数据库导出完成")
|
||||
|
||||
# 2. 导出知识库数据
|
||||
kb_meta_data: dict[str, Any] = {
|
||||
"knowledge_bases": [],
|
||||
"kb_documents": [],
|
||||
"kb_media": [],
|
||||
}
|
||||
if self.kb_manager:
|
||||
if progress_callback:
|
||||
await progress_callback(
|
||||
"kb_metadata", 0, 100, "正在导出知识库元数据..."
|
||||
)
|
||||
kb_meta_data = await self._export_kb_metadata()
|
||||
kb_meta_json = json.dumps(
|
||||
kb_meta_data, ensure_ascii=False, indent=2, default=str
|
||||
)
|
||||
zf.writestr("databases/kb_metadata.json", kb_meta_json)
|
||||
self._add_checksum("databases/kb_metadata.json", kb_meta_json)
|
||||
if progress_callback:
|
||||
await progress_callback(
|
||||
"kb_metadata", 100, 100, "知识库元数据导出完成"
|
||||
)
|
||||
|
||||
# 导出每个知识库的文档数据
|
||||
kb_insts = self.kb_manager.kb_insts
|
||||
total_kbs = len(kb_insts)
|
||||
for idx, (kb_id, kb_helper) in enumerate(kb_insts.items()):
|
||||
if progress_callback:
|
||||
await progress_callback(
|
||||
"kb_documents",
|
||||
idx,
|
||||
total_kbs,
|
||||
f"正在导出知识库 {kb_helper.kb.kb_name} 的文档数据...",
|
||||
)
|
||||
doc_data = await self._export_kb_documents(kb_helper)
|
||||
doc_json = json.dumps(
|
||||
doc_data, ensure_ascii=False, indent=2, default=str
|
||||
)
|
||||
doc_path = f"databases/kb_{kb_id}/documents.json"
|
||||
zf.writestr(doc_path, doc_json)
|
||||
self._add_checksum(doc_path, doc_json)
|
||||
|
||||
# 导出 FAISS 索引文件
|
||||
await self._export_faiss_index(zf, kb_helper, kb_id)
|
||||
|
||||
# 导出知识库多媒体文件
|
||||
await self._export_kb_media_files(zf, kb_helper, kb_id)
|
||||
|
||||
if progress_callback:
|
||||
await progress_callback(
|
||||
"kb_documents", total_kbs, total_kbs, "知识库文档导出完成"
|
||||
)
|
||||
|
||||
# 3. 导出配置文件
|
||||
if progress_callback:
|
||||
await progress_callback("config", 0, 100, "正在导出配置文件...")
|
||||
if os.path.exists(self.config_path):
|
||||
with open(self.config_path, encoding="utf-8") as f:
|
||||
config_content = f.read()
|
||||
zf.writestr("config/cmd_config.json", config_content)
|
||||
self._add_checksum("config/cmd_config.json", config_content)
|
||||
if progress_callback:
|
||||
await progress_callback("config", 100, 100, "配置文件导出完成")
|
||||
|
||||
# 4. 导出附件文件
|
||||
if progress_callback:
|
||||
await progress_callback("attachments", 0, 100, "正在导出附件...")
|
||||
await self._export_attachments(zf, main_data.get("attachments", []))
|
||||
if progress_callback:
|
||||
await progress_callback("attachments", 100, 100, "附件导出完成")
|
||||
|
||||
# 5. 导出插件和其他目录
|
||||
if progress_callback:
|
||||
await progress_callback(
|
||||
"directories", 0, 100, "正在导出插件和数据目录..."
|
||||
)
|
||||
dir_stats = await self._export_directories(zf)
|
||||
if progress_callback:
|
||||
await progress_callback("directories", 100, 100, "目录导出完成")
|
||||
|
||||
# 6. 生成 manifest
|
||||
if progress_callback:
|
||||
await progress_callback("manifest", 0, 100, "正在生成清单...")
|
||||
manifest = self._generate_manifest(main_data, kb_meta_data, dir_stats)
|
||||
manifest_json = json.dumps(manifest, ensure_ascii=False, indent=2)
|
||||
zf.writestr("manifest.json", manifest_json)
|
||||
if progress_callback:
|
||||
await progress_callback("manifest", 100, 100, "清单生成完成")
|
||||
|
||||
logger.info(f"备份导出完成: {zip_path}")
|
||||
return zip_path
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"备份导出失败: {e}")
|
||||
# 清理失败的文件
|
||||
if os.path.exists(zip_path):
|
||||
os.remove(zip_path)
|
||||
raise
|
||||
|
||||
async def _export_main_database(self) -> dict[str, list[dict]]:
|
||||
"""导出主数据库所有表"""
|
||||
export_data: dict[str, list[dict]] = {}
|
||||
|
||||
async with self.main_db.get_db() as session:
|
||||
for table_name, model_class in MAIN_DB_MODELS.items():
|
||||
try:
|
||||
result = await session.execute(select(model_class))
|
||||
records = result.scalars().all()
|
||||
export_data[table_name] = [
|
||||
self._model_to_dict(record) for record in records
|
||||
]
|
||||
logger.debug(
|
||||
f"导出表 {table_name}: {len(export_data[table_name])} 条记录"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"导出表 {table_name} 失败: {e}")
|
||||
export_data[table_name] = []
|
||||
|
||||
return export_data
|
||||
|
||||
async def _export_kb_metadata(self) -> dict[str, list[dict]]:
|
||||
"""导出知识库元数据库"""
|
||||
if not self.kb_manager:
|
||||
return {"knowledge_bases": [], "kb_documents": [], "kb_media": []}
|
||||
|
||||
export_data: dict[str, list[dict]] = {}
|
||||
|
||||
async with self.kb_manager.kb_db.get_db() as session:
|
||||
for table_name, model_class in KB_METADATA_MODELS.items():
|
||||
try:
|
||||
result = await session.execute(select(model_class))
|
||||
records = result.scalars().all()
|
||||
export_data[table_name] = [
|
||||
self._model_to_dict(record) for record in records
|
||||
]
|
||||
logger.debug(
|
||||
f"导出知识库表 {table_name}: {len(export_data[table_name])} 条记录"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"导出知识库表 {table_name} 失败: {e}")
|
||||
export_data[table_name] = []
|
||||
|
||||
return export_data
|
||||
|
||||
async def _export_kb_documents(self, kb_helper: Any) -> dict[str, Any]:
|
||||
"""导出知识库的文档块数据"""
|
||||
try:
|
||||
from astrbot.core.db.vec_db.faiss_impl.vec_db import FaissVecDB
|
||||
|
||||
vec_db: FaissVecDB = kb_helper.vec_db
|
||||
if not vec_db or not vec_db.document_storage:
|
||||
return {"documents": []}
|
||||
|
||||
# 获取所有文档
|
||||
docs = await vec_db.document_storage.get_documents(
|
||||
metadata_filters={},
|
||||
offset=0,
|
||||
limit=None, # 获取全部
|
||||
)
|
||||
|
||||
return {"documents": docs}
|
||||
except Exception as e:
|
||||
logger.warning(f"导出知识库文档失败: {e}")
|
||||
return {"documents": []}
|
||||
|
||||
async def _export_faiss_index(
|
||||
self,
|
||||
zf: zipfile.ZipFile,
|
||||
kb_helper: Any,
|
||||
kb_id: str,
|
||||
) -> None:
|
||||
"""导出 FAISS 索引文件"""
|
||||
try:
|
||||
index_path = kb_helper.kb_dir / "index.faiss"
|
||||
if index_path.exists():
|
||||
archive_path = f"databases/kb_{kb_id}/index.faiss"
|
||||
zf.write(str(index_path), archive_path)
|
||||
logger.debug(f"导出 FAISS 索引: {archive_path}")
|
||||
except Exception as e:
|
||||
logger.warning(f"导出 FAISS 索引失败: {e}")
|
||||
|
||||
async def _export_kb_media_files(
|
||||
self, zf: zipfile.ZipFile, kb_helper: Any, kb_id: str
|
||||
) -> None:
|
||||
"""导出知识库的多媒体文件"""
|
||||
try:
|
||||
media_dir = kb_helper.kb_medias_dir
|
||||
if not media_dir.exists():
|
||||
return
|
||||
|
||||
for root, _, files in os.walk(media_dir):
|
||||
for file in files:
|
||||
file_path = Path(root) / file
|
||||
# 计算相对路径
|
||||
rel_path = file_path.relative_to(kb_helper.kb_dir)
|
||||
archive_path = f"files/kb_media/{kb_id}/{rel_path}"
|
||||
zf.write(str(file_path), archive_path)
|
||||
except Exception as e:
|
||||
logger.warning(f"导出知识库媒体文件失败: {e}")
|
||||
|
||||
async def _export_directories(
|
||||
self, zf: zipfile.ZipFile
|
||||
) -> dict[str, dict[str, int]]:
|
||||
"""导出插件和其他数据目录
|
||||
|
||||
Returns:
|
||||
dict: 每个目录的统计信息 {dir_name: {"files": count, "size": bytes}}
|
||||
"""
|
||||
stats: dict[str, dict[str, int]] = {}
|
||||
backup_directories = get_backup_directories()
|
||||
|
||||
for dir_name, dir_path in backup_directories.items():
|
||||
full_path = Path(dir_path)
|
||||
if not full_path.exists():
|
||||
logger.debug(f"目录不存在,跳过: {full_path}")
|
||||
continue
|
||||
|
||||
file_count = 0
|
||||
total_size = 0
|
||||
|
||||
try:
|
||||
for root, dirs, files in os.walk(full_path):
|
||||
# 跳过 __pycache__ 目录
|
||||
dirs[:] = [d for d in dirs if d != "__pycache__"]
|
||||
|
||||
for file in files:
|
||||
# 跳过 .pyc 文件
|
||||
if file.endswith(".pyc"):
|
||||
continue
|
||||
|
||||
file_path = Path(root) / file
|
||||
try:
|
||||
# 计算相对路径
|
||||
rel_path = file_path.relative_to(full_path)
|
||||
archive_path = f"directories/{dir_name}/{rel_path}"
|
||||
zf.write(str(file_path), archive_path)
|
||||
file_count += 1
|
||||
total_size += file_path.stat().st_size
|
||||
except Exception as e:
|
||||
logger.warning(f"导出文件 {file_path} 失败: {e}")
|
||||
|
||||
stats[dir_name] = {"files": file_count, "size": total_size}
|
||||
logger.debug(
|
||||
f"导出目录 {dir_name}: {file_count} 个文件, {total_size} 字节"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"导出目录 {dir_path} 失败: {e}")
|
||||
stats[dir_name] = {"files": 0, "size": 0}
|
||||
|
||||
return stats
|
||||
|
||||
async def _export_attachments(
|
||||
self, zf: zipfile.ZipFile, attachments: list[dict]
|
||||
) -> None:
|
||||
"""导出附件文件"""
|
||||
for attachment in attachments:
|
||||
try:
|
||||
file_path = attachment.get("path", "")
|
||||
if file_path and os.path.exists(file_path):
|
||||
# 使用 attachment_id 作为文件名
|
||||
attachment_id = attachment.get("attachment_id", "")
|
||||
ext = os.path.splitext(file_path)[1]
|
||||
archive_path = f"files/attachments/{attachment_id}{ext}"
|
||||
zf.write(file_path, archive_path)
|
||||
except Exception as e:
|
||||
logger.warning(f"导出附件失败: {e}")
|
||||
|
||||
def _model_to_dict(self, record: Any) -> dict:
|
||||
"""将 SQLModel 实例转换为字典
|
||||
|
||||
这是数据库无关的序列化方式,支持未来迁移到其他数据库。
|
||||
"""
|
||||
# 使用 SQLModel 内置的 model_dump 方法(如果可用)
|
||||
if hasattr(record, "model_dump"):
|
||||
data = record.model_dump(mode="python")
|
||||
# 处理 datetime 类型
|
||||
for key, value in data.items():
|
||||
if isinstance(value, datetime):
|
||||
data[key] = value.isoformat()
|
||||
return data
|
||||
|
||||
# 回退到手动提取
|
||||
data = {}
|
||||
# 使用 inspect 获取表信息
|
||||
from sqlalchemy import inspect as sa_inspect
|
||||
|
||||
mapper = sa_inspect(record.__class__)
|
||||
for column in mapper.columns:
|
||||
value = getattr(record, column.name)
|
||||
# 处理 datetime 类型 - 统一转为 ISO 格式字符串
|
||||
if isinstance(value, datetime):
|
||||
value = value.isoformat()
|
||||
data[column.name] = value
|
||||
return data
|
||||
|
||||
def _add_checksum(self, path: str, content: str | bytes) -> None:
|
||||
"""计算并添加文件校验和"""
|
||||
if isinstance(content, str):
|
||||
content = content.encode("utf-8")
|
||||
checksum = hashlib.sha256(content).hexdigest()
|
||||
self._checksums[path] = f"sha256:{checksum}"
|
||||
|
||||
def _generate_manifest(
|
||||
self,
|
||||
main_data: dict[str, list[dict]],
|
||||
kb_meta_data: dict[str, list[dict]],
|
||||
dir_stats: dict[str, dict[str, int]] | None = None,
|
||||
) -> dict:
|
||||
"""生成备份清单"""
|
||||
if dir_stats is None:
|
||||
dir_stats = {}
|
||||
# 收集知识库 ID
|
||||
kb_document_tables = {}
|
||||
if self.kb_manager:
|
||||
for kb_id in self.kb_manager.kb_insts.keys():
|
||||
kb_document_tables[kb_id] = "documents"
|
||||
|
||||
# 收集附件文件列表
|
||||
attachment_files = []
|
||||
for attachment in main_data.get("attachments", []):
|
||||
attachment_id = attachment.get("attachment_id", "")
|
||||
path = attachment.get("path", "")
|
||||
if attachment_id and path:
|
||||
ext = os.path.splitext(path)[1]
|
||||
attachment_files.append(f"{attachment_id}{ext}")
|
||||
|
||||
# 收集知识库媒体文件
|
||||
kb_media_files: dict[str, list[str]] = {}
|
||||
if self.kb_manager:
|
||||
for kb_id, kb_helper in self.kb_manager.kb_insts.items():
|
||||
media_files: list[str] = []
|
||||
media_dir = kb_helper.kb_medias_dir
|
||||
if media_dir.exists():
|
||||
for root, _, files in os.walk(media_dir):
|
||||
for file in files:
|
||||
media_files.append(file)
|
||||
if media_files:
|
||||
kb_media_files[kb_id] = media_files
|
||||
|
||||
manifest = {
|
||||
"version": BACKUP_MANIFEST_VERSION,
|
||||
"astrbot_version": VERSION,
|
||||
"exported_at": datetime.now(timezone.utc).isoformat(),
|
||||
"origin": "exported", # 标记备份来源:exported=本实例导出, uploaded=用户上传
|
||||
"schema_version": {
|
||||
"main_db": "v4",
|
||||
"kb_db": "v1",
|
||||
},
|
||||
"tables": {
|
||||
"main_db": list(main_data.keys()),
|
||||
"kb_metadata": list(kb_meta_data.keys()),
|
||||
"kb_documents": kb_document_tables,
|
||||
},
|
||||
"files": {
|
||||
"attachments": attachment_files,
|
||||
"kb_media": kb_media_files,
|
||||
},
|
||||
"directories": list(dir_stats.keys()),
|
||||
"checksums": self._checksums,
|
||||
"statistics": {
|
||||
"main_db": {
|
||||
table: len(records) for table, records in main_data.items()
|
||||
},
|
||||
"kb_metadata": {
|
||||
table: len(records) for table, records in kb_meta_data.items()
|
||||
},
|
||||
"directories": dir_stats,
|
||||
},
|
||||
}
|
||||
|
||||
return manifest
|
||||
@@ -1,761 +0,0 @@
|
||||
"""AstrBot 数据导入器
|
||||
|
||||
负责从 ZIP 备份文件恢复所有数据。
|
||||
导入时进行版本校验:
|
||||
- 主版本(前两位)不同时直接拒绝导入
|
||||
- 小版本(第三位)不同时提示警告,用户可选择强制导入
|
||||
- 版本匹配时也需要用户确认
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import zipfile
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from sqlalchemy import delete
|
||||
|
||||
from astrbot.core import logger
|
||||
from astrbot.core.config.default import VERSION
|
||||
from astrbot.core.db import BaseDatabase
|
||||
from astrbot.core.utils.astrbot_path import (
|
||||
get_astrbot_data_path,
|
||||
get_astrbot_knowledge_base_path,
|
||||
)
|
||||
from astrbot.core.utils.version_comparator import VersionComparator
|
||||
|
||||
# 从共享常量模块导入
|
||||
from .constants import (
|
||||
KB_METADATA_MODELS,
|
||||
MAIN_DB_MODELS,
|
||||
get_backup_directories,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from astrbot.core.knowledge_base.kb_mgr import KnowledgeBaseManager
|
||||
|
||||
|
||||
def _get_major_version(version_str: str) -> str:
|
||||
"""提取版本的主版本部分(前两位)
|
||||
|
||||
Args:
|
||||
version_str: 版本字符串,如 "4.9.1", "4.10.0-beta"
|
||||
|
||||
Returns:
|
||||
主版本字符串,如 "4.9", "4.10"
|
||||
"""
|
||||
if not version_str:
|
||||
return "0.0"
|
||||
# 移除 v 前缀和预发布标签
|
||||
version = version_str.lower().replace("v", "").split("-")[0].split("+")[0]
|
||||
parts = [p for p in version.split(".") if p] # 过滤空字符串
|
||||
if len(parts) >= 2:
|
||||
return f"{parts[0]}.{parts[1]}"
|
||||
elif len(parts) == 1 and parts[0]:
|
||||
return f"{parts[0]}.0"
|
||||
return "0.0"
|
||||
|
||||
|
||||
CMD_CONFIG_FILE_PATH = os.path.join(get_astrbot_data_path(), "cmd_config.json")
|
||||
KB_PATH = get_astrbot_knowledge_base_path()
|
||||
|
||||
|
||||
@dataclass
|
||||
class ImportPreCheckResult:
|
||||
"""导入预检查结果
|
||||
|
||||
用于在实际导入前检查备份文件的版本兼容性,
|
||||
并返回确认信息让用户决定是否继续导入。
|
||||
"""
|
||||
|
||||
# 检查是否通过(文件有效且版本可导入)
|
||||
valid: bool = False
|
||||
# 是否可以导入(版本兼容)
|
||||
can_import: bool = False
|
||||
# 版本状态: match(完全匹配), minor_diff(小版本差异), major_diff(主版本不同,拒绝)
|
||||
version_status: str = ""
|
||||
# 备份文件中的 AstrBot 版本
|
||||
backup_version: str = ""
|
||||
# 当前运行的 AstrBot 版本
|
||||
current_version: str = VERSION
|
||||
# 备份创建时间
|
||||
backup_time: str = ""
|
||||
# 确认消息(显示给用户)
|
||||
confirm_message: str = ""
|
||||
# 警告消息列表
|
||||
warnings: list[str] = field(default_factory=list)
|
||||
# 错误消息(如果检查失败)
|
||||
error: str = ""
|
||||
# 备份包含的内容摘要
|
||||
backup_summary: dict = field(default_factory=dict)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"valid": self.valid,
|
||||
"can_import": self.can_import,
|
||||
"version_status": self.version_status,
|
||||
"backup_version": self.backup_version,
|
||||
"current_version": self.current_version,
|
||||
"backup_time": self.backup_time,
|
||||
"confirm_message": self.confirm_message,
|
||||
"warnings": self.warnings,
|
||||
"error": self.error,
|
||||
"backup_summary": self.backup_summary,
|
||||
}
|
||||
|
||||
|
||||
class ImportResult:
|
||||
"""导入结果"""
|
||||
|
||||
def __init__(self):
|
||||
self.success = True
|
||||
self.imported_tables: dict[str, int] = {}
|
||||
self.imported_files: dict[str, int] = {}
|
||||
self.imported_directories: dict[str, int] = {}
|
||||
self.warnings: list[str] = []
|
||||
self.errors: list[str] = []
|
||||
|
||||
def add_warning(self, msg: str) -> None:
|
||||
self.warnings.append(msg)
|
||||
logger.warning(msg)
|
||||
|
||||
def add_error(self, msg: str) -> None:
|
||||
self.errors.append(msg)
|
||||
self.success = False
|
||||
logger.error(msg)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"success": self.success,
|
||||
"imported_tables": self.imported_tables,
|
||||
"imported_files": self.imported_files,
|
||||
"imported_directories": self.imported_directories,
|
||||
"warnings": self.warnings,
|
||||
"errors": self.errors,
|
||||
}
|
||||
|
||||
|
||||
class AstrBotImporter:
|
||||
"""AstrBot 数据导入器
|
||||
|
||||
导入备份文件中的所有数据,包括:
|
||||
- 主数据库所有表
|
||||
- 知识库元数据和文档
|
||||
- 配置文件
|
||||
- 附件文件
|
||||
- 知识库多媒体文件
|
||||
- 插件目录(data/plugins)
|
||||
- 插件数据目录(data/plugin_data)
|
||||
- 配置目录(data/config)
|
||||
- T2I 模板目录(data/t2i_templates)
|
||||
- WebChat 数据目录(data/webchat)
|
||||
- 临时文件目录(data/temp)
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
main_db: BaseDatabase,
|
||||
kb_manager: "KnowledgeBaseManager | None" = None,
|
||||
config_path: str = CMD_CONFIG_FILE_PATH,
|
||||
kb_root_dir: str = KB_PATH,
|
||||
):
|
||||
self.main_db = main_db
|
||||
self.kb_manager = kb_manager
|
||||
self.config_path = config_path
|
||||
self.kb_root_dir = kb_root_dir
|
||||
|
||||
def pre_check(self, zip_path: str) -> ImportPreCheckResult:
|
||||
"""预检查备份文件
|
||||
|
||||
在实际导入前检查备份文件的有效性和版本兼容性。
|
||||
返回检查结果供前端显示确认对话框。
|
||||
|
||||
Args:
|
||||
zip_path: ZIP 备份文件路径
|
||||
|
||||
Returns:
|
||||
ImportPreCheckResult: 预检查结果
|
||||
"""
|
||||
result = ImportPreCheckResult()
|
||||
result.current_version = VERSION
|
||||
|
||||
if not os.path.exists(zip_path):
|
||||
result.error = f"备份文件不存在: {zip_path}"
|
||||
return result
|
||||
|
||||
try:
|
||||
with zipfile.ZipFile(zip_path, "r") as zf:
|
||||
# 读取 manifest
|
||||
try:
|
||||
manifest_data = zf.read("manifest.json")
|
||||
manifest = json.loads(manifest_data)
|
||||
except KeyError:
|
||||
result.error = "备份文件缺少 manifest.json,不是有效的 AstrBot 备份"
|
||||
return result
|
||||
except json.JSONDecodeError as e:
|
||||
result.error = f"manifest.json 格式错误: {e}"
|
||||
return result
|
||||
|
||||
# 提取基本信息
|
||||
result.backup_version = manifest.get("astrbot_version", "未知")
|
||||
result.backup_time = manifest.get("exported_at", "未知")
|
||||
result.valid = True
|
||||
|
||||
# 构建备份摘要
|
||||
result.backup_summary = {
|
||||
"tables": list(manifest.get("tables", {}).keys()),
|
||||
"has_knowledge_bases": manifest.get("has_knowledge_bases", False),
|
||||
"has_config": manifest.get("has_config", False),
|
||||
"directories": manifest.get("directories", []),
|
||||
}
|
||||
|
||||
# 检查版本兼容性
|
||||
version_check = self._check_version_compatibility(result.backup_version)
|
||||
result.version_status = version_check["status"]
|
||||
result.can_import = version_check["can_import"]
|
||||
|
||||
# 版本信息由前端根据 version_status 和 i18n 生成显示
|
||||
# 不再将版本消息添加到 warnings 列表中,避免中文硬编码
|
||||
# warnings 列表保留用于其他非版本相关的警告
|
||||
|
||||
return result
|
||||
|
||||
except zipfile.BadZipFile:
|
||||
result.error = "无效的 ZIP 文件"
|
||||
return result
|
||||
except Exception as e:
|
||||
result.error = f"检查备份文件失败: {e}"
|
||||
return result
|
||||
|
||||
def _check_version_compatibility(self, backup_version: str) -> dict:
|
||||
"""检查版本兼容性
|
||||
|
||||
规则:
|
||||
- 主版本(前两位,如 4.9)必须一致,否则拒绝
|
||||
- 小版本(第三位,如 4.9.1 vs 4.9.2)不同时,警告但允许导入
|
||||
|
||||
Returns:
|
||||
dict: {status, can_import, message}
|
||||
"""
|
||||
if not backup_version:
|
||||
return {
|
||||
"status": "major_diff",
|
||||
"can_import": False,
|
||||
"message": "备份文件缺少版本信息",
|
||||
}
|
||||
|
||||
# 提取主版本(前两位)进行比较
|
||||
backup_major = _get_major_version(backup_version)
|
||||
current_major = _get_major_version(VERSION)
|
||||
|
||||
# 比较主版本
|
||||
if VersionComparator.compare_version(backup_major, current_major) != 0:
|
||||
return {
|
||||
"status": "major_diff",
|
||||
"can_import": False,
|
||||
"message": (
|
||||
f"主版本不兼容: 备份版本 {backup_version}, 当前版本 {VERSION}。"
|
||||
f"跨主版本导入可能导致数据损坏,请使用相同主版本的 AstrBot。"
|
||||
),
|
||||
}
|
||||
|
||||
# 比较完整版本
|
||||
version_cmp = VersionComparator.compare_version(backup_version, VERSION)
|
||||
if version_cmp != 0:
|
||||
return {
|
||||
"status": "minor_diff",
|
||||
"can_import": True,
|
||||
"message": (
|
||||
f"小版本差异: 备份版本 {backup_version}, 当前版本 {VERSION}。"
|
||||
),
|
||||
}
|
||||
|
||||
return {
|
||||
"status": "match",
|
||||
"can_import": True,
|
||||
"message": "版本匹配",
|
||||
}
|
||||
|
||||
async def import_all(
|
||||
self,
|
||||
zip_path: str,
|
||||
mode: str = "replace", # "replace" 清空后导入
|
||||
progress_callback: Any | None = None,
|
||||
) -> ImportResult:
|
||||
"""从 ZIP 文件导入所有数据
|
||||
|
||||
Args:
|
||||
zip_path: ZIP 备份文件路径
|
||||
mode: 导入模式,目前仅支持 "replace"(清空后导入)
|
||||
progress_callback: 进度回调函数,接收参数 (stage, current, total, message)
|
||||
|
||||
Returns:
|
||||
ImportResult: 导入结果
|
||||
"""
|
||||
result = ImportResult()
|
||||
|
||||
if not os.path.exists(zip_path):
|
||||
result.add_error(f"备份文件不存在: {zip_path}")
|
||||
return result
|
||||
|
||||
logger.info(f"开始从 {zip_path} 导入备份")
|
||||
|
||||
try:
|
||||
with zipfile.ZipFile(zip_path, "r") as zf:
|
||||
# 1. 读取并验证 manifest
|
||||
if progress_callback:
|
||||
await progress_callback("validate", 0, 100, "正在验证备份文件...")
|
||||
|
||||
try:
|
||||
manifest_data = zf.read("manifest.json")
|
||||
manifest = json.loads(manifest_data)
|
||||
except KeyError:
|
||||
result.add_error("备份文件缺少 manifest.json")
|
||||
return result
|
||||
except json.JSONDecodeError as e:
|
||||
result.add_error(f"manifest.json 格式错误: {e}")
|
||||
return result
|
||||
|
||||
# 版本校验
|
||||
try:
|
||||
self._validate_version(manifest)
|
||||
except ValueError as e:
|
||||
result.add_error(str(e))
|
||||
return result
|
||||
|
||||
if progress_callback:
|
||||
await progress_callback("validate", 100, 100, "验证完成")
|
||||
|
||||
# 2. 导入主数据库
|
||||
if progress_callback:
|
||||
await progress_callback("main_db", 0, 100, "正在导入主数据库...")
|
||||
|
||||
try:
|
||||
main_data_content = zf.read("databases/main_db.json")
|
||||
main_data = json.loads(main_data_content)
|
||||
|
||||
if mode == "replace":
|
||||
await self._clear_main_db()
|
||||
|
||||
imported = await self._import_main_database(main_data)
|
||||
result.imported_tables.update(imported)
|
||||
except Exception as e:
|
||||
result.add_error(f"导入主数据库失败: {e}")
|
||||
return result
|
||||
|
||||
if progress_callback:
|
||||
await progress_callback("main_db", 100, 100, "主数据库导入完成")
|
||||
|
||||
# 3. 导入知识库
|
||||
if self.kb_manager and "databases/kb_metadata.json" in zf.namelist():
|
||||
if progress_callback:
|
||||
await progress_callback("kb", 0, 100, "正在导入知识库...")
|
||||
|
||||
try:
|
||||
kb_meta_content = zf.read("databases/kb_metadata.json")
|
||||
kb_meta_data = json.loads(kb_meta_content)
|
||||
|
||||
if mode == "replace":
|
||||
await self._clear_kb_data()
|
||||
|
||||
await self._import_knowledge_bases(zf, kb_meta_data, result)
|
||||
except Exception as e:
|
||||
result.add_warning(f"导入知识库失败: {e}")
|
||||
|
||||
if progress_callback:
|
||||
await progress_callback("kb", 100, 100, "知识库导入完成")
|
||||
|
||||
# 4. 导入配置文件
|
||||
if progress_callback:
|
||||
await progress_callback("config", 0, 100, "正在导入配置文件...")
|
||||
|
||||
if "config/cmd_config.json" in zf.namelist():
|
||||
try:
|
||||
config_content = zf.read("config/cmd_config.json")
|
||||
# 备份现有配置
|
||||
if os.path.exists(self.config_path):
|
||||
backup_path = f"{self.config_path}.bak"
|
||||
shutil.copy2(self.config_path, backup_path)
|
||||
|
||||
with open(self.config_path, "wb") as f:
|
||||
f.write(config_content)
|
||||
result.imported_files["config"] = 1
|
||||
except Exception as e:
|
||||
result.add_warning(f"导入配置文件失败: {e}")
|
||||
|
||||
if progress_callback:
|
||||
await progress_callback("config", 100, 100, "配置文件导入完成")
|
||||
|
||||
# 5. 导入附件文件
|
||||
if progress_callback:
|
||||
await progress_callback("attachments", 0, 100, "正在导入附件...")
|
||||
|
||||
attachment_count = await self._import_attachments(
|
||||
zf, main_data.get("attachments", [])
|
||||
)
|
||||
result.imported_files["attachments"] = attachment_count
|
||||
|
||||
if progress_callback:
|
||||
await progress_callback("attachments", 100, 100, "附件导入完成")
|
||||
|
||||
# 6. 导入插件和其他目录
|
||||
if progress_callback:
|
||||
await progress_callback(
|
||||
"directories", 0, 100, "正在导入插件和数据目录..."
|
||||
)
|
||||
|
||||
dir_stats = await self._import_directories(zf, manifest, result)
|
||||
result.imported_directories = dir_stats
|
||||
|
||||
if progress_callback:
|
||||
await progress_callback("directories", 100, 100, "目录导入完成")
|
||||
|
||||
logger.info(f"备份导入完成: {result.to_dict()}")
|
||||
return result
|
||||
|
||||
except zipfile.BadZipFile:
|
||||
result.add_error("无效的 ZIP 文件")
|
||||
return result
|
||||
except Exception as e:
|
||||
result.add_error(f"导入失败: {e}")
|
||||
return result
|
||||
|
||||
def _validate_version(self, manifest: dict) -> None:
|
||||
"""验证版本兼容性 - 仅允许相同主版本导入
|
||||
|
||||
注意:此方法仅在 import_all 中调用,用于双重校验。
|
||||
前端应先调用 pre_check 获取详细的版本信息并让用户确认。
|
||||
"""
|
||||
backup_version = manifest.get("astrbot_version")
|
||||
if not backup_version:
|
||||
raise ValueError("备份文件缺少版本信息")
|
||||
|
||||
# 使用新的版本兼容性检查
|
||||
version_check = self._check_version_compatibility(backup_version)
|
||||
|
||||
if version_check["status"] == "major_diff":
|
||||
raise ValueError(version_check["message"])
|
||||
|
||||
# minor_diff 和 match 都允许导入
|
||||
if version_check["status"] == "minor_diff":
|
||||
logger.warning(f"版本差异警告: {version_check['message']}")
|
||||
|
||||
async def _clear_main_db(self) -> None:
|
||||
"""清空主数据库所有表"""
|
||||
async with self.main_db.get_db() as session:
|
||||
async with session.begin():
|
||||
for table_name, model_class in MAIN_DB_MODELS.items():
|
||||
try:
|
||||
await session.execute(delete(model_class))
|
||||
logger.debug(f"已清空表 {table_name}")
|
||||
except Exception as e:
|
||||
logger.warning(f"清空表 {table_name} 失败: {e}")
|
||||
|
||||
async def _clear_kb_data(self) -> None:
|
||||
"""清空知识库数据"""
|
||||
if not self.kb_manager:
|
||||
return
|
||||
|
||||
# 清空知识库元数据表
|
||||
async with self.kb_manager.kb_db.get_db() as session:
|
||||
async with session.begin():
|
||||
for table_name, model_class in KB_METADATA_MODELS.items():
|
||||
try:
|
||||
await session.execute(delete(model_class))
|
||||
logger.debug(f"已清空知识库表 {table_name}")
|
||||
except Exception as e:
|
||||
logger.warning(f"清空知识库表 {table_name} 失败: {e}")
|
||||
|
||||
# 删除知识库文件目录
|
||||
for kb_id in list(self.kb_manager.kb_insts.keys()):
|
||||
try:
|
||||
kb_helper = self.kb_manager.kb_insts[kb_id]
|
||||
await kb_helper.terminate()
|
||||
if kb_helper.kb_dir.exists():
|
||||
shutil.rmtree(kb_helper.kb_dir)
|
||||
except Exception as e:
|
||||
logger.warning(f"清理知识库 {kb_id} 失败: {e}")
|
||||
|
||||
self.kb_manager.kb_insts.clear()
|
||||
|
||||
async def _import_main_database(
|
||||
self, data: dict[str, list[dict]]
|
||||
) -> dict[str, int]:
|
||||
"""导入主数据库数据"""
|
||||
imported: dict[str, int] = {}
|
||||
|
||||
async with self.main_db.get_db() as session:
|
||||
async with session.begin():
|
||||
for table_name, rows in data.items():
|
||||
model_class = MAIN_DB_MODELS.get(table_name)
|
||||
if not model_class:
|
||||
logger.warning(f"未知的表: {table_name}")
|
||||
continue
|
||||
|
||||
count = 0
|
||||
for row in rows:
|
||||
try:
|
||||
# 转换 datetime 字符串为 datetime 对象
|
||||
row = self._convert_datetime_fields(row, model_class)
|
||||
obj = model_class(**row)
|
||||
session.add(obj)
|
||||
count += 1
|
||||
except Exception as e:
|
||||
logger.warning(f"导入记录到 {table_name} 失败: {e}")
|
||||
|
||||
imported[table_name] = count
|
||||
logger.debug(f"导入表 {table_name}: {count} 条记录")
|
||||
|
||||
return imported
|
||||
|
||||
async def _import_knowledge_bases(
|
||||
self,
|
||||
zf: zipfile.ZipFile,
|
||||
kb_meta_data: dict[str, list[dict]],
|
||||
result: ImportResult,
|
||||
) -> None:
|
||||
"""导入知识库数据"""
|
||||
if not self.kb_manager:
|
||||
return
|
||||
|
||||
# 1. 导入知识库元数据
|
||||
async with self.kb_manager.kb_db.get_db() as session:
|
||||
async with session.begin():
|
||||
for table_name, rows in kb_meta_data.items():
|
||||
model_class = KB_METADATA_MODELS.get(table_name)
|
||||
if not model_class:
|
||||
continue
|
||||
|
||||
count = 0
|
||||
for row in rows:
|
||||
try:
|
||||
row = self._convert_datetime_fields(row, model_class)
|
||||
obj = model_class(**row)
|
||||
session.add(obj)
|
||||
count += 1
|
||||
except Exception as e:
|
||||
logger.warning(f"导入知识库记录到 {table_name} 失败: {e}")
|
||||
|
||||
result.imported_tables[f"kb_{table_name}"] = count
|
||||
|
||||
# 2. 导入每个知识库的文档和文件
|
||||
for kb_data in kb_meta_data.get("knowledge_bases", []):
|
||||
kb_id = kb_data.get("kb_id")
|
||||
if not kb_id:
|
||||
continue
|
||||
|
||||
# 创建知识库目录
|
||||
kb_dir = Path(self.kb_root_dir) / kb_id
|
||||
kb_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 导入文档数据
|
||||
doc_path = f"databases/kb_{kb_id}/documents.json"
|
||||
if doc_path in zf.namelist():
|
||||
try:
|
||||
doc_content = zf.read(doc_path)
|
||||
doc_data = json.loads(doc_content)
|
||||
|
||||
# 导入到文档存储数据库
|
||||
await self._import_kb_documents(kb_id, doc_data)
|
||||
except Exception as e:
|
||||
result.add_warning(f"导入知识库 {kb_id} 的文档失败: {e}")
|
||||
|
||||
# 导入 FAISS 索引
|
||||
faiss_path = f"databases/kb_{kb_id}/index.faiss"
|
||||
if faiss_path in zf.namelist():
|
||||
try:
|
||||
target_path = kb_dir / "index.faiss"
|
||||
with zf.open(faiss_path) as src, open(target_path, "wb") as dst:
|
||||
dst.write(src.read())
|
||||
except Exception as e:
|
||||
result.add_warning(f"导入知识库 {kb_id} 的 FAISS 索引失败: {e}")
|
||||
|
||||
# 导入媒体文件
|
||||
media_prefix = f"files/kb_media/{kb_id}/"
|
||||
for name in zf.namelist():
|
||||
if name.startswith(media_prefix):
|
||||
try:
|
||||
rel_path = name[len(media_prefix) :]
|
||||
target_path = kb_dir / rel_path
|
||||
target_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with zf.open(name) as src, open(target_path, "wb") as dst:
|
||||
dst.write(src.read())
|
||||
except Exception as e:
|
||||
result.add_warning(f"导入媒体文件 {name} 失败: {e}")
|
||||
|
||||
# 3. 重新加载知识库实例
|
||||
await self.kb_manager.load_kbs()
|
||||
|
||||
async def _import_kb_documents(self, kb_id: str, doc_data: dict) -> None:
|
||||
"""导入知识库文档到向量数据库"""
|
||||
from astrbot.core.db.vec_db.faiss_impl.document_storage import DocumentStorage
|
||||
|
||||
kb_dir = Path(self.kb_root_dir) / kb_id
|
||||
doc_db_path = kb_dir / "doc.db"
|
||||
|
||||
# 初始化文档存储
|
||||
doc_storage = DocumentStorage(str(doc_db_path))
|
||||
await doc_storage.initialize()
|
||||
|
||||
try:
|
||||
documents = doc_data.get("documents", [])
|
||||
for doc in documents:
|
||||
try:
|
||||
await doc_storage.insert_document(
|
||||
doc_id=doc.get("doc_id", ""),
|
||||
text=doc.get("text", ""),
|
||||
metadata=json.loads(doc.get("metadata", "{}")),
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"导入文档块失败: {e}")
|
||||
finally:
|
||||
await doc_storage.close()
|
||||
|
||||
async def _import_attachments(
|
||||
self,
|
||||
zf: zipfile.ZipFile,
|
||||
attachments: list[dict],
|
||||
) -> int:
|
||||
"""导入附件文件"""
|
||||
count = 0
|
||||
|
||||
attachments_dir = Path(self.config_path).parent / "attachments"
|
||||
attachments_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
attachment_prefix = "files/attachments/"
|
||||
for name in zf.namelist():
|
||||
if name.startswith(attachment_prefix) and name != attachment_prefix:
|
||||
try:
|
||||
# 从附件记录中找到原始路径
|
||||
attachment_id = os.path.splitext(os.path.basename(name))[0]
|
||||
original_path = None
|
||||
for att in attachments:
|
||||
if att.get("attachment_id") == attachment_id:
|
||||
original_path = att.get("path")
|
||||
break
|
||||
|
||||
if original_path:
|
||||
target_path = Path(original_path)
|
||||
else:
|
||||
target_path = attachments_dir / os.path.basename(name)
|
||||
|
||||
target_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with zf.open(name) as src, open(target_path, "wb") as dst:
|
||||
dst.write(src.read())
|
||||
count += 1
|
||||
except Exception as e:
|
||||
logger.warning(f"导入附件 {name} 失败: {e}")
|
||||
|
||||
return count
|
||||
|
||||
async def _import_directories(
|
||||
self,
|
||||
zf: zipfile.ZipFile,
|
||||
manifest: dict,
|
||||
result: ImportResult,
|
||||
) -> dict[str, int]:
|
||||
"""导入插件和其他数据目录
|
||||
|
||||
Args:
|
||||
zf: ZIP 文件对象
|
||||
manifest: 备份清单
|
||||
result: 导入结果对象
|
||||
|
||||
Returns:
|
||||
dict: 每个目录导入的文件数量
|
||||
"""
|
||||
dir_stats: dict[str, int] = {}
|
||||
|
||||
# 检查备份版本是否支持目录备份(需要版本 >= 1.1)
|
||||
backup_version = manifest.get("version", "1.0")
|
||||
if VersionComparator.compare_version(backup_version, "1.1") < 0:
|
||||
logger.info("备份版本不支持目录备份,跳过目录导入")
|
||||
return dir_stats
|
||||
|
||||
backed_up_dirs = manifest.get("directories", [])
|
||||
backup_directories = get_backup_directories()
|
||||
|
||||
for dir_name in backed_up_dirs:
|
||||
if dir_name not in backup_directories:
|
||||
result.add_warning(f"未知的目录类型: {dir_name}")
|
||||
continue
|
||||
|
||||
target_dir = Path(backup_directories[dir_name])
|
||||
archive_prefix = f"directories/{dir_name}/"
|
||||
|
||||
file_count = 0
|
||||
|
||||
try:
|
||||
# 获取该目录下的所有文件
|
||||
dir_files = [
|
||||
name
|
||||
for name in zf.namelist()
|
||||
if name.startswith(archive_prefix) and name != archive_prefix
|
||||
]
|
||||
|
||||
if not dir_files:
|
||||
continue
|
||||
|
||||
# 备份现有目录(如果存在)
|
||||
if target_dir.exists():
|
||||
backup_path = Path(f"{target_dir}.bak")
|
||||
if backup_path.exists():
|
||||
shutil.rmtree(backup_path)
|
||||
shutil.move(str(target_dir), str(backup_path))
|
||||
logger.debug(f"已备份现有目录 {target_dir} 到 {backup_path}")
|
||||
|
||||
# 创建目标目录
|
||||
target_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 解压文件
|
||||
for name in dir_files:
|
||||
try:
|
||||
# 计算相对路径
|
||||
rel_path = name[len(archive_prefix) :]
|
||||
if not rel_path: # 跳过目录条目
|
||||
continue
|
||||
|
||||
target_path = target_dir / rel_path
|
||||
target_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with zf.open(name) as src, open(target_path, "wb") as dst:
|
||||
dst.write(src.read())
|
||||
file_count += 1
|
||||
except Exception as e:
|
||||
result.add_warning(f"导入文件 {name} 失败: {e}")
|
||||
|
||||
dir_stats[dir_name] = file_count
|
||||
logger.debug(f"导入目录 {dir_name}: {file_count} 个文件")
|
||||
|
||||
except Exception as e:
|
||||
result.add_warning(f"导入目录 {dir_name} 失败: {e}")
|
||||
dir_stats[dir_name] = 0
|
||||
|
||||
return dir_stats
|
||||
|
||||
def _convert_datetime_fields(self, row: dict, model_class: type) -> dict:
|
||||
"""转换 datetime 字符串字段为 datetime 对象"""
|
||||
result = row.copy()
|
||||
|
||||
# 获取模型的 datetime 字段
|
||||
from sqlalchemy import inspect as sa_inspect
|
||||
|
||||
try:
|
||||
mapper = sa_inspect(model_class)
|
||||
for column in mapper.columns:
|
||||
if column.name in result and result[column.name] is not None:
|
||||
# 检查是否是 datetime 类型的列
|
||||
from sqlalchemy import DateTime
|
||||
|
||||
if isinstance(column.type, DateTime):
|
||||
value = result[column.name]
|
||||
if isinstance(value, str):
|
||||
# 解析 ISO 格式的日期时间字符串
|
||||
result[column.name] = datetime.fromisoformat(value)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return result
|
||||
@@ -80,8 +80,6 @@ class AstrBotConfig(dict):
|
||||
if v["type"] == "object":
|
||||
conf[k] = {}
|
||||
_parse_schema(v["items"], conf[k])
|
||||
elif v["type"] == "template_list":
|
||||
conf[k] = default
|
||||
else:
|
||||
conf[k] = default
|
||||
|
||||
|
||||
+242
-300
@@ -1,11 +1,10 @@
|
||||
"""如需修改配置,请在 `data/cmd_config.json` 中修改或者在管理面板中可视化修改。"""
|
||||
|
||||
import os
|
||||
from typing import Any, TypedDict
|
||||
|
||||
from astrbot.core.utils.astrbot_path import get_astrbot_data_path
|
||||
|
||||
VERSION = "4.11.0"
|
||||
VERSION = "4.8.0"
|
||||
DB_PATH = os.path.join(get_astrbot_data_path(), "data_v4.db")
|
||||
|
||||
WEBHOOK_SUPPORTED_PLATFORMS = [
|
||||
@@ -62,8 +61,7 @@ DEFAULT_CONFIG = {
|
||||
"ignore_bot_self_message": False,
|
||||
"ignore_at_all": False,
|
||||
},
|
||||
"provider_sources": [], # provider sources
|
||||
"provider": [], # models from provider_sources
|
||||
"provider": [],
|
||||
"provider_settings": {
|
||||
"enable": True,
|
||||
"default_provider_id": "",
|
||||
@@ -83,16 +81,6 @@ DEFAULT_CONFIG = {
|
||||
"default_personality": "default",
|
||||
"persona_pool": ["*"],
|
||||
"prompt_prefix": "{{prompt}}",
|
||||
"context_limit_reached_strategy": "truncate_by_turns", # or llm_compress
|
||||
"llm_compress_instruction": (
|
||||
"Based on our full conversation history, produce a concise summary of key takeaways and/or project progress.\n"
|
||||
"1. Systematically cover all core topics discussed and the final conclusion/outcome for each; clearly highlight the latest primary focus.\n"
|
||||
"2. If any tools were used, summarize tool usage (total call count) and extract the most valuable insights from tool outputs.\n"
|
||||
"3. If there was an initial user goal, state it first and describe the current progress/status.\n"
|
||||
"4. Write the summary in the user's language.\n"
|
||||
),
|
||||
"llm_compress_keep_recent": 4,
|
||||
"llm_compress_provider_id": "",
|
||||
"max_context_length": -1,
|
||||
"dequeue_context_length": 1,
|
||||
"streaming_response": False,
|
||||
@@ -120,7 +108,6 @@ DEFAULT_CONFIG = {
|
||||
"provider_id": "",
|
||||
"dual_output": False,
|
||||
"use_file_service": False,
|
||||
"trigger_probability": 1.0,
|
||||
},
|
||||
"provider_ltm_settings": {
|
||||
"group_icl_enable": False,
|
||||
@@ -183,24 +170,6 @@ DEFAULT_CONFIG = {
|
||||
}
|
||||
|
||||
|
||||
class ChatProviderTemplate(TypedDict):
|
||||
id: str
|
||||
provider_source_id: str
|
||||
model: str
|
||||
modalities: list
|
||||
custom_extra_body: dict[str, Any]
|
||||
max_context_tokens: int
|
||||
|
||||
|
||||
CHAT_PROVIDER_TEMPLATE = {
|
||||
"id": "",
|
||||
"provide_source_id": "",
|
||||
"model": "",
|
||||
"modalities": [],
|
||||
"custom_extra_body": {},
|
||||
"max_context_tokens": 0,
|
||||
}
|
||||
|
||||
"""
|
||||
AstrBot v3 时代的配置元数据,目前仅承担以下功能:
|
||||
|
||||
@@ -239,7 +208,7 @@ CONFIG_METADATA_2 = {
|
||||
"callback_server_host": "0.0.0.0",
|
||||
"port": 6196,
|
||||
},
|
||||
"OneBot v11 (QQ 个人号等)": {
|
||||
"QQ 个人号(OneBot v11)": {
|
||||
"id": "default",
|
||||
"type": "aiocqhttp",
|
||||
"enable": False,
|
||||
@@ -247,6 +216,16 @@ CONFIG_METADATA_2 = {
|
||||
"ws_reverse_port": 6199,
|
||||
"ws_reverse_token": "",
|
||||
},
|
||||
"WeChatPadPro": {
|
||||
"id": "wechatpadpro",
|
||||
"type": "wechatpadpro",
|
||||
"enable": False,
|
||||
"admin_key": "stay33",
|
||||
"host": "这里填写你的局域网IP或者公网服务器IP",
|
||||
"port": 8059,
|
||||
"wpp_active_message_poll": False,
|
||||
"wpp_active_message_poll_interval": 3,
|
||||
},
|
||||
"微信公众平台": {
|
||||
"id": "weixin_official_account",
|
||||
"type": "weixin_official_account",
|
||||
@@ -376,16 +355,6 @@ CONFIG_METADATA_2 = {
|
||||
"satori_heartbeat_interval": 10,
|
||||
"satori_reconnect_delay": 5,
|
||||
},
|
||||
"WeChatPadPro": {
|
||||
"id": "wechatpadpro",
|
||||
"type": "wechatpadpro",
|
||||
"enable": False,
|
||||
"admin_key": "stay33",
|
||||
"host": "这里填写你的局域网IP或者公网服务器IP",
|
||||
"port": 8059,
|
||||
"wpp_active_message_poll": False,
|
||||
"wpp_active_message_poll_interval": 3,
|
||||
},
|
||||
# "WebChat": {
|
||||
# "id": "webchat",
|
||||
# "type": "webchat",
|
||||
@@ -874,7 +843,6 @@ CONFIG_METADATA_2 = {
|
||||
"metadata": {
|
||||
"provider": {
|
||||
"type": "list",
|
||||
# provider sources templates
|
||||
"config_template": {
|
||||
"OpenAI": {
|
||||
"id": "openai",
|
||||
@@ -885,10 +853,107 @@ CONFIG_METADATA_2 = {
|
||||
"key": [],
|
||||
"api_base": "https://api.openai.com/v1",
|
||||
"timeout": 120,
|
||||
"model_config": {"model": "gpt-4o-mini", "temperature": 0.4},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
"hint": "也兼容所有与 OpenAI API 兼容的服务。",
|
||||
},
|
||||
"Google Gemini": {
|
||||
"id": "google_gemini",
|
||||
"Azure OpenAI": {
|
||||
"id": "azure",
|
||||
"provider": "azure",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"api_version": "2024-05-01-preview",
|
||||
"key": [],
|
||||
"api_base": "",
|
||||
"timeout": 120,
|
||||
"model_config": {"model": "gpt-4o-mini", "temperature": 0.4},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"xAI": {
|
||||
"id": "xai",
|
||||
"provider": "xai",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"api_base": "https://api.x.ai/v1",
|
||||
"timeout": 120,
|
||||
"model_config": {"model": "grok-2-latest", "temperature": 0.4},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"xai_native_search": False,
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"Anthropic": {
|
||||
"hint": "注意Claude系列模型的温度调节范围为0到1.0,超出可能导致报错",
|
||||
"id": "claude",
|
||||
"provider": "anthropic",
|
||||
"type": "anthropic_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"api_base": "https://api.anthropic.com/v1",
|
||||
"timeout": 120,
|
||||
"model_config": {
|
||||
"model": "claude-3-5-sonnet-latest",
|
||||
"max_tokens": 4096,
|
||||
"temperature": 0.2,
|
||||
},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"Ollama": {
|
||||
"hint": "启用前请确保已正确安装并运行 Ollama 服务端,Ollama默认不带鉴权,无需修改key",
|
||||
"id": "ollama_default",
|
||||
"provider": "ollama",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": ["ollama"], # ollama 的 key 默认是 ollama
|
||||
"api_base": "http://localhost:11434/v1",
|
||||
"model_config": {"model": "llama3.1-8b", "temperature": 0.4},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"LM Studio": {
|
||||
"id": "lm_studio",
|
||||
"provider": "lm_studio",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": ["lmstudio"],
|
||||
"api_base": "http://localhost:1234/v1",
|
||||
"model_config": {
|
||||
"model": "llama-3.1-8b",
|
||||
},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"Gemini(OpenAI兼容)": {
|
||||
"id": "gemini_default",
|
||||
"provider": "google",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"api_base": "https://generativelanguage.googleapis.com/v1beta/openai/",
|
||||
"timeout": 120,
|
||||
"model_config": {
|
||||
"model": "gemini-1.5-flash",
|
||||
"temperature": 0.4,
|
||||
},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"Gemini": {
|
||||
"id": "gemini_default",
|
||||
"provider": "google",
|
||||
"type": "googlegenai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
@@ -896,6 +961,10 @@ CONFIG_METADATA_2 = {
|
||||
"key": [],
|
||||
"api_base": "https://generativelanguage.googleapis.com/",
|
||||
"timeout": 120,
|
||||
"model_config": {
|
||||
"model": "gemini-2.0-flash-exp",
|
||||
"temperature": 0.4,
|
||||
},
|
||||
"gm_resp_image_modal": False,
|
||||
"gm_native_search": False,
|
||||
"gm_native_coderunner": False,
|
||||
@@ -906,44 +975,13 @@ CONFIG_METADATA_2 = {
|
||||
"sexually_explicit": "BLOCK_MEDIUM_AND_ABOVE",
|
||||
"dangerous_content": "BLOCK_MEDIUM_AND_ABOVE",
|
||||
},
|
||||
"gm_thinking_config": {"budget": 0, "level": "HIGH"},
|
||||
},
|
||||
"Anthropic": {
|
||||
"id": "anthropic",
|
||||
"provider": "anthropic",
|
||||
"type": "anthropic_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"api_base": "https://api.anthropic.com/v1",
|
||||
"timeout": 120,
|
||||
"anth_thinking_config": {"budget": 0},
|
||||
},
|
||||
"Moonshot": {
|
||||
"id": "moonshot",
|
||||
"provider": "moonshot",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"timeout": 120,
|
||||
"api_base": "https://api.moonshot.cn/v1",
|
||||
"custom_headers": {},
|
||||
},
|
||||
"xAI": {
|
||||
"id": "xai",
|
||||
"provider": "xai",
|
||||
"type": "xai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"api_base": "https://api.x.ai/v1",
|
||||
"timeout": 120,
|
||||
"custom_headers": {},
|
||||
"xai_native_search": False,
|
||||
"gm_thinking_config": {
|
||||
"budget": 0,
|
||||
},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"DeepSeek": {
|
||||
"id": "deepseek",
|
||||
"id": "deepseek_default",
|
||||
"provider": "deepseek",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
@@ -951,75 +989,13 @@ CONFIG_METADATA_2 = {
|
||||
"key": [],
|
||||
"api_base": "https://api.deepseek.com/v1",
|
||||
"timeout": 120,
|
||||
"model_config": {"model": "deepseek-chat", "temperature": 0.4},
|
||||
"custom_headers": {},
|
||||
},
|
||||
"Zhipu": {
|
||||
"id": "zhipu",
|
||||
"provider": "zhipu",
|
||||
"type": "zhipu_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"timeout": 120,
|
||||
"api_base": "https://open.bigmodel.cn/api/paas/v4/",
|
||||
"custom_headers": {},
|
||||
},
|
||||
"Azure OpenAI": {
|
||||
"id": "azure_openai",
|
||||
"provider": "azure",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"api_version": "2024-05-01-preview",
|
||||
"key": [],
|
||||
"api_base": "",
|
||||
"timeout": 120,
|
||||
"custom_headers": {},
|
||||
},
|
||||
"Ollama": {
|
||||
"id": "ollama",
|
||||
"provider": "ollama",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": ["ollama"], # ollama 的 key 默认是 ollama
|
||||
"api_base": "http://127.0.0.1:11434/v1",
|
||||
"custom_headers": {},
|
||||
},
|
||||
"LM Studio": {
|
||||
"id": "lm_studio",
|
||||
"provider": "lm_studio",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": ["lmstudio"],
|
||||
"api_base": "http://127.0.0.1:1234/v1",
|
||||
"custom_headers": {},
|
||||
},
|
||||
"ModelStack": {
|
||||
"id": "modelstack",
|
||||
"provider": "modelstack",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"api_base": "https://modelstack.app/v1",
|
||||
"timeout": 120,
|
||||
"custom_headers": {},
|
||||
},
|
||||
"Gemini_OpenAI_API": {
|
||||
"id": "google_gemini_openai",
|
||||
"provider": "google",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"api_base": "https://generativelanguage.googleapis.com/v1beta/openai/",
|
||||
"timeout": 120,
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "tool_use"],
|
||||
},
|
||||
"Groq": {
|
||||
"id": "groq",
|
||||
"id": "groq_default",
|
||||
"provider": "groq",
|
||||
"type": "groq_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
@@ -1027,7 +1003,13 @@ CONFIG_METADATA_2 = {
|
||||
"key": [],
|
||||
"api_base": "https://api.groq.com/openai/v1",
|
||||
"timeout": 120,
|
||||
"model_config": {
|
||||
"model": "openai/gpt-oss-20b",
|
||||
"temperature": 0.4,
|
||||
},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "tool_use"],
|
||||
},
|
||||
"302.AI": {
|
||||
"id": "302ai",
|
||||
@@ -1038,9 +1020,12 @@ CONFIG_METADATA_2 = {
|
||||
"key": [],
|
||||
"api_base": "https://api.302.ai/v1",
|
||||
"timeout": 120,
|
||||
"model_config": {"model": "gpt-4.1-mini", "temperature": 0.4},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"SiliconFlow": {
|
||||
"硅基流动": {
|
||||
"id": "siliconflow",
|
||||
"provider": "siliconflow",
|
||||
"type": "openai_chat_completion",
|
||||
@@ -1049,9 +1034,15 @@ CONFIG_METADATA_2 = {
|
||||
"key": [],
|
||||
"timeout": 120,
|
||||
"api_base": "https://api.siliconflow.cn/v1",
|
||||
"model_config": {
|
||||
"model": "deepseek-ai/DeepSeek-V3",
|
||||
"temperature": 0.4,
|
||||
},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"PPIO": {
|
||||
"PPIO派欧云": {
|
||||
"id": "ppio",
|
||||
"provider": "ppio",
|
||||
"type": "openai_chat_completion",
|
||||
@@ -1060,9 +1051,14 @@ CONFIG_METADATA_2 = {
|
||||
"key": [],
|
||||
"api_base": "https://api.ppinfra.com/v3/openai",
|
||||
"timeout": 120,
|
||||
"model_config": {
|
||||
"model": "deepseek/deepseek-r1",
|
||||
"temperature": 0.4,
|
||||
},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
},
|
||||
"TokenPony": {
|
||||
"小马算力": {
|
||||
"id": "tokenpony",
|
||||
"provider": "tokenpony",
|
||||
"type": "openai_chat_completion",
|
||||
@@ -1071,9 +1067,14 @@ CONFIG_METADATA_2 = {
|
||||
"key": [],
|
||||
"api_base": "https://api.tokenpony.cn/v1",
|
||||
"timeout": 120,
|
||||
"model_config": {
|
||||
"model": "kimi-k2-instruct-0905",
|
||||
"temperature": 0.7,
|
||||
},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
},
|
||||
"Compshare": {
|
||||
"优云智算": {
|
||||
"id": "compshare",
|
||||
"provider": "compshare",
|
||||
"type": "openai_chat_completion",
|
||||
@@ -1082,18 +1083,42 @@ CONFIG_METADATA_2 = {
|
||||
"key": [],
|
||||
"api_base": "https://api.modelverse.cn/v1",
|
||||
"timeout": 120,
|
||||
"model_config": {
|
||||
"model": "moonshotai/Kimi-K2-Instruct",
|
||||
},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"ModelScope": {
|
||||
"id": "modelscope",
|
||||
"provider": "modelscope",
|
||||
"Kimi": {
|
||||
"id": "moonshot",
|
||||
"provider": "moonshot",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"timeout": 120,
|
||||
"api_base": "https://api-inference.modelscope.cn/v1",
|
||||
"api_base": "https://api.moonshot.cn/v1",
|
||||
"model_config": {"model": "moonshot-v1-8k", "temperature": 0.4},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"智谱 AI": {
|
||||
"id": "zhipu_default",
|
||||
"provider": "zhipu",
|
||||
"type": "zhipu_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"timeout": 120,
|
||||
"api_base": "https://open.bigmodel.cn/api/paas/v4/",
|
||||
"model_config": {
|
||||
"model": "glm-4-flash",
|
||||
},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"Dify": {
|
||||
"id": "dify_app_default",
|
||||
@@ -1108,6 +1133,7 @@ CONFIG_METADATA_2 = {
|
||||
"dify_query_input_key": "astrbot_text_query",
|
||||
"variables": {},
|
||||
"timeout": 60,
|
||||
"hint": "请确保你在 AstrBot 里设置的 APP 类型和 Dify 里面创建的应用的类型一致!",
|
||||
},
|
||||
"Coze": {
|
||||
"id": "coze",
|
||||
@@ -1138,6 +1164,20 @@ CONFIG_METADATA_2 = {
|
||||
"variables": {},
|
||||
"timeout": 60,
|
||||
},
|
||||
"ModelScope": {
|
||||
"id": "modelscope",
|
||||
"provider": "modelscope",
|
||||
"type": "openai_chat_completion",
|
||||
"provider_type": "chat_completion",
|
||||
"enable": True,
|
||||
"key": [],
|
||||
"timeout": 120,
|
||||
"api_base": "https://api-inference.modelscope.cn/v1",
|
||||
"model_config": {"model": "Qwen/Qwen3-32B", "temperature": 0.4},
|
||||
"custom_headers": {},
|
||||
"custom_extra_body": {},
|
||||
"modalities": ["text", "image", "tool_use"],
|
||||
},
|
||||
"FastGPT": {
|
||||
"id": "fastgpt",
|
||||
"provider": "fastgpt",
|
||||
@@ -1161,6 +1201,7 @@ CONFIG_METADATA_2 = {
|
||||
"model": "whisper-1",
|
||||
},
|
||||
"Whisper(Local)": {
|
||||
"hint": "启用前请 pip 安装 openai-whisper 库(N卡用户大约下载 2GB,主要是 torch 和 cuda,CPU 用户大约下载 1 GB),并且安装 ffmpeg。否则将无法正常转文字。",
|
||||
"provider": "openai",
|
||||
"type": "openai_whisper_selfhost",
|
||||
"provider_type": "speech_to_text",
|
||||
@@ -1169,6 +1210,7 @@ CONFIG_METADATA_2 = {
|
||||
"model": "tiny",
|
||||
},
|
||||
"SenseVoice(Local)": {
|
||||
"hint": "启用前请 pip 安装 funasr、funasr_onnx、torchaudio、torch、modelscope、jieba 库(默认使用CPU,大约下载 1 GB),并且安装 ffmpeg。否则将无法正常转文字。",
|
||||
"type": "sensevoice_stt_selfhost",
|
||||
"provider": "sensevoice",
|
||||
"provider_type": "speech_to_text",
|
||||
@@ -1190,6 +1232,7 @@ CONFIG_METADATA_2 = {
|
||||
"timeout": "20",
|
||||
},
|
||||
"Edge TTS": {
|
||||
"hint": "提示:使用这个服务前需要安装有 ffmpeg,并且可以直接在终端调用 ffmpeg 指令。",
|
||||
"id": "edge_tts",
|
||||
"provider": "microsoft",
|
||||
"type": "edge_tts",
|
||||
@@ -1299,7 +1342,7 @@ CONFIG_METADATA_2 = {
|
||||
"minimax-is-timber-weight": False,
|
||||
"minimax-voice-id": "female-shaonv",
|
||||
"minimax-timber-weight": '[\n {\n "voice_id": "Chinese (Mandarin)_Warm_Girl",\n "weight": 25\n },\n {\n "voice_id": "Chinese (Mandarin)_BashfulGirl",\n "weight": 50\n }\n]',
|
||||
"minimax-voice-emotion": "auto",
|
||||
"minimax-voice-emotion": "neutral",
|
||||
"minimax-voice-latex": False,
|
||||
"minimax-voice-english-normalization": False,
|
||||
"timeout": 20,
|
||||
@@ -1405,10 +1448,6 @@ CONFIG_METADATA_2 = {
|
||||
},
|
||||
},
|
||||
"items": {
|
||||
"provider_source_id": {
|
||||
"invisible": True,
|
||||
"type": "string",
|
||||
},
|
||||
"xai_native_search": {
|
||||
"description": "启用原生搜索功能",
|
||||
"type": "bool",
|
||||
@@ -1463,32 +1502,7 @@ CONFIG_METADATA_2 = {
|
||||
"description": "自定义请求体参数",
|
||||
"type": "dict",
|
||||
"items": {},
|
||||
"hint": "用于在请求时添加额外的参数,如 temperature、top_p、max_tokens 等。",
|
||||
"template_schema": {
|
||||
"temperature": {
|
||||
"name": "Temperature",
|
||||
"description": "温度参数",
|
||||
"hint": "控制输出的随机性,范围通常为 0-2。值越高越随机。",
|
||||
"type": "float",
|
||||
"default": 0.6,
|
||||
"slider": {"min": 0, "max": 2, "step": 0.1},
|
||||
},
|
||||
"top_p": {
|
||||
"name": "Top-p",
|
||||
"description": "Top-p 采样",
|
||||
"hint": "核采样参数,范围通常为 0-1。控制模型考虑的概率质量。",
|
||||
"type": "float",
|
||||
"default": 1.0,
|
||||
"slider": {"min": 0, "max": 1, "step": 0.01},
|
||||
},
|
||||
"max_tokens": {
|
||||
"name": "Max Tokens",
|
||||
"description": "最大令牌数",
|
||||
"hint": "生成的最大令牌数。",
|
||||
"type": "int",
|
||||
"default": 8192,
|
||||
},
|
||||
},
|
||||
"hint": "此处添加的键值对将被合并到发送给 API 的 extra_body 中。值可以是字符串、数字或布尔值。",
|
||||
},
|
||||
"provider": {
|
||||
"type": "string",
|
||||
@@ -1804,35 +1818,13 @@ CONFIG_METADATA_2 = {
|
||||
},
|
||||
},
|
||||
"gm_thinking_config": {
|
||||
"description": "Thinking Config",
|
||||
"description": "Gemini思考设置",
|
||||
"type": "object",
|
||||
"items": {
|
||||
"budget": {
|
||||
"description": "Thinking Budget",
|
||||
"description": "思考预算",
|
||||
"type": "int",
|
||||
"hint": "Guides the model on the specific number of thinking tokens to use for reasoning. See: https://ai.google.dev/gemini-api/docs/thinking#set-budget",
|
||||
},
|
||||
"level": {
|
||||
"description": "Thinking Level",
|
||||
"type": "string",
|
||||
"hint": "Recommended for Gemini 3 models and onwards, lets you control reasoning behavior.See: https://ai.google.dev/gemini-api/docs/thinking#thinking-levels",
|
||||
"options": [
|
||||
"MINIMAL",
|
||||
"LOW",
|
||||
"MEDIUM",
|
||||
"HIGH",
|
||||
],
|
||||
},
|
||||
},
|
||||
},
|
||||
"anth_thinking_config": {
|
||||
"description": "Thinking Config",
|
||||
"type": "object",
|
||||
"items": {
|
||||
"budget": {
|
||||
"description": "Thinking Budget",
|
||||
"type": "int",
|
||||
"hint": "Anthropic thinking.budget_tokens param. Must >= 1024. See: https://platform.claude.com/docs/en/build-with-claude/extended-thinking",
|
||||
"hint": "模型应该生成的思考Token的数量,设为0关闭思考。除gemini-2.5-flash外的模型会静默忽略此参数。",
|
||||
},
|
||||
},
|
||||
},
|
||||
@@ -1907,18 +1899,15 @@ CONFIG_METADATA_2 = {
|
||||
"minimax-voice-emotion": {
|
||||
"type": "string",
|
||||
"description": "情绪",
|
||||
"hint": "控制合成语音的情绪。当为 auto 时,将根据文本内容自动选择情绪。",
|
||||
"hint": "控制合成语音的情绪",
|
||||
"options": [
|
||||
"auto",
|
||||
"happy",
|
||||
"sad",
|
||||
"angry",
|
||||
"fearful",
|
||||
"disgusted",
|
||||
"surprised",
|
||||
"calm",
|
||||
"fluent",
|
||||
"whisper",
|
||||
"neutral",
|
||||
],
|
||||
},
|
||||
"minimax-voice-latex": {
|
||||
@@ -2016,6 +2005,7 @@ CONFIG_METADATA_2 = {
|
||||
"id": {
|
||||
"description": "ID",
|
||||
"type": "string",
|
||||
"hint": "模型提供商名字。",
|
||||
},
|
||||
"type": {
|
||||
"description": "模型提供商种类",
|
||||
@@ -2035,20 +2025,29 @@ CONFIG_METADATA_2 = {
|
||||
"description": "API Key",
|
||||
"type": "list",
|
||||
"items": {"type": "string"},
|
||||
"hint": "提供商 API Key。",
|
||||
},
|
||||
"api_base": {
|
||||
"description": "API Base URL",
|
||||
"type": "string",
|
||||
"hint": "API Base URL 请在模型提供商处获得。如出现 404 报错,尝试在地址末尾加上 /v1",
|
||||
},
|
||||
"model": {
|
||||
"description": "模型 ID",
|
||||
"type": "string",
|
||||
"hint": "模型名称,如 gpt-4o-mini, deepseek-chat。",
|
||||
},
|
||||
"max_context_tokens": {
|
||||
"description": "模型上下文窗口大小",
|
||||
"type": "int",
|
||||
"hint": "模型最大上下文 Token 大小。如果为 0,则会自动从模型元数据填充(如有),也可手动修改。",
|
||||
"model_config": {
|
||||
"description": "模型配置",
|
||||
"type": "object",
|
||||
"items": {
|
||||
"model": {
|
||||
"description": "模型名称",
|
||||
"type": "string",
|
||||
"hint": "模型名称,如 gpt-4o-mini, deepseek-chat。",
|
||||
},
|
||||
"max_tokens": {
|
||||
"description": "模型最大输出长度(tokens)",
|
||||
"type": "int",
|
||||
},
|
||||
"temperature": {"description": "温度", "type": "float"},
|
||||
"top_p": {"description": "Top P值", "type": "float"},
|
||||
},
|
||||
},
|
||||
"dify_api_key": {
|
||||
"description": "API Key",
|
||||
@@ -2210,9 +2209,6 @@ CONFIG_METADATA_2 = {
|
||||
"use_file_service": {
|
||||
"type": "bool",
|
||||
},
|
||||
"trigger_probability": {
|
||||
"type": "float",
|
||||
},
|
||||
},
|
||||
},
|
||||
"provider_ltm_settings": {
|
||||
@@ -2423,14 +2419,6 @@ CONFIG_METADATA_3 = {
|
||||
"provider_tts_settings.enable": True,
|
||||
},
|
||||
},
|
||||
"provider_tts_settings.trigger_probability": {
|
||||
"description": "TTS 触发概率",
|
||||
"type": "float",
|
||||
"slider": {"min": 0, "max": 1, "step": 0.05},
|
||||
"condition": {
|
||||
"provider_tts_settings.enable": True,
|
||||
},
|
||||
},
|
||||
"provider_settings.image_caption_prompt": {
|
||||
"description": "图片转述提示词",
|
||||
"type": "text",
|
||||
@@ -2557,66 +2545,6 @@ CONFIG_METADATA_3 = {
|
||||
# "provider_settings.enable": True,
|
||||
# },
|
||||
# },
|
||||
"truncate_and_compress": {
|
||||
"description": "上下文管理策略",
|
||||
"type": "object",
|
||||
"items": {
|
||||
"provider_settings.max_context_length": {
|
||||
"description": "最多携带对话轮数",
|
||||
"type": "int",
|
||||
"hint": "超出这个数量时丢弃最旧的部分,一轮聊天记为 1 条,-1 为不限制",
|
||||
"condition": {
|
||||
"provider_settings.agent_runner_type": "local",
|
||||
},
|
||||
},
|
||||
"provider_settings.dequeue_context_length": {
|
||||
"description": "丢弃对话轮数",
|
||||
"type": "int",
|
||||
"hint": "超出最多携带对话轮数时, 一次丢弃的聊天轮数",
|
||||
"condition": {
|
||||
"provider_settings.agent_runner_type": "local",
|
||||
},
|
||||
},
|
||||
"provider_settings.context_limit_reached_strategy": {
|
||||
"description": "超出模型上下文窗口时的处理方式",
|
||||
"type": "string",
|
||||
"options": ["truncate_by_turns", "llm_compress"],
|
||||
"labels": ["按对话轮数截断", "由 LLM 压缩上下文"],
|
||||
"condition": {
|
||||
"provider_settings.agent_runner_type": "local",
|
||||
},
|
||||
"hint": "",
|
||||
},
|
||||
"provider_settings.llm_compress_instruction": {
|
||||
"description": "上下文压缩提示词",
|
||||
"type": "text",
|
||||
"hint": "如果为空则使用默认提示词。",
|
||||
"condition": {
|
||||
"provider_settings.context_limit_reached_strategy": "llm_compress",
|
||||
"provider_settings.agent_runner_type": "local",
|
||||
},
|
||||
},
|
||||
"provider_settings.llm_compress_keep_recent": {
|
||||
"description": "压缩时保留最近对话轮数",
|
||||
"type": "int",
|
||||
"hint": "始终保留的最近 N 轮对话。",
|
||||
"condition": {
|
||||
"provider_settings.context_limit_reached_strategy": "llm_compress",
|
||||
"provider_settings.agent_runner_type": "local",
|
||||
},
|
||||
},
|
||||
"provider_settings.llm_compress_provider_id": {
|
||||
"description": "用于上下文压缩的模型提供商 ID",
|
||||
"type": "string",
|
||||
"_special": "select_provider",
|
||||
"hint": "留空时将降级为“按对话轮数截断”的策略。",
|
||||
"condition": {
|
||||
"provider_settings.context_limit_reached_strategy": "llm_compress",
|
||||
"provider_settings.agent_runner_type": "local",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
"others": {
|
||||
"description": "其他配置",
|
||||
"type": "object",
|
||||
@@ -2681,6 +2609,22 @@ CONFIG_METADATA_3 = {
|
||||
"provider_settings.streaming_response": True,
|
||||
},
|
||||
},
|
||||
"provider_settings.max_context_length": {
|
||||
"description": "最多携带对话轮数",
|
||||
"type": "int",
|
||||
"hint": "超出这个数量时丢弃最旧的部分,一轮聊天记为 1 条,-1 为不限制",
|
||||
"condition": {
|
||||
"provider_settings.agent_runner_type": "local",
|
||||
},
|
||||
},
|
||||
"provider_settings.dequeue_context_length": {
|
||||
"description": "丢弃对话轮数",
|
||||
"type": "int",
|
||||
"hint": "超出最多携带对话轮数时, 一次丢弃的聊天轮数",
|
||||
"condition": {
|
||||
"provider_settings.agent_runner_type": "local",
|
||||
},
|
||||
},
|
||||
"provider_settings.wake_prefix": {
|
||||
"description": "LLM 聊天额外唤醒前缀 ",
|
||||
"type": "string",
|
||||
@@ -3042,7 +2986,6 @@ CONFIG_METADATA_3 = {
|
||||
"description": "回复概率",
|
||||
"type": "float",
|
||||
"hint": "0.0-1.0 之间的数值",
|
||||
"slider": {"min": 0, "max": 1, "step": 0.05},
|
||||
"condition": {
|
||||
"provider_ltm_settings.active_reply.enable": True,
|
||||
},
|
||||
@@ -3150,5 +3093,4 @@ DEFAULT_VALUE_MAP = {
|
||||
"text": "",
|
||||
"list": [],
|
||||
"object": {},
|
||||
"template_list": [],
|
||||
}
|
||||
|
||||
@@ -79,7 +79,6 @@ class ConfigMetadataI18n:
|
||||
"_special",
|
||||
"invisible",
|
||||
"options",
|
||||
"slider",
|
||||
]:
|
||||
if attr in field_data:
|
||||
field_result[attr] = field_data[attr]
|
||||
|
||||
@@ -69,7 +69,6 @@ class ConversationManager:
|
||||
persona_id=conv_v2.persona_id,
|
||||
created_at=created_at,
|
||||
updated_at=updated_at,
|
||||
token_usage=conv_v2.token_usage,
|
||||
)
|
||||
|
||||
async def new_conversation(
|
||||
@@ -257,7 +256,6 @@ class ConversationManager:
|
||||
history: list[dict] | None = None,
|
||||
title: str | None = None,
|
||||
persona_id: str | None = None,
|
||||
token_usage: int | None = None,
|
||||
) -> None:
|
||||
"""更新会话的对话.
|
||||
|
||||
@@ -265,7 +263,6 @@ class ConversationManager:
|
||||
unified_msg_origin (str): 统一的消息来源字符串。格式为 platform_name:message_type:session_id
|
||||
conversation_id (str): 对话 ID, 是 uuid 格式的字符串
|
||||
history (List[Dict]): 对话历史记录, 是一个字典列表, 每个字典包含 role 和 content 字段
|
||||
token_usage (int | None): token 使用量。None 表示不更新
|
||||
|
||||
"""
|
||||
if not conversation_id:
|
||||
@@ -277,7 +274,6 @@ class ConversationManager:
|
||||
title=title,
|
||||
persona_id=persona_id,
|
||||
content=history,
|
||||
token_usage=token_usage,
|
||||
)
|
||||
|
||||
async def update_conversation_title(
|
||||
|
||||
@@ -33,7 +33,6 @@ from astrbot.core.star.context import Context
|
||||
from astrbot.core.star.star_handler import EventType, star_handlers_registry, star_map
|
||||
from astrbot.core.umop_config_router import UmopConfigRouter
|
||||
from astrbot.core.updator import AstrBotUpdator
|
||||
from astrbot.core.utils.llm_metadata import update_llm_metadata
|
||||
from astrbot.core.utils.migra_helper import migra
|
||||
|
||||
from . import astrbot_config, html_renderer
|
||||
@@ -90,7 +89,6 @@ class AstrBotCoreLifecycle:
|
||||
|
||||
# 初始化 UMOP 配置路由器
|
||||
self.umop_config_router = UmopConfigRouter(sp=sp)
|
||||
await self.umop_config_router.initialize()
|
||||
|
||||
# 初始化 AstrBot 配置管理器
|
||||
self.astrbot_config_mgr = AstrBotConfigManager(
|
||||
@@ -187,8 +185,6 @@ class AstrBotCoreLifecycle:
|
||||
# 初始化关闭控制面板的事件
|
||||
self.dashboard_shutdown_event = asyncio.Event()
|
||||
|
||||
asyncio.create_task(update_llm_metadata())
|
||||
|
||||
def _load(self) -> None:
|
||||
"""加载事件总线和任务并初始化."""
|
||||
# 创建一个异步任务来执行事件总线的 dispatch() 方法
|
||||
|
||||
@@ -9,8 +9,6 @@ from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_asyn
|
||||
|
||||
from astrbot.core.db.po import (
|
||||
Attachment,
|
||||
CommandConfig,
|
||||
CommandConflict,
|
||||
ConversationV2,
|
||||
Persona,
|
||||
PlatformMessageHistory,
|
||||
@@ -152,7 +150,6 @@ class BaseDatabase(abc.ABC):
|
||||
title: str | None = None,
|
||||
persona_id: str | None = None,
|
||||
content: list[dict] | None = None,
|
||||
token_usage: int | None = None,
|
||||
) -> None:
|
||||
"""Update a conversation's history."""
|
||||
...
|
||||
@@ -317,76 +314,6 @@ class BaseDatabase(abc.ABC):
|
||||
"""Clear all preferences for a specific scope ID."""
|
||||
...
|
||||
|
||||
@abc.abstractmethod
|
||||
async def get_command_configs(self) -> list[CommandConfig]:
|
||||
"""Get all stored command configurations."""
|
||||
...
|
||||
|
||||
@abc.abstractmethod
|
||||
async def get_command_config(self, handler_full_name: str) -> CommandConfig | None:
|
||||
"""Fetch a single command configuration by handler."""
|
||||
...
|
||||
|
||||
@abc.abstractmethod
|
||||
async def upsert_command_config(
|
||||
self,
|
||||
handler_full_name: str,
|
||||
plugin_name: str,
|
||||
module_path: str,
|
||||
original_command: str,
|
||||
*,
|
||||
resolved_command: str | None = None,
|
||||
enabled: bool | None = None,
|
||||
keep_original_alias: bool | None = None,
|
||||
conflict_key: str | None = None,
|
||||
resolution_strategy: str | None = None,
|
||||
note: str | None = None,
|
||||
extra_data: dict | None = None,
|
||||
auto_managed: bool | None = None,
|
||||
) -> CommandConfig:
|
||||
"""Create or update a command configuration."""
|
||||
...
|
||||
|
||||
@abc.abstractmethod
|
||||
async def delete_command_config(self, handler_full_name: str) -> None:
|
||||
"""Delete a single command configuration."""
|
||||
...
|
||||
|
||||
@abc.abstractmethod
|
||||
async def delete_command_configs(self, handler_full_names: list[str]) -> None:
|
||||
"""Bulk delete command configurations."""
|
||||
...
|
||||
|
||||
@abc.abstractmethod
|
||||
async def list_command_conflicts(
|
||||
self,
|
||||
status: str | None = None,
|
||||
) -> list[CommandConflict]:
|
||||
"""List recorded command conflict entries."""
|
||||
...
|
||||
|
||||
@abc.abstractmethod
|
||||
async def upsert_command_conflict(
|
||||
self,
|
||||
conflict_key: str,
|
||||
handler_full_name: str,
|
||||
plugin_name: str,
|
||||
*,
|
||||
status: str | None = None,
|
||||
resolution: str | None = None,
|
||||
resolved_command: str | None = None,
|
||||
note: str | None = None,
|
||||
extra_data: dict | None = None,
|
||||
auto_generated: bool | None = None,
|
||||
) -> CommandConflict:
|
||||
"""Create or update a conflict record."""
|
||||
...
|
||||
|
||||
@abc.abstractmethod
|
||||
async def delete_command_conflicts(self, ids: list[int]) -> None:
|
||||
"""Delete conflict records."""
|
||||
...
|
||||
|
||||
# @abc.abstractmethod
|
||||
# async def insert_llm_message(
|
||||
# self,
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
"""Migration script to add token_usage column to conversations table.
|
||||
|
||||
This migration adds the token_usage field to track token consumption for each conversation.
|
||||
|
||||
Changes:
|
||||
- Adds token_usage column to conversations table (default: 0)
|
||||
"""
|
||||
|
||||
from sqlalchemy import text
|
||||
|
||||
from astrbot.api import logger, sp
|
||||
from astrbot.core.db import BaseDatabase
|
||||
|
||||
|
||||
async def migrate_token_usage(db_helper: BaseDatabase):
|
||||
"""Add token_usage column to conversations table.
|
||||
|
||||
This migration adds a new column to track token consumption in conversations.
|
||||
"""
|
||||
# 检查是否已经完成迁移
|
||||
migration_done = await db_helper.get_preference(
|
||||
"global", "global", "migration_done_token_usage_1"
|
||||
)
|
||||
if migration_done:
|
||||
return
|
||||
|
||||
logger.info("开始执行数据库迁移(添加 conversations.token_usage 列)...")
|
||||
|
||||
# 这里只适配了 SQLite。因为截止至这一版本,AstrBot 仅支持 SQLite。
|
||||
|
||||
try:
|
||||
async with db_helper.get_db() as session:
|
||||
# 检查列是否已存在
|
||||
result = await session.execute(text("PRAGMA table_info(conversations)"))
|
||||
columns = result.fetchall()
|
||||
column_names = [col[1] for col in columns]
|
||||
|
||||
if "token_usage" in column_names:
|
||||
logger.info("token_usage 列已存在,跳过迁移")
|
||||
await sp.put_async(
|
||||
"global", "global", "migration_done_token_usage_1", True
|
||||
)
|
||||
return
|
||||
|
||||
# 添加 token_usage 列
|
||||
await session.execute(
|
||||
text(
|
||||
"ALTER TABLE conversations ADD COLUMN token_usage INTEGER NOT NULL DEFAULT 0"
|
||||
)
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
logger.info("token_usage 列添加成功")
|
||||
|
||||
# 标记迁移完成
|
||||
await sp.put_async("global", "global", "migration_done_token_usage_1", True)
|
||||
logger.info("token_usage 迁移完成")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"迁移过程中发生错误: {e}", exc_info=True)
|
||||
raise
|
||||
@@ -54,11 +54,6 @@ class ConversationV2(SQLModel, table=True):
|
||||
)
|
||||
title: str | None = Field(default=None, max_length=255)
|
||||
persona_id: str | None = Field(default=None)
|
||||
token_usage: int = Field(default=0, nullable=False)
|
||||
"""content is a list of OpenAI-formated messages in list[dict] format.
|
||||
token_usage is the total token value of the messages.
|
||||
when 0, will use estimated token counter.
|
||||
"""
|
||||
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
@@ -239,65 +234,6 @@ class Attachment(SQLModel, table=True):
|
||||
)
|
||||
|
||||
|
||||
class CommandConfig(SQLModel, table=True):
|
||||
"""Per-command configuration overrides for dashboard management."""
|
||||
|
||||
__tablename__ = "command_configs" # type: ignore
|
||||
|
||||
handler_full_name: str = Field(
|
||||
primary_key=True,
|
||||
max_length=512,
|
||||
)
|
||||
plugin_name: str = Field(nullable=False, max_length=255)
|
||||
module_path: str = Field(nullable=False, max_length=255)
|
||||
original_command: str = Field(nullable=False, max_length=255)
|
||||
resolved_command: str | None = Field(default=None, max_length=255)
|
||||
enabled: bool = Field(default=True, nullable=False)
|
||||
keep_original_alias: bool = Field(default=False, nullable=False)
|
||||
conflict_key: str | None = Field(default=None, max_length=255)
|
||||
resolution_strategy: str | None = Field(default=None, max_length=64)
|
||||
note: str | None = Field(default=None, sa_type=Text)
|
||||
extra_data: dict | None = Field(default=None, sa_type=JSON)
|
||||
auto_managed: bool = Field(default=False, nullable=False)
|
||||
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
|
||||
updated_at: datetime = Field(
|
||||
default_factory=lambda: datetime.now(timezone.utc),
|
||||
sa_column_kwargs={"onupdate": datetime.now(timezone.utc)},
|
||||
)
|
||||
|
||||
|
||||
class CommandConflict(SQLModel, table=True):
|
||||
"""Conflict tracking for duplicated command names."""
|
||||
|
||||
__tablename__ = "command_conflicts" # type: ignore
|
||||
|
||||
id: int | None = Field(
|
||||
default=None, primary_key=True, sa_column_kwargs={"autoincrement": True}
|
||||
)
|
||||
conflict_key: str = Field(nullable=False, max_length=255)
|
||||
handler_full_name: str = Field(nullable=False, max_length=512)
|
||||
plugin_name: str = Field(nullable=False, max_length=255)
|
||||
status: str = Field(default="pending", max_length=32)
|
||||
resolution: str | None = Field(default=None, max_length=64)
|
||||
resolved_command: str | None = Field(default=None, max_length=255)
|
||||
note: str | None = Field(default=None, sa_type=Text)
|
||||
extra_data: dict | None = Field(default=None, sa_type=JSON)
|
||||
auto_generated: bool = Field(default=False, nullable=False)
|
||||
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
|
||||
updated_at: datetime = Field(
|
||||
default_factory=lambda: datetime.now(timezone.utc),
|
||||
sa_column_kwargs={"onupdate": datetime.now(timezone.utc)},
|
||||
)
|
||||
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"conflict_key",
|
||||
"handler_full_name",
|
||||
name="uix_conflict_handler",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Conversation:
|
||||
"""LLM 对话类
|
||||
@@ -318,8 +254,6 @@ class Conversation:
|
||||
persona_id: str | None = ""
|
||||
created_at: int = 0
|
||||
updated_at: int = 0
|
||||
token_usage: int = 0
|
||||
"""对话的总 token 数量。AstrBot 会保留最近一次 LLM 请求返回的总 token 数,方便统计。token_usage 可能为 0,表示未知。"""
|
||||
|
||||
|
||||
class Personality(TypedDict):
|
||||
|
||||
+1
-245
@@ -1,7 +1,6 @@
|
||||
import asyncio
|
||||
import threading
|
||||
import typing as T
|
||||
from collections.abc import Awaitable, Callable
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
from sqlalchemy import CursorResult
|
||||
@@ -11,8 +10,6 @@ from sqlmodel import col, delete, desc, func, or_, select, text, update
|
||||
from astrbot.core.db import BaseDatabase
|
||||
from astrbot.core.db.po import (
|
||||
Attachment,
|
||||
CommandConfig,
|
||||
CommandConflict,
|
||||
ConversationV2,
|
||||
Persona,
|
||||
PlatformMessageHistory,
|
||||
@@ -29,7 +26,6 @@ from astrbot.core.db.po import (
|
||||
)
|
||||
|
||||
NOT_GIVEN = T.TypeVar("NOT_GIVEN")
|
||||
TxResult = T.TypeVar("TxResult")
|
||||
|
||||
|
||||
class SQLiteDatabase(BaseDatabase):
|
||||
@@ -241,9 +237,7 @@ class SQLiteDatabase(BaseDatabase):
|
||||
session.add(new_conversation)
|
||||
return new_conversation
|
||||
|
||||
async def update_conversation(
|
||||
self, cid, title=None, persona_id=None, content=None, token_usage=None
|
||||
):
|
||||
async def update_conversation(self, cid, title=None, persona_id=None, content=None):
|
||||
async with self.get_db() as session:
|
||||
session: AsyncSession
|
||||
async with session.begin():
|
||||
@@ -257,8 +251,6 @@ class SQLiteDatabase(BaseDatabase):
|
||||
values["persona_id"] = persona_id
|
||||
if content is not None:
|
||||
values["content"] = content
|
||||
if token_usage is not None:
|
||||
values["token_usage"] = token_usage
|
||||
if not values:
|
||||
return None
|
||||
query = query.values(**values)
|
||||
@@ -678,242 +670,6 @@ class SQLiteDatabase(BaseDatabase):
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
# ====
|
||||
# Command Configuration & Conflict Tracking
|
||||
# ====
|
||||
|
||||
async def _run_in_tx(
|
||||
self,
|
||||
fn: Callable[[AsyncSession], Awaitable[TxResult]],
|
||||
) -> TxResult:
|
||||
async with self.get_db() as session:
|
||||
session: AsyncSession
|
||||
async with session.begin():
|
||||
return await fn(session)
|
||||
|
||||
@staticmethod
|
||||
def _apply_updates(model, **updates) -> None:
|
||||
for field, value in updates.items():
|
||||
if value is not None:
|
||||
setattr(model, field, value)
|
||||
|
||||
@staticmethod
|
||||
def _new_command_config(
|
||||
handler_full_name: str,
|
||||
plugin_name: str,
|
||||
module_path: str,
|
||||
original_command: str,
|
||||
*,
|
||||
resolved_command: str | None = None,
|
||||
enabled: bool | None = None,
|
||||
keep_original_alias: bool | None = None,
|
||||
conflict_key: str | None = None,
|
||||
resolution_strategy: str | None = None,
|
||||
note: str | None = None,
|
||||
extra_data: dict | None = None,
|
||||
auto_managed: bool | None = None,
|
||||
) -> CommandConfig:
|
||||
return CommandConfig(
|
||||
handler_full_name=handler_full_name,
|
||||
plugin_name=plugin_name,
|
||||
module_path=module_path,
|
||||
original_command=original_command,
|
||||
resolved_command=resolved_command,
|
||||
enabled=True if enabled is None else enabled,
|
||||
keep_original_alias=False
|
||||
if keep_original_alias is None
|
||||
else keep_original_alias,
|
||||
conflict_key=conflict_key or original_command,
|
||||
resolution_strategy=resolution_strategy,
|
||||
note=note,
|
||||
extra_data=extra_data,
|
||||
auto_managed=bool(auto_managed),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _new_command_conflict(
|
||||
conflict_key: str,
|
||||
handler_full_name: str,
|
||||
plugin_name: str,
|
||||
*,
|
||||
status: str | None = None,
|
||||
resolution: str | None = None,
|
||||
resolved_command: str | None = None,
|
||||
note: str | None = None,
|
||||
extra_data: dict | None = None,
|
||||
auto_generated: bool | None = None,
|
||||
) -> CommandConflict:
|
||||
return CommandConflict(
|
||||
conflict_key=conflict_key,
|
||||
handler_full_name=handler_full_name,
|
||||
plugin_name=plugin_name,
|
||||
status=status or "pending",
|
||||
resolution=resolution,
|
||||
resolved_command=resolved_command,
|
||||
note=note,
|
||||
extra_data=extra_data,
|
||||
auto_generated=bool(auto_generated),
|
||||
)
|
||||
|
||||
async def get_command_configs(self) -> list[CommandConfig]:
|
||||
async with self.get_db() as session:
|
||||
session: AsyncSession
|
||||
result = await session.execute(select(CommandConfig))
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def get_command_config(
|
||||
self,
|
||||
handler_full_name: str,
|
||||
) -> CommandConfig | None:
|
||||
async with self.get_db() as session:
|
||||
session: AsyncSession
|
||||
return await session.get(CommandConfig, handler_full_name)
|
||||
|
||||
async def upsert_command_config(
|
||||
self,
|
||||
handler_full_name: str,
|
||||
plugin_name: str,
|
||||
module_path: str,
|
||||
original_command: str,
|
||||
*,
|
||||
resolved_command: str | None = None,
|
||||
enabled: bool | None = None,
|
||||
keep_original_alias: bool | None = None,
|
||||
conflict_key: str | None = None,
|
||||
resolution_strategy: str | None = None,
|
||||
note: str | None = None,
|
||||
extra_data: dict | None = None,
|
||||
auto_managed: bool | None = None,
|
||||
) -> CommandConfig:
|
||||
async def _op(session: AsyncSession) -> CommandConfig:
|
||||
config = await session.get(CommandConfig, handler_full_name)
|
||||
if not config:
|
||||
config = self._new_command_config(
|
||||
handler_full_name,
|
||||
plugin_name,
|
||||
module_path,
|
||||
original_command,
|
||||
resolved_command=resolved_command,
|
||||
enabled=enabled,
|
||||
keep_original_alias=keep_original_alias,
|
||||
conflict_key=conflict_key,
|
||||
resolution_strategy=resolution_strategy,
|
||||
note=note,
|
||||
extra_data=extra_data,
|
||||
auto_managed=auto_managed,
|
||||
)
|
||||
session.add(config)
|
||||
else:
|
||||
self._apply_updates(
|
||||
config,
|
||||
plugin_name=plugin_name,
|
||||
module_path=module_path,
|
||||
original_command=original_command,
|
||||
resolved_command=resolved_command,
|
||||
enabled=enabled,
|
||||
keep_original_alias=keep_original_alias,
|
||||
conflict_key=conflict_key,
|
||||
resolution_strategy=resolution_strategy,
|
||||
note=note,
|
||||
extra_data=extra_data,
|
||||
auto_managed=auto_managed,
|
||||
)
|
||||
await session.flush()
|
||||
await session.refresh(config)
|
||||
return config
|
||||
|
||||
return await self._run_in_tx(_op)
|
||||
|
||||
async def delete_command_config(self, handler_full_name: str) -> None:
|
||||
await self.delete_command_configs([handler_full_name])
|
||||
|
||||
async def delete_command_configs(self, handler_full_names: list[str]) -> None:
|
||||
if not handler_full_names:
|
||||
return
|
||||
|
||||
async def _op(session: AsyncSession) -> None:
|
||||
await session.execute(
|
||||
delete(CommandConfig).where(
|
||||
col(CommandConfig.handler_full_name).in_(handler_full_names),
|
||||
),
|
||||
)
|
||||
|
||||
await self._run_in_tx(_op)
|
||||
|
||||
async def list_command_conflicts(
|
||||
self,
|
||||
status: str | None = None,
|
||||
) -> list[CommandConflict]:
|
||||
async with self.get_db() as session:
|
||||
session: AsyncSession
|
||||
query = select(CommandConflict)
|
||||
if status:
|
||||
query = query.where(CommandConflict.status == status)
|
||||
result = await session.execute(query)
|
||||
return list(result.scalars().all())
|
||||
|
||||
async def upsert_command_conflict(
|
||||
self,
|
||||
conflict_key: str,
|
||||
handler_full_name: str,
|
||||
plugin_name: str,
|
||||
*,
|
||||
status: str | None = None,
|
||||
resolution: str | None = None,
|
||||
resolved_command: str | None = None,
|
||||
note: str | None = None,
|
||||
extra_data: dict | None = None,
|
||||
auto_generated: bool | None = None,
|
||||
) -> CommandConflict:
|
||||
async def _op(session: AsyncSession) -> CommandConflict:
|
||||
result = await session.execute(
|
||||
select(CommandConflict).where(
|
||||
CommandConflict.conflict_key == conflict_key,
|
||||
CommandConflict.handler_full_name == handler_full_name,
|
||||
),
|
||||
)
|
||||
record = result.scalar_one_or_none()
|
||||
if not record:
|
||||
record = self._new_command_conflict(
|
||||
conflict_key,
|
||||
handler_full_name,
|
||||
plugin_name,
|
||||
status=status,
|
||||
resolution=resolution,
|
||||
resolved_command=resolved_command,
|
||||
note=note,
|
||||
extra_data=extra_data,
|
||||
auto_generated=auto_generated,
|
||||
)
|
||||
session.add(record)
|
||||
else:
|
||||
self._apply_updates(
|
||||
record,
|
||||
plugin_name=plugin_name,
|
||||
status=status,
|
||||
resolution=resolution,
|
||||
resolved_command=resolved_command,
|
||||
note=note,
|
||||
extra_data=extra_data,
|
||||
auto_generated=auto_generated,
|
||||
)
|
||||
await session.flush()
|
||||
await session.refresh(record)
|
||||
return record
|
||||
|
||||
return await self._run_in_tx(_op)
|
||||
|
||||
async def delete_command_conflicts(self, ids: list[int]) -> None:
|
||||
if not ids:
|
||||
return
|
||||
|
||||
async def _op(session: AsyncSession) -> None:
|
||||
await session.execute(
|
||||
delete(CommandConflict).where(col(CommandConflict.id).in_(ids)),
|
||||
)
|
||||
|
||||
await self._run_in_tx(_op)
|
||||
|
||||
# ====
|
||||
# Deprecated Methods
|
||||
# ====
|
||||
|
||||
@@ -149,16 +149,8 @@ class RecursiveCharacterChunker(BaseChunker):
|
||||
分割后的文本块列表
|
||||
|
||||
"""
|
||||
if chunk_size is None:
|
||||
chunk_size = self.chunk_size
|
||||
if overlap is None:
|
||||
overlap = self.chunk_overlap
|
||||
if chunk_size <= 0:
|
||||
raise ValueError("chunk_size must be greater than 0")
|
||||
if overlap < 0:
|
||||
raise ValueError("chunk_overlap must be non-negative")
|
||||
if overlap >= chunk_size:
|
||||
raise ValueError("chunk_overlap must be less than chunk_size")
|
||||
chunk_size = chunk_size or self.chunk_size
|
||||
overlap = overlap or self.chunk_overlap
|
||||
result = []
|
||||
for i in range(0, len(text), chunk_size - overlap):
|
||||
end = min(i + chunk_size, len(text))
|
||||
|
||||
+2
-3
@@ -24,7 +24,6 @@ import asyncio
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from asyncio import Queue
|
||||
from collections import deque
|
||||
|
||||
@@ -58,7 +57,7 @@ def is_plugin_path(pathname):
|
||||
return False
|
||||
|
||||
norm_path = os.path.normpath(pathname)
|
||||
return ("data/plugins" in norm_path) or ("astrbot/builtin_stars/" in norm_path)
|
||||
return ("data/plugins" in norm_path) or ("packages/" in norm_path)
|
||||
|
||||
|
||||
def get_short_level_name(level_name):
|
||||
@@ -149,7 +148,7 @@ class LogQueueHandler(logging.Handler):
|
||||
self.log_broker.publish(
|
||||
{
|
||||
"level": record.levelname,
|
||||
"time": time.time(),
|
||||
"time": record.asctime,
|
||||
"data": log_entry,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -629,11 +629,12 @@ class Nodes(BaseMessageComponent):
|
||||
|
||||
class Json(BaseMessageComponent):
|
||||
type = ComponentType.Json
|
||||
data: dict
|
||||
data: str | dict
|
||||
resid: int | None = 0
|
||||
|
||||
def __init__(self, data: str | dict, **_):
|
||||
if isinstance(data, str):
|
||||
data = json.loads(data)
|
||||
def __init__(self, data, **_):
|
||||
if isinstance(data, dict):
|
||||
data = json.dumps(data)
|
||||
super().__init__(data=data, **_)
|
||||
|
||||
|
||||
|
||||
@@ -38,7 +38,7 @@ class AgentRequestSubStage(Stage):
|
||||
)
|
||||
return
|
||||
|
||||
if not await SessionServiceManager.should_process_llm_request(event):
|
||||
if not SessionServiceManager.should_process_llm_request(event):
|
||||
logger.debug(
|
||||
f"The session {event.unified_msg_origin} has disabled AI capability, skipping processing."
|
||||
)
|
||||
|
||||
@@ -1,12 +1,11 @@
|
||||
"""本地 Agent 模式的 LLM 调用 Stage"""
|
||||
|
||||
import asyncio
|
||||
import copy
|
||||
import json
|
||||
from collections.abc import AsyncGenerator
|
||||
|
||||
from astrbot.core import logger
|
||||
from astrbot.core.agent.message import Message
|
||||
from astrbot.core.agent.response import AgentStats
|
||||
from astrbot.core.agent.tool import ToolSet
|
||||
from astrbot.core.astr_agent_context import AstrAgentContext
|
||||
from astrbot.core.conversation_mgr import Conversation
|
||||
@@ -24,7 +23,6 @@ from astrbot.core.provider.entities import (
|
||||
)
|
||||
from astrbot.core.star.star_handler import EventType, star_map
|
||||
from astrbot.core.utils.file_extract import extract_file_moonshotai
|
||||
from astrbot.core.utils.llm_metadata import LLM_METADATAS
|
||||
from astrbot.core.utils.metrics import Metric
|
||||
from astrbot.core.utils.session_lock import session_lock_manager
|
||||
|
||||
@@ -42,6 +40,11 @@ class InternalAgentSubStage(Stage):
|
||||
self.ctx = ctx
|
||||
conf = ctx.astrbot_config
|
||||
settings = conf["provider_settings"]
|
||||
self.max_context_length = settings["max_context_length"] # int
|
||||
self.dequeue_context_length: int = min(
|
||||
max(1, settings["dequeue_context_length"]),
|
||||
self.max_context_length - 1,
|
||||
)
|
||||
self.streaming_response: bool = settings["streaming_response"]
|
||||
self.unsupported_streaming_strategy: str = settings[
|
||||
"unsupported_streaming_strategy"
|
||||
@@ -61,25 +64,6 @@ class InternalAgentSubStage(Stage):
|
||||
"moonshotai_api_key", ""
|
||||
)
|
||||
|
||||
# 上下文管理相关
|
||||
self.context_limit_reached_strategy: str = settings.get(
|
||||
"context_limit_reached_strategy", "truncate_by_turns"
|
||||
)
|
||||
self.llm_compress_instruction: str = settings.get(
|
||||
"llm_compress_instruction", ""
|
||||
)
|
||||
self.llm_compress_keep_recent: int = settings.get("llm_compress_keep_recent", 4)
|
||||
self.llm_compress_provider_id: str = settings.get(
|
||||
"llm_compress_provider_id", ""
|
||||
)
|
||||
self.max_context_length = settings["max_context_length"] # int
|
||||
self.dequeue_context_length: int = min(
|
||||
max(1, settings["dequeue_context_length"]),
|
||||
self.max_context_length - 1,
|
||||
)
|
||||
if self.dequeue_context_length <= 0:
|
||||
self.dequeue_context_length = 1
|
||||
|
||||
self.conv_manager = ctx.plugin_manager.context.conversation_manager
|
||||
|
||||
def _select_provider(self, event: AstrMessageEvent):
|
||||
@@ -182,6 +166,34 @@ class InternalAgentSubStage(Stage):
|
||||
},
|
||||
)
|
||||
|
||||
def _truncate_contexts(
|
||||
self,
|
||||
contexts: list[dict],
|
||||
) -> list[dict]:
|
||||
"""截断上下文列表,确保不超过最大长度"""
|
||||
if self.max_context_length == -1:
|
||||
return contexts
|
||||
|
||||
if len(contexts) // 2 <= self.max_context_length:
|
||||
return contexts
|
||||
|
||||
truncated_contexts = contexts[
|
||||
-(self.max_context_length - self.dequeue_context_length + 1) * 2 :
|
||||
]
|
||||
# 找到第一个role 为 user 的索引,确保上下文格式正确
|
||||
index = next(
|
||||
(
|
||||
i
|
||||
for i, item in enumerate(truncated_contexts)
|
||||
if item.get("role") == "user"
|
||||
),
|
||||
None,
|
||||
)
|
||||
if index is not None and index > 0:
|
||||
truncated_contexts = truncated_contexts[index:]
|
||||
|
||||
return truncated_contexts
|
||||
|
||||
def _modalities_fix(
|
||||
self,
|
||||
provider: Provider,
|
||||
@@ -282,8 +294,6 @@ class InternalAgentSubStage(Stage):
|
||||
event: AstrMessageEvent,
|
||||
req: ProviderRequest,
|
||||
llm_response: LLMResponse | None,
|
||||
all_messages: list[Message],
|
||||
runner_stats: AgentStats | None,
|
||||
):
|
||||
if (
|
||||
not req
|
||||
@@ -297,255 +307,217 @@ class InternalAgentSubStage(Stage):
|
||||
logger.debug("LLM 响应为空,不保存记录。")
|
||||
return
|
||||
|
||||
# using agent context messages to save to history
|
||||
message_to_save = []
|
||||
for message in all_messages:
|
||||
if message.role == "system":
|
||||
# we do not save system messages to history
|
||||
continue
|
||||
if message.role in ["assistant", "user"] and getattr(
|
||||
message, "_no_save", None
|
||||
):
|
||||
# we do not save user and assistant messages that are marked as _no_save
|
||||
continue
|
||||
message_to_save.append(message.model_dump())
|
||||
|
||||
# get token usage from agent runner stats
|
||||
token_usage = None
|
||||
if runner_stats:
|
||||
token_usage = runner_stats.token_usage.total
|
||||
if req.contexts is None:
|
||||
req.contexts = []
|
||||
|
||||
# 历史上下文
|
||||
messages = copy.deepcopy(req.contexts)
|
||||
# 这一轮对话请求的用户输入
|
||||
messages.append(await req.assemble_context())
|
||||
# 这一轮对话的 LLM 响应
|
||||
if req.tool_calls_result:
|
||||
if not isinstance(req.tool_calls_result, list):
|
||||
messages.extend(req.tool_calls_result.to_openai_messages())
|
||||
elif isinstance(req.tool_calls_result, list):
|
||||
for tcr in req.tool_calls_result:
|
||||
messages.extend(tcr.to_openai_messages())
|
||||
messages.append({"role": "assistant", "content": llm_response.completion_text})
|
||||
messages = list(filter(lambda item: "_no_save" not in item, messages))
|
||||
await self.conv_manager.update_conversation(
|
||||
event.unified_msg_origin,
|
||||
req.conversation.cid,
|
||||
history=message_to_save,
|
||||
token_usage=token_usage,
|
||||
history=messages,
|
||||
)
|
||||
|
||||
def _get_compress_provider(self) -> Provider | None:
|
||||
if not self.llm_compress_provider_id:
|
||||
return None
|
||||
if self.context_limit_reached_strategy != "llm_compress":
|
||||
return None
|
||||
provider = self.ctx.plugin_manager.context.get_provider_by_id(
|
||||
self.llm_compress_provider_id,
|
||||
)
|
||||
if provider is None:
|
||||
logger.warning(
|
||||
f"未找到指定的上下文压缩模型 {self.llm_compress_provider_id},将跳过压缩。",
|
||||
)
|
||||
return None
|
||||
if not isinstance(provider, Provider):
|
||||
logger.warning(
|
||||
f"指定的上下文压缩模型 {self.llm_compress_provider_id} 不是对话模型,将跳过压缩。"
|
||||
)
|
||||
return None
|
||||
return provider
|
||||
def _fix_messages(self, messages: list[dict]) -> list[dict]:
|
||||
"""验证并且修复上下文"""
|
||||
fixed_messages = []
|
||||
for message in messages:
|
||||
if message.get("role") == "tool":
|
||||
# tool block 前面必须要有 user 和 assistant block
|
||||
if len(fixed_messages) < 2:
|
||||
# 这种情况可能是上下文被截断导致的
|
||||
# 我们直接将之前的上下文都清空
|
||||
fixed_messages = []
|
||||
else:
|
||||
fixed_messages.append(message)
|
||||
else:
|
||||
fixed_messages.append(message)
|
||||
return fixed_messages
|
||||
|
||||
async def process(
|
||||
self, event: AstrMessageEvent, provider_wake_prefix: str
|
||||
) -> AsyncGenerator[None, None]:
|
||||
req: ProviderRequest | None = None
|
||||
|
||||
try:
|
||||
provider = self._select_provider(event)
|
||||
if provider is None:
|
||||
return
|
||||
if not isinstance(provider, Provider):
|
||||
logger.error(
|
||||
f"选择的提供商类型无效({type(provider)}),跳过 LLM 请求处理。"
|
||||
provider = self._select_provider(event)
|
||||
if provider is None:
|
||||
return
|
||||
if not isinstance(provider, Provider):
|
||||
logger.error(f"选择的提供商类型无效({type(provider)}),跳过 LLM 请求处理。")
|
||||
return
|
||||
|
||||
streaming_response = self.streaming_response
|
||||
if (enable_streaming := event.get_extra("enable_streaming")) is not None:
|
||||
streaming_response = bool(enable_streaming)
|
||||
|
||||
logger.debug("ready to request llm provider")
|
||||
async with session_lock_manager.acquire_lock(event.unified_msg_origin):
|
||||
logger.debug("acquired session lock for llm request")
|
||||
if event.get_extra("provider_request"):
|
||||
req = event.get_extra("provider_request")
|
||||
assert isinstance(req, ProviderRequest), (
|
||||
"provider_request 必须是 ProviderRequest 类型。"
|
||||
)
|
||||
return
|
||||
|
||||
streaming_response = self.streaming_response
|
||||
if (enable_streaming := event.get_extra("enable_streaming")) is not None:
|
||||
streaming_response = bool(enable_streaming)
|
||||
if req.conversation:
|
||||
req.contexts = json.loads(req.conversation.history)
|
||||
|
||||
logger.debug("ready to request llm provider")
|
||||
|
||||
# 通知等待调用 LLM(在获取锁之前)
|
||||
await call_event_hook(event, EventType.OnWaitingLLMRequestEvent)
|
||||
|
||||
async with session_lock_manager.acquire_lock(event.unified_msg_origin):
|
||||
logger.debug("acquired session lock for llm request")
|
||||
if event.get_extra("provider_request"):
|
||||
req = event.get_extra("provider_request")
|
||||
assert isinstance(req, ProviderRequest), (
|
||||
"provider_request 必须是 ProviderRequest 类型。"
|
||||
)
|
||||
|
||||
if req.conversation:
|
||||
req.contexts = json.loads(req.conversation.history)
|
||||
|
||||
else:
|
||||
req = ProviderRequest()
|
||||
req.prompt = ""
|
||||
req.image_urls = []
|
||||
if sel_model := event.get_extra("selected_model"):
|
||||
req.model = sel_model
|
||||
if provider_wake_prefix and not event.message_str.startswith(
|
||||
provider_wake_prefix
|
||||
):
|
||||
return
|
||||
|
||||
req.prompt = event.message_str[len(provider_wake_prefix) :]
|
||||
# func_tool selection 现在已经转移到 astrbot/builtin_stars/astrbot 插件中进行选择。
|
||||
# req.func_tool = self.ctx.plugin_manager.context.get_llm_tool_manager()
|
||||
for comp in event.message_obj.message:
|
||||
if isinstance(comp, Image):
|
||||
image_path = await comp.convert_to_file_path()
|
||||
req.image_urls.append(image_path)
|
||||
|
||||
conversation = await self._get_session_conv(event)
|
||||
req.conversation = conversation
|
||||
req.contexts = json.loads(conversation.history)
|
||||
|
||||
event.set_extra("provider_request", req)
|
||||
|
||||
# fix contexts json str
|
||||
if isinstance(req.contexts, str):
|
||||
req.contexts = json.loads(req.contexts)
|
||||
|
||||
# apply file extract
|
||||
if self.file_extract_enabled:
|
||||
try:
|
||||
await self._apply_file_extract(event, req)
|
||||
except Exception as e:
|
||||
logger.error(f"Error occurred while applying file extract: {e}")
|
||||
|
||||
if not req.prompt and not req.image_urls:
|
||||
else:
|
||||
req = ProviderRequest()
|
||||
req.prompt = ""
|
||||
req.image_urls = []
|
||||
if sel_model := event.get_extra("selected_model"):
|
||||
req.model = sel_model
|
||||
if provider_wake_prefix and not event.message_str.startswith(
|
||||
provider_wake_prefix
|
||||
):
|
||||
return
|
||||
|
||||
# call event hook
|
||||
if await call_event_hook(event, EventType.OnLLMRequestEvent, req):
|
||||
return
|
||||
req.prompt = event.message_str[len(provider_wake_prefix) :]
|
||||
# func_tool selection 现在已经转移到 packages/astrbot 插件中进行选择。
|
||||
# req.func_tool = self.ctx.plugin_manager.context.get_llm_tool_manager()
|
||||
for comp in event.message_obj.message:
|
||||
if isinstance(comp, Image):
|
||||
image_path = await comp.convert_to_file_path()
|
||||
req.image_urls.append(image_path)
|
||||
|
||||
# apply knowledge base feature
|
||||
await self._apply_kb(event, req)
|
||||
conversation = await self._get_session_conv(event)
|
||||
req.conversation = conversation
|
||||
req.contexts = json.loads(conversation.history)
|
||||
|
||||
# truncate contexts to fit max length
|
||||
# NOW moved to ContextManager inside ToolLoopAgentRunner
|
||||
# if req.contexts:
|
||||
# req.contexts = self._truncate_contexts(req.contexts)
|
||||
# self._fix_messages(req.contexts)
|
||||
event.set_extra("provider_request", req)
|
||||
|
||||
# session_id
|
||||
if not req.session_id:
|
||||
req.session_id = event.unified_msg_origin
|
||||
# fix contexts json str
|
||||
if isinstance(req.contexts, str):
|
||||
req.contexts = json.loads(req.contexts)
|
||||
|
||||
# check provider modalities, if provider does not support image/tool_use, clear them in request.
|
||||
self._modalities_fix(provider, req)
|
||||
# apply file extract
|
||||
if self.file_extract_enabled:
|
||||
try:
|
||||
await self._apply_file_extract(event, req)
|
||||
except Exception as e:
|
||||
logger.error(f"Error occurred while applying file extract: {e}")
|
||||
|
||||
# filter tools, only keep tools from this pipeline's selected plugins
|
||||
self._plugin_tool_fix(event, req)
|
||||
if not req.prompt and not req.image_urls:
|
||||
return
|
||||
|
||||
stream_to_general = (
|
||||
self.unsupported_streaming_strategy == "turn_off"
|
||||
and not event.platform_meta.support_streaming_message
|
||||
)
|
||||
# call event hook
|
||||
if await call_event_hook(event, EventType.OnLLMRequestEvent, req):
|
||||
return
|
||||
|
||||
# run agent
|
||||
agent_runner = AgentRunner()
|
||||
logger.debug(
|
||||
f"handle provider[id: {provider.provider_config['id']}] request: {req}",
|
||||
)
|
||||
astr_agent_ctx = AstrAgentContext(
|
||||
context=self.ctx.plugin_manager.context,
|
||||
event=event,
|
||||
)
|
||||
# apply knowledge base feature
|
||||
await self._apply_kb(event, req)
|
||||
|
||||
# inject model context length limit
|
||||
if provider.provider_config.get("max_context_tokens", 0) <= 0:
|
||||
model = provider.get_model()
|
||||
if model_info := LLM_METADATAS.get(model):
|
||||
provider.provider_config["max_context_tokens"] = model_info[
|
||||
"limit"
|
||||
]["context"]
|
||||
# truncate contexts to fit max length
|
||||
if req.contexts:
|
||||
req.contexts = self._truncate_contexts(req.contexts)
|
||||
self._fix_messages(req.contexts)
|
||||
|
||||
await agent_runner.reset(
|
||||
provider=provider,
|
||||
request=req,
|
||||
run_context=AgentContextWrapper(
|
||||
context=astr_agent_ctx,
|
||||
tool_call_timeout=self.tool_call_timeout,
|
||||
),
|
||||
tool_executor=FunctionToolExecutor(),
|
||||
agent_hooks=MAIN_AGENT_HOOKS,
|
||||
streaming=streaming_response,
|
||||
llm_compress_instruction=self.llm_compress_instruction,
|
||||
llm_compress_keep_recent=self.llm_compress_keep_recent,
|
||||
llm_compress_provider=self._get_compress_provider(),
|
||||
truncate_turns=self.dequeue_context_length,
|
||||
enforce_max_turns=self.max_context_length,
|
||||
)
|
||||
# session_id
|
||||
if not req.session_id:
|
||||
req.session_id = event.unified_msg_origin
|
||||
|
||||
if streaming_response and not stream_to_general:
|
||||
# 流式响应
|
||||
event.set_result(
|
||||
MessageEventResult()
|
||||
.set_result_content_type(ResultContentType.STREAMING_RESULT)
|
||||
.set_async_stream(
|
||||
run_agent(
|
||||
agent_runner,
|
||||
self.max_step,
|
||||
self.show_tool_use,
|
||||
show_reasoning=self.show_reasoning,
|
||||
),
|
||||
),
|
||||
)
|
||||
yield
|
||||
if agent_runner.done():
|
||||
if final_llm_resp := agent_runner.get_final_llm_resp():
|
||||
if final_llm_resp.completion_text:
|
||||
chain = (
|
||||
MessageChain()
|
||||
.message(final_llm_resp.completion_text)
|
||||
.chain
|
||||
)
|
||||
elif final_llm_resp.result_chain:
|
||||
chain = final_llm_resp.result_chain.chain
|
||||
else:
|
||||
chain = MessageChain().chain
|
||||
event.set_result(
|
||||
MessageEventResult(
|
||||
chain=chain,
|
||||
result_content_type=ResultContentType.STREAMING_FINISH,
|
||||
),
|
||||
)
|
||||
else:
|
||||
async for _ in run_agent(
|
||||
agent_runner,
|
||||
self.max_step,
|
||||
self.show_tool_use,
|
||||
stream_to_general,
|
||||
show_reasoning=self.show_reasoning,
|
||||
):
|
||||
yield
|
||||
# check provider modalities, if provider does not support image/tool_use, clear them in request.
|
||||
self._modalities_fix(provider, req)
|
||||
|
||||
await self._save_to_history(
|
||||
event,
|
||||
req,
|
||||
agent_runner.get_final_llm_resp(),
|
||||
agent_runner.run_context.messages,
|
||||
agent_runner.stats,
|
||||
)
|
||||
# filter tools, only keep tools from this pipeline's selected plugins
|
||||
self._plugin_tool_fix(event, req)
|
||||
|
||||
# 异步处理 WebChat 特殊情况
|
||||
if event.get_platform_name() == "webchat":
|
||||
asyncio.create_task(self._handle_webchat(event, req, provider))
|
||||
stream_to_general = (
|
||||
self.unsupported_streaming_strategy == "turn_off"
|
||||
and not event.platform_meta.support_streaming_message
|
||||
)
|
||||
# 备份 req.contexts
|
||||
backup_contexts = copy.deepcopy(req.contexts)
|
||||
|
||||
asyncio.create_task(
|
||||
Metric.upload(
|
||||
llm_tick=1,
|
||||
model_name=agent_runner.provider.get_model(),
|
||||
provider_type=agent_runner.provider.meta().type,
|
||||
# run agent
|
||||
agent_runner = AgentRunner()
|
||||
logger.debug(
|
||||
f"handle provider[id: {provider.provider_config['id']}] request: {req}",
|
||||
)
|
||||
astr_agent_ctx = AstrAgentContext(
|
||||
context=self.ctx.plugin_manager.context,
|
||||
event=event,
|
||||
)
|
||||
await agent_runner.reset(
|
||||
provider=provider,
|
||||
request=req,
|
||||
run_context=AgentContextWrapper(
|
||||
context=astr_agent_ctx,
|
||||
tool_call_timeout=self.tool_call_timeout,
|
||||
),
|
||||
tool_executor=FunctionToolExecutor(),
|
||||
agent_hooks=MAIN_AGENT_HOOKS,
|
||||
streaming=streaming_response,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error occurred while processing agent: {e}")
|
||||
await event.send(
|
||||
MessageChain().message(
|
||||
f"Error occurred while processing agent request: {e}"
|
||||
if streaming_response and not stream_to_general:
|
||||
# 流式响应
|
||||
event.set_result(
|
||||
MessageEventResult()
|
||||
.set_result_content_type(ResultContentType.STREAMING_RESULT)
|
||||
.set_async_stream(
|
||||
run_agent(
|
||||
agent_runner,
|
||||
self.max_step,
|
||||
self.show_tool_use,
|
||||
show_reasoning=self.show_reasoning,
|
||||
),
|
||||
),
|
||||
)
|
||||
)
|
||||
yield
|
||||
if agent_runner.done():
|
||||
if final_llm_resp := agent_runner.get_final_llm_resp():
|
||||
if final_llm_resp.completion_text:
|
||||
chain = (
|
||||
MessageChain()
|
||||
.message(final_llm_resp.completion_text)
|
||||
.chain
|
||||
)
|
||||
elif final_llm_resp.result_chain:
|
||||
chain = final_llm_resp.result_chain.chain
|
||||
else:
|
||||
chain = MessageChain().chain
|
||||
event.set_result(
|
||||
MessageEventResult(
|
||||
chain=chain,
|
||||
result_content_type=ResultContentType.STREAMING_FINISH,
|
||||
),
|
||||
)
|
||||
else:
|
||||
async for _ in run_agent(
|
||||
agent_runner,
|
||||
self.max_step,
|
||||
self.show_tool_use,
|
||||
stream_to_general,
|
||||
show_reasoning=self.show_reasoning,
|
||||
):
|
||||
yield
|
||||
|
||||
# 恢复备份的 contexts
|
||||
req.contexts = backup_contexts
|
||||
|
||||
await self._save_to_history(event, req, agent_runner.get_final_llm_resp())
|
||||
|
||||
# 异步处理 WebChat 特殊情况
|
||||
if event.get_platform_name() == "webchat":
|
||||
asyncio.create_task(self._handle_webchat(event, req, provider))
|
||||
|
||||
asyncio.create_task(
|
||||
Metric.upload(
|
||||
llm_tick=1,
|
||||
model_name=agent_runner.provider.get_model(),
|
||||
provider_type=agent_runner.provider.meta().type,
|
||||
),
|
||||
)
|
||||
|
||||
@@ -119,7 +119,7 @@ class RespondStage(Stage):
|
||||
|
||||
if (result := event.get_result()) is None:
|
||||
return False
|
||||
if self.only_llm_result and not result.is_llm_result():
|
||||
if self.only_llm_result and result.is_llm_result():
|
||||
return False
|
||||
|
||||
if event.get_platform_name() in [
|
||||
@@ -158,11 +158,7 @@ class RespondStage(Stage):
|
||||
result = event.get_result()
|
||||
if result is None:
|
||||
return
|
||||
if event.get_extra("_streaming_finished", False):
|
||||
# prevent some plugin make result content type to LLM_RESULT after streaming finished, lead to send again
|
||||
return
|
||||
if result.result_content_type == ResultContentType.STREAMING_FINISH:
|
||||
event.set_extra("_streaming_finished", True)
|
||||
return
|
||||
|
||||
logger.info(
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import random
|
||||
import re
|
||||
import time
|
||||
import traceback
|
||||
@@ -43,18 +42,6 @@ class ResultDecorateStage(Stage):
|
||||
"forward_threshold"
|
||||
]
|
||||
|
||||
trigger_probability = ctx.astrbot_config["provider_tts_settings"].get(
|
||||
"trigger_probability",
|
||||
1,
|
||||
)
|
||||
try:
|
||||
self.tts_trigger_probability = max(
|
||||
0.0,
|
||||
min(float(trigger_probability), 1.0),
|
||||
)
|
||||
except (TypeError, ValueError):
|
||||
self.tts_trigger_probability = 1.0
|
||||
|
||||
# 分段回复
|
||||
self.words_count_threshold = int(
|
||||
ctx.astrbot_config["platform_settings"]["segmented_reply"][
|
||||
@@ -98,9 +85,6 @@ class ResultDecorateStage(Stage):
|
||||
self.content_safe_check_stage = stage_cls()
|
||||
await self.content_safe_check_stage.initialize(ctx)
|
||||
|
||||
provider_cfg = ctx.astrbot_config.get("provider_settings", {})
|
||||
self.show_reasoning = provider_cfg.get("display_reasoning_text", False)
|
||||
|
||||
def _split_text_by_words(self, text: str) -> list[str]:
|
||||
"""使用分段词列表分段文本"""
|
||||
if not self.split_words_pattern:
|
||||
@@ -257,75 +241,63 @@ class ResultDecorateStage(Stage):
|
||||
event.unified_msg_origin,
|
||||
)
|
||||
|
||||
should_tts = (
|
||||
bool(self.ctx.astrbot_config["provider_tts_settings"]["enable"])
|
||||
and result.is_llm_result()
|
||||
and await SessionServiceManager.should_process_tts_request(event)
|
||||
and random.random() <= self.tts_trigger_probability
|
||||
and tts_provider
|
||||
)
|
||||
if should_tts and not tts_provider:
|
||||
logger.warning(
|
||||
f"会话 {event.unified_msg_origin} 未配置文本转语音模型。",
|
||||
)
|
||||
|
||||
if (
|
||||
not should_tts
|
||||
and self.show_reasoning
|
||||
and event.get_extra("_llm_reasoning_content")
|
||||
self.ctx.astrbot_config["provider_tts_settings"]["enable"]
|
||||
and result.is_llm_result()
|
||||
and SessionServiceManager.should_process_tts_request(event)
|
||||
):
|
||||
# inject reasoning content to chain
|
||||
reasoning_content = event.get_extra("_llm_reasoning_content")
|
||||
result.chain.insert(0, Plain(f"🤔 思考: {reasoning_content}\n"))
|
||||
if not tts_provider:
|
||||
logger.warning(
|
||||
f"会话 {event.unified_msg_origin} 未配置文本转语音模型。",
|
||||
)
|
||||
else:
|
||||
new_chain = []
|
||||
for comp in result.chain:
|
||||
if isinstance(comp, Plain) and len(comp.text) > 1:
|
||||
try:
|
||||
logger.info(f"TTS 请求: {comp.text}")
|
||||
audio_path = await tts_provider.get_audio(comp.text)
|
||||
logger.info(f"TTS 结果: {audio_path}")
|
||||
if not audio_path:
|
||||
logger.error(
|
||||
f"由于 TTS 音频文件未找到,消息段转语音失败: {comp.text}",
|
||||
)
|
||||
new_chain.append(comp)
|
||||
continue
|
||||
|
||||
if should_tts and tts_provider:
|
||||
new_chain = []
|
||||
for comp in result.chain:
|
||||
if isinstance(comp, Plain) and len(comp.text) > 1:
|
||||
try:
|
||||
logger.info(f"TTS 请求: {comp.text}")
|
||||
audio_path = await tts_provider.get_audio(comp.text)
|
||||
logger.info(f"TTS 结果: {audio_path}")
|
||||
if not audio_path:
|
||||
logger.error(
|
||||
f"由于 TTS 音频文件未找到,消息段转语音失败: {comp.text}",
|
||||
use_file_service = self.ctx.astrbot_config[
|
||||
"provider_tts_settings"
|
||||
]["use_file_service"]
|
||||
callback_api_base = self.ctx.astrbot_config[
|
||||
"callback_api_base"
|
||||
]
|
||||
dual_output = self.ctx.astrbot_config[
|
||||
"provider_tts_settings"
|
||||
]["dual_output"]
|
||||
|
||||
url = None
|
||||
if use_file_service and callback_api_base:
|
||||
token = await file_token_service.register_file(
|
||||
audio_path,
|
||||
)
|
||||
url = f"{callback_api_base}/api/file/{token}"
|
||||
logger.debug(f"已注册:{url}")
|
||||
|
||||
new_chain.append(
|
||||
Record(
|
||||
file=url or audio_path,
|
||||
url=url or audio_path,
|
||||
),
|
||||
)
|
||||
if dual_output:
|
||||
new_chain.append(comp)
|
||||
except Exception:
|
||||
logger.error(traceback.format_exc())
|
||||
logger.error("TTS 失败,使用文本发送。")
|
||||
new_chain.append(comp)
|
||||
continue
|
||||
|
||||
use_file_service = self.ctx.astrbot_config[
|
||||
"provider_tts_settings"
|
||||
]["use_file_service"]
|
||||
callback_api_base = self.ctx.astrbot_config[
|
||||
"callback_api_base"
|
||||
]
|
||||
dual_output = self.ctx.astrbot_config[
|
||||
"provider_tts_settings"
|
||||
]["dual_output"]
|
||||
|
||||
url = None
|
||||
if use_file_service and callback_api_base:
|
||||
token = await file_token_service.register_file(
|
||||
audio_path,
|
||||
)
|
||||
url = f"{callback_api_base}/api/file/{token}"
|
||||
logger.debug(f"已注册:{url}")
|
||||
|
||||
new_chain.append(
|
||||
Record(
|
||||
file=url or audio_path,
|
||||
url=url or audio_path,
|
||||
),
|
||||
)
|
||||
if dual_output:
|
||||
new_chain.append(comp)
|
||||
except Exception:
|
||||
logger.error(traceback.format_exc())
|
||||
logger.error("TTS 失败,使用文本发送。")
|
||||
else:
|
||||
new_chain.append(comp)
|
||||
else:
|
||||
new_chain.append(comp)
|
||||
result.chain = new_chain
|
||||
result.chain = new_chain
|
||||
|
||||
# 文本转图片
|
||||
elif (
|
||||
|
||||
@@ -21,7 +21,7 @@ class SessionStatusCheckStage(Stage):
|
||||
event: AstrMessageEvent,
|
||||
) -> None | AsyncGenerator[None, None]:
|
||||
# 检查会话是否整体启用
|
||||
if not await SessionServiceManager.is_session_enabled(event.unified_msg_origin):
|
||||
if not SessionServiceManager.is_session_enabled(event.unified_msg_origin):
|
||||
logger.debug(f"会话 {event.unified_msg_origin} 已被关闭,已终止事件传播。")
|
||||
|
||||
# workaround for #2309
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
from collections.abc import AsyncGenerator, Callable
|
||||
from collections.abc import AsyncGenerator
|
||||
|
||||
from astrbot import logger
|
||||
from astrbot.core.message.components import At, AtAll, Reply
|
||||
from astrbot.core.message.message_event_result import MessageChain, MessageEventResult
|
||||
from astrbot.core.platform.astr_message_event import AstrMessageEvent
|
||||
from astrbot.core.platform.message_type import MessageType
|
||||
from astrbot.core.star.filter.command_group import CommandGroupFilter
|
||||
from astrbot.core.star.filter.permission import PermissionTypeFilter
|
||||
from astrbot.core.star.session_plugin_manager import SessionPluginManager
|
||||
@@ -14,23 +13,6 @@ from astrbot.core.star.star_handler import EventType, star_handlers_registry
|
||||
from ..context import PipelineContext
|
||||
from ..stage import Stage, register_stage
|
||||
|
||||
UNIQUE_SESSION_ID_BUILDERS: dict[str, Callable[[AstrMessageEvent], str | None]] = {
|
||||
"aiocqhttp": lambda e: f"{e.get_sender_id()}_{e.get_group_id()}",
|
||||
"slack": lambda e: f"{e.get_sender_id()}_{e.get_group_id()}",
|
||||
"dingtalk": lambda e: e.get_sender_id(),
|
||||
"qq_official": lambda e: e.get_sender_id(),
|
||||
"qq_official_webhook": lambda e: e.get_sender_id(),
|
||||
"lark": lambda e: f"{e.get_sender_id()}%{e.get_group_id()}",
|
||||
"misskey": lambda e: f"{e.get_session_id()}_{e.get_sender_id()}",
|
||||
"wechatpadpro": lambda e: f"{e.get_group_id()}#{e.get_sender_id()}",
|
||||
}
|
||||
|
||||
|
||||
def build_unique_session_id(event: AstrMessageEvent) -> str | None:
|
||||
platform = event.get_platform_name()
|
||||
builder = UNIQUE_SESSION_ID_BUILDERS.get(platform)
|
||||
return builder(event) if builder else None
|
||||
|
||||
|
||||
@register_stage
|
||||
class WakingCheckStage(Stage):
|
||||
@@ -71,27 +53,18 @@ class WakingCheckStage(Stage):
|
||||
self.disable_builtin_commands = self.ctx.astrbot_config.get(
|
||||
"disable_builtin_commands", False
|
||||
)
|
||||
platform_settings = self.ctx.astrbot_config.get("platform_settings", {})
|
||||
self.unique_session = platform_settings.get("unique_session", False)
|
||||
|
||||
async def process(
|
||||
self,
|
||||
event: AstrMessageEvent,
|
||||
) -> None | AsyncGenerator[None, None]:
|
||||
# apply unique session
|
||||
if self.unique_session and event.message_obj.type == MessageType.GROUP_MESSAGE:
|
||||
sid = build_unique_session_id(event)
|
||||
if sid:
|
||||
event.session_id = sid
|
||||
|
||||
# ignore bot self message
|
||||
if (
|
||||
self.ignore_bot_self_message
|
||||
and event.get_self_id() == event.get_sender_id()
|
||||
):
|
||||
# 忽略机器人自己发送的消息
|
||||
event.stop_event()
|
||||
return
|
||||
|
||||
# 设置 sender 身份
|
||||
event.message_str = event.message_str.strip()
|
||||
for admin_id in self.ctx.astrbot_config["admins_id"]:
|
||||
@@ -163,8 +136,7 @@ class WakingCheckStage(Stage):
|
||||
):
|
||||
if (
|
||||
self.disable_builtin_commands
|
||||
and handler.handler_module_path
|
||||
== "astrbot.builtin_stars.builtin_commands.main"
|
||||
and handler.handler_module_path == "packages.builtin_commands.main"
|
||||
):
|
||||
logger.debug("skipping builtin command")
|
||||
continue
|
||||
@@ -227,7 +199,7 @@ class WakingCheckStage(Stage):
|
||||
event._extras.pop("parsed_params", None)
|
||||
|
||||
# 根据会话配置过滤插件处理器
|
||||
activated_handlers = await SessionPluginManager.filter_handlers_by_session(
|
||||
activated_handlers = SessionPluginManager.filter_handlers_by_session(
|
||||
event,
|
||||
activated_handlers,
|
||||
)
|
||||
|
||||
@@ -112,6 +112,10 @@ class PlatformManager:
|
||||
from .sources.satori.satori_adapter import (
|
||||
SatoriPlatformAdapter, # noqa: F401
|
||||
)
|
||||
case "github_webhook":
|
||||
from .sources.github_webhook.github_webhook_adapter import (
|
||||
GitHubWebhookPlatformAdapter, # noqa: F401
|
||||
)
|
||||
except (ImportError, ModuleNotFoundError) as e:
|
||||
logger.error(
|
||||
f"加载平台适配器 {platform_config['type']} 失败,原因:{e}。请检查依赖库是否安装。提示:可以在 管理面板->平台日志->安装Pip库 中安装依赖库。",
|
||||
|
||||
@@ -41,6 +41,7 @@ class AiocqhttpAdapter(Platform):
|
||||
super().__init__(platform_config, event_queue)
|
||||
|
||||
self.settings = platform_settings
|
||||
self.unique_session = platform_settings["unique_session"]
|
||||
self.host = platform_config["ws_reverse_host"]
|
||||
self.port = platform_config["ws_reverse_port"]
|
||||
|
||||
@@ -135,11 +136,14 @@ class AiocqhttpAdapter(Platform):
|
||||
abm.group_id = str(event.group_id)
|
||||
else:
|
||||
abm.type = MessageType.FRIEND_MESSAGE
|
||||
abm.session_id = (
|
||||
str(event.group_id)
|
||||
if abm.type == MessageType.GROUP_MESSAGE
|
||||
else abm.sender.user_id
|
||||
)
|
||||
if self.unique_session and abm.type == MessageType.GROUP_MESSAGE:
|
||||
abm.session_id = str(abm.sender.user_id) + "_" + str(event.group_id)
|
||||
else:
|
||||
abm.session_id = (
|
||||
str(event.group_id)
|
||||
if abm.type == MessageType.GROUP_MESSAGE
|
||||
else abm.sender.user_id
|
||||
)
|
||||
abm.message_str = ""
|
||||
abm.message = []
|
||||
abm.timestamp = int(time.time())
|
||||
@@ -160,11 +164,16 @@ class AiocqhttpAdapter(Platform):
|
||||
abm.type = MessageType.GROUP_MESSAGE
|
||||
else:
|
||||
abm.type = MessageType.FRIEND_MESSAGE
|
||||
abm.session_id = (
|
||||
str(event.group_id)
|
||||
if abm.type == MessageType.GROUP_MESSAGE
|
||||
else abm.sender.user_id
|
||||
)
|
||||
if self.unique_session and abm.type == MessageType.GROUP_MESSAGE:
|
||||
abm.session_id = (
|
||||
str(abm.sender.user_id) + "_" + str(event.group_id)
|
||||
) # 也保留群组 id
|
||||
else:
|
||||
abm.session_id = (
|
||||
str(event.group_id)
|
||||
if abm.type == MessageType.GROUP_MESSAGE
|
||||
else abm.sender.user_id
|
||||
)
|
||||
abm.message_str = ""
|
||||
abm.message = []
|
||||
abm.raw_message = event
|
||||
@@ -201,11 +210,16 @@ class AiocqhttpAdapter(Platform):
|
||||
abm.group.group_name = event.get("group_name", "N/A")
|
||||
elif event["message_type"] == "private":
|
||||
abm.type = MessageType.FRIEND_MESSAGE
|
||||
abm.session_id = (
|
||||
str(event.group_id)
|
||||
if abm.type == MessageType.GROUP_MESSAGE
|
||||
else abm.sender.user_id
|
||||
)
|
||||
if self.unique_session and abm.type == MessageType.GROUP_MESSAGE:
|
||||
abm.session_id = (
|
||||
abm.sender.user_id + "_" + str(event.group_id)
|
||||
) # 也保留群组 id
|
||||
else:
|
||||
abm.session_id = (
|
||||
str(event.group_id)
|
||||
if abm.type == MessageType.GROUP_MESSAGE
|
||||
else abm.sender.user_id
|
||||
)
|
||||
|
||||
abm.message_id = str(event.message_id)
|
||||
abm.message = []
|
||||
@@ -371,25 +385,10 @@ class AiocqhttpAdapter(Platform):
|
||||
logger.error(f"获取 @ 用户信息失败: {e},此消息段将被忽略。")
|
||||
|
||||
message_str += "".join(at_parts)
|
||||
elif t == "markdown":
|
||||
text = m["data"].get("markdown") or m["data"].get("content", "")
|
||||
abm.message.append(Plain(text=text))
|
||||
message_str += text
|
||||
else:
|
||||
for m in m_group:
|
||||
try:
|
||||
if t not in ComponentTypes:
|
||||
logger.warning(
|
||||
f"不支持的消息段类型,已忽略: {t}, data={m['data']}"
|
||||
)
|
||||
continue
|
||||
a = ComponentTypes[t](**m["data"])
|
||||
abm.message.append(a)
|
||||
except Exception as e:
|
||||
logger.exception(
|
||||
f"消息段解析失败: type={t}, data={m['data']}. {e}"
|
||||
)
|
||||
continue
|
||||
a = ComponentTypes[t](**m["data"])
|
||||
abm.message.append(a)
|
||||
|
||||
abm.timestamp = int(time.time())
|
||||
abm.message_str = message_str
|
||||
|
||||
@@ -50,6 +50,8 @@ class DingtalkPlatformAdapter(Platform):
|
||||
) -> None:
|
||||
super().__init__(platform_config, event_queue)
|
||||
|
||||
self.unique_session = platform_settings["unique_session"]
|
||||
|
||||
self.client_id = platform_config["client_id"]
|
||||
self.client_secret = platform_config["client_secret"]
|
||||
|
||||
@@ -127,7 +129,10 @@ class DingtalkPlatformAdapter(Platform):
|
||||
if id := self._id_to_sid(user.dingtalk_id):
|
||||
abm.message.append(At(qq=id))
|
||||
abm.group_id = message.conversation_id
|
||||
abm.session_id = abm.group_id
|
||||
if self.unique_session:
|
||||
abm.session_id = abm.sender.user_id
|
||||
else:
|
||||
abm.session_id = abm.group_id
|
||||
else:
|
||||
abm.session_id = abm.sender.user_id
|
||||
|
||||
|
||||
@@ -25,20 +25,6 @@ class DingtalkMessageEvent(AstrMessageEvent):
|
||||
client: dingtalk_stream.ChatbotHandler,
|
||||
message: MessageChain,
|
||||
):
|
||||
icm = cast(dingtalk_stream.ChatbotMessage, self.message_obj.raw_message)
|
||||
ats = []
|
||||
# fixes: #4218
|
||||
# 钉钉 at 机器人需要使用 sender_staff_id 而不是 sender_id
|
||||
for i in message.chain:
|
||||
if isinstance(i, Comp.At):
|
||||
print(i.qq, icm.sender_id, icm.sender_staff_id)
|
||||
if str(i.qq) in str(icm.sender_id or ""):
|
||||
# 适配器会将开头的 $:LWCP_v1:$ 去掉,因此我们用 in 判断
|
||||
ats.append(f"@{icm.sender_staff_id}")
|
||||
else:
|
||||
ats.append(f"@{i.qq}")
|
||||
at_str = " ".join(ats)
|
||||
|
||||
for segment in message.chain:
|
||||
if isinstance(segment, Comp.Plain):
|
||||
segment.text = segment.text.strip()
|
||||
@@ -46,7 +32,7 @@ class DingtalkMessageEvent(AstrMessageEvent):
|
||||
None,
|
||||
client.reply_markdown,
|
||||
segment.text,
|
||||
f"{at_str} {segment.text}".strip(),
|
||||
segment.text,
|
||||
cast(dingtalk_stream.ChatbotMessage, self.message_obj.raw_message),
|
||||
)
|
||||
elif isinstance(segment, Comp.Image):
|
||||
|
||||
@@ -0,0 +1,315 @@
|
||||
import asyncio
|
||||
import hashlib
|
||||
import hmac
|
||||
from typing import Any, cast
|
||||
|
||||
from astrbot import logger
|
||||
from astrbot.api.event import MessageChain
|
||||
from astrbot.api.message_components import Plain
|
||||
from astrbot.api.platform import (
|
||||
AstrBotMessage,
|
||||
MessageMember,
|
||||
MessageType,
|
||||
Platform,
|
||||
PlatformMetadata,
|
||||
)
|
||||
from astrbot.core.platform.astr_message_event import MessageSesion
|
||||
from astrbot.core.platform.platform import PlatformStatus
|
||||
from astrbot.core.utils.webhook_utils import log_webhook_info
|
||||
|
||||
from ...register import register_platform_adapter
|
||||
from .github_webhook_event import GitHubWebhookMessageEvent
|
||||
|
||||
|
||||
@register_platform_adapter(
|
||||
"github_webhook",
|
||||
"GitHub Webhook 适配器",
|
||||
support_streaming_message=False,
|
||||
)
|
||||
class GitHubWebhookPlatformAdapter(Platform):
|
||||
"""GitHub Webhook 平台适配器
|
||||
|
||||
支持的事件:
|
||||
- issues (created)
|
||||
- issue_comment (created)
|
||||
- pull_request (opened)
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
platform_config: dict,
|
||||
platform_settings: dict,
|
||||
event_queue: asyncio.Queue,
|
||||
) -> None:
|
||||
super().__init__(platform_config, event_queue)
|
||||
|
||||
self.unified_webhook_mode = platform_config.get("unified_webhook_mode", True)
|
||||
self.webhook_secret = platform_config.get("webhook_secret", "")
|
||||
self.shutdown_event = asyncio.Event()
|
||||
|
||||
async def send_by_session(
|
||||
self,
|
||||
session: MessageSesion,
|
||||
message_chain: MessageChain,
|
||||
):
|
||||
"""GitHub Webhook 是单向接收,不支持主动发送消息"""
|
||||
logger.warning("GitHub Webhook 适配器不支持 send_by_session")
|
||||
|
||||
def meta(self) -> PlatformMetadata:
|
||||
return PlatformMetadata(
|
||||
name="github_webhook",
|
||||
description="GitHub Webhook 适配器",
|
||||
id=cast(str, self.config.get("id")),
|
||||
)
|
||||
|
||||
async def run(self):
|
||||
"""运行适配器"""
|
||||
self.status = PlatformStatus.RUNNING
|
||||
|
||||
# 如果启用统一 webhook 模式
|
||||
webhook_uuid = self.config.get("webhook_uuid")
|
||||
if self.unified_webhook_mode and webhook_uuid:
|
||||
log_webhook_info(f"{self.meta().id}(GitHub Webhook)", webhook_uuid)
|
||||
# 保持运行状态,等待 shutdown
|
||||
await self.shutdown_event.wait()
|
||||
else:
|
||||
logger.warning("GitHub Webhook 适配器需要启用统一 webhook 模式")
|
||||
await self.shutdown_event.wait()
|
||||
|
||||
async def webhook_callback(self, request: Any) -> Any:
|
||||
"""统一 Webhook 回调入口
|
||||
|
||||
处理 GitHub webhook 事件
|
||||
|
||||
Args:
|
||||
request: Quart 请求对象
|
||||
|
||||
Returns:
|
||||
响应数据
|
||||
"""
|
||||
try:
|
||||
# 获取事件类型
|
||||
event_type = request.headers.get("X-GitHub-Event", "")
|
||||
|
||||
# 获取请求数据
|
||||
payload = await request.json
|
||||
|
||||
# 验证 webhook 签名(如果配置了 secret)
|
||||
if self.webhook_secret:
|
||||
if not await self._verify_signature(request, payload):
|
||||
logger.warning("GitHub webhook 签名验证失败")
|
||||
return {"error": "Invalid signature"}, 401
|
||||
|
||||
logger.debug(f"收到 GitHub Webhook 事件: {event_type}")
|
||||
|
||||
# 处理不同类型的事件
|
||||
if event_type == "issues":
|
||||
await self._handle_issue_event(payload)
|
||||
elif event_type == "issue_comment":
|
||||
await self._handle_issue_comment_event(payload)
|
||||
elif event_type == "pull_request":
|
||||
await self._handle_pull_request_event(payload)
|
||||
elif event_type == "ping":
|
||||
# GitHub webhook 验证事件
|
||||
return {"message": "pong"}
|
||||
else:
|
||||
logger.debug(f"忽略不支持的 GitHub 事件类型: {event_type}")
|
||||
|
||||
return {"status": "ok"}
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"处理 GitHub webhook 回调时发生错误: {e}", exc_info=True)
|
||||
return {"error": str(e)}, 500
|
||||
|
||||
async def _verify_signature(self, request: Any, payload: dict) -> bool:
|
||||
"""验证 GitHub webhook 签名
|
||||
|
||||
Args:
|
||||
request: Quart 请求对象
|
||||
payload: 请求负载数据
|
||||
|
||||
Returns:
|
||||
签名是否有效
|
||||
"""
|
||||
signature_header = request.headers.get("X-Hub-Signature-256", "")
|
||||
if not signature_header:
|
||||
# 如果没有签名头,检查是否有旧版本的签名
|
||||
signature_header = request.headers.get("X-Hub-Signature", "")
|
||||
if not signature_header:
|
||||
return False
|
||||
|
||||
# 获取原始请求体
|
||||
body = await request.get_data()
|
||||
|
||||
# 计算 HMAC
|
||||
if signature_header.startswith("sha256="):
|
||||
expected_signature = hmac.new(
|
||||
self.webhook_secret.encode("utf-8"),
|
||||
body,
|
||||
hashlib.sha256,
|
||||
).hexdigest()
|
||||
received_signature = signature_header.replace("sha256=", "")
|
||||
elif signature_header.startswith("sha1="):
|
||||
expected_signature = hmac.new(
|
||||
self.webhook_secret.encode("utf-8"),
|
||||
body,
|
||||
hashlib.sha1,
|
||||
).hexdigest()
|
||||
received_signature = signature_header.replace("sha1=", "")
|
||||
else:
|
||||
return False
|
||||
|
||||
# 使用 hmac.compare_digest 防止时序攻击
|
||||
return hmac.compare_digest(expected_signature, received_signature)
|
||||
|
||||
async def _handle_issue_event(self, payload: dict):
|
||||
"""处理 issue 事件"""
|
||||
action = payload.get("action", "")
|
||||
|
||||
# 只处理创建事件
|
||||
if action != "created" and action != "opened":
|
||||
return
|
||||
|
||||
issue = payload.get("issue", {})
|
||||
repo = payload.get("repository", {})
|
||||
sender = payload.get("sender", {})
|
||||
|
||||
# 构造消息文本
|
||||
message_text = (
|
||||
f"📝 新 Issue 创建\n"
|
||||
f"仓库: {repo.get('full_name', 'unknown')}\n"
|
||||
f"标题: {issue.get('title', 'No title')}\n"
|
||||
f"作者: {sender.get('login', 'unknown')}\n"
|
||||
f"链接: {issue.get('html_url', '')}\n"
|
||||
f"内容:\n{issue.get('body', 'No description')[:200]}"
|
||||
)
|
||||
|
||||
# 创建 AstrBotMessage
|
||||
abm = self._create_message(
|
||||
message_text,
|
||||
sender.get("login", "unknown"),
|
||||
sender.get("login", "unknown"),
|
||||
repo.get("full_name", "unknown"),
|
||||
)
|
||||
|
||||
# 提交事件
|
||||
self.commit_event(
|
||||
GitHubWebhookMessageEvent(
|
||||
message_text,
|
||||
abm,
|
||||
self.meta(),
|
||||
repo.get("full_name", "unknown"),
|
||||
"issues",
|
||||
payload,
|
||||
)
|
||||
)
|
||||
|
||||
async def _handle_issue_comment_event(self, payload: dict):
|
||||
"""处理 issue 评论事件"""
|
||||
action = payload.get("action", "")
|
||||
|
||||
# 只处理创建事件
|
||||
if action != "created":
|
||||
return
|
||||
|
||||
issue = payload.get("issue", {})
|
||||
comment = payload.get("comment", {})
|
||||
repo = payload.get("repository", {})
|
||||
sender = payload.get("sender", {})
|
||||
|
||||
# 构造消息文本
|
||||
message_text = (
|
||||
f"💬 新 Issue 评论\n"
|
||||
f"仓库: {repo.get('full_name', 'unknown')}\n"
|
||||
f"Issue: {issue.get('title', 'No title')}\n"
|
||||
f"评论者: {sender.get('login', 'unknown')}\n"
|
||||
f"链接: {comment.get('html_url', '')}\n"
|
||||
f"内容:\n{comment.get('body', 'No comment')[:200]}"
|
||||
)
|
||||
|
||||
# 创建 AstrBotMessage
|
||||
abm = self._create_message(
|
||||
message_text,
|
||||
sender.get("login", "unknown"),
|
||||
sender.get("login", "unknown"),
|
||||
repo.get("full_name", "unknown"),
|
||||
)
|
||||
|
||||
# 提交事件
|
||||
self.commit_event(
|
||||
GitHubWebhookMessageEvent(
|
||||
message_text,
|
||||
abm,
|
||||
self.meta(),
|
||||
repo.get("full_name", "unknown"),
|
||||
"issue_comment",
|
||||
payload,
|
||||
)
|
||||
)
|
||||
|
||||
async def _handle_pull_request_event(self, payload: dict):
|
||||
"""处理 pull request 事件"""
|
||||
action = payload.get("action", "")
|
||||
|
||||
# 只处理打开事件
|
||||
if action != "opened":
|
||||
return
|
||||
|
||||
pr = payload.get("pull_request", {})
|
||||
repo = payload.get("repository", {})
|
||||
sender = payload.get("sender", {})
|
||||
|
||||
# 构造消息文本
|
||||
message_text = (
|
||||
f"🔀 新 Pull Request\n"
|
||||
f"仓库: {repo.get('full_name', 'unknown')}\n"
|
||||
f"标题: {pr.get('title', 'No title')}\n"
|
||||
f"作者: {sender.get('login', 'unknown')}\n"
|
||||
f"链接: {pr.get('html_url', '')}\n"
|
||||
f"内容:\n{pr.get('body', 'No description')[:200]}"
|
||||
)
|
||||
|
||||
# 创建 AstrBotMessage
|
||||
abm = self._create_message(
|
||||
message_text,
|
||||
sender.get("login", "unknown"),
|
||||
sender.get("login", "unknown"),
|
||||
repo.get("full_name", "unknown"),
|
||||
)
|
||||
|
||||
# 提交事件
|
||||
self.commit_event(
|
||||
GitHubWebhookMessageEvent(
|
||||
message_text,
|
||||
abm,
|
||||
self.meta(),
|
||||
repo.get("full_name", "unknown"),
|
||||
"pull_request",
|
||||
payload,
|
||||
)
|
||||
)
|
||||
|
||||
def _create_message(
|
||||
self,
|
||||
message_text: str,
|
||||
user_id: str,
|
||||
nickname: str,
|
||||
session_id: str,
|
||||
) -> AstrBotMessage:
|
||||
"""创建 AstrBotMessage 对象"""
|
||||
abm = AstrBotMessage()
|
||||
abm.type = MessageType.GROUP_MESSAGE
|
||||
abm.self_id = self.client_self_id
|
||||
abm.session_id = session_id
|
||||
abm.message_id = ""
|
||||
abm.sender = MessageMember(user_id=user_id, nickname=nickname)
|
||||
abm.message = [Plain(message_text)]
|
||||
abm.message_str = message_text
|
||||
abm.raw_message = message_text
|
||||
|
||||
return abm
|
||||
|
||||
async def terminate(self):
|
||||
"""终止适配器运行"""
|
||||
self.shutdown_event.set()
|
||||
logger.info("GitHub Webhook 适配器已经被优雅地关闭")
|
||||
@@ -0,0 +1,22 @@
|
||||
from astrbot.api.platform import AstrBotMessage, PlatformMetadata
|
||||
|
||||
from ...astr_message_event import AstrMessageEvent
|
||||
|
||||
|
||||
class GitHubWebhookMessageEvent(AstrMessageEvent):
|
||||
"""GitHub Webhook 消息事件"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message_str: str,
|
||||
message_obj: AstrBotMessage,
|
||||
platform_meta: PlatformMetadata,
|
||||
session_id: str,
|
||||
event_type: str,
|
||||
event_data: dict,
|
||||
):
|
||||
super().__init__(message_str, message_obj, platform_meta, session_id)
|
||||
self.event_type = event_type
|
||||
"""GitHub 事件类型: issues, issue_comment, pull_request"""
|
||||
self.event_data = event_data
|
||||
"""原始事件数据"""
|
||||
@@ -44,6 +44,8 @@ class LarkPlatformAdapter(Platform):
|
||||
) -> None:
|
||||
super().__init__(platform_config, event_queue)
|
||||
|
||||
self.unique_session = platform_settings["unique_session"]
|
||||
|
||||
self.appid = platform_config["app_id"]
|
||||
self.appsecret = platform_config["app_secret"]
|
||||
self.domain = platform_config.get("domain", lark.FEISHU_DOMAIN)
|
||||
@@ -79,12 +81,7 @@ class LarkPlatformAdapter(Platform):
|
||||
)
|
||||
|
||||
self.lark_api = (
|
||||
lark.Client.builder()
|
||||
.app_id(self.appid)
|
||||
.app_secret(self.appsecret)
|
||||
.log_level(lark.LogLevel.ERROR)
|
||||
.domain(self.domain)
|
||||
.build()
|
||||
lark.Client.builder().app_id(self.appid).app_secret(self.appsecret).build()
|
||||
)
|
||||
|
||||
self.webhook_server = None
|
||||
@@ -315,8 +312,14 @@ class LarkPlatformAdapter(Platform):
|
||||
user_id=event.event.sender.sender_id.open_id,
|
||||
nickname=event.event.sender.sender_id.open_id[:8],
|
||||
)
|
||||
if abm.type == MessageType.GROUP_MESSAGE:
|
||||
abm.session_id = abm.group_id
|
||||
# 独立会话
|
||||
if not self.unique_session:
|
||||
if abm.type == MessageType.GROUP_MESSAGE:
|
||||
abm.session_id = abm.group_id
|
||||
else:
|
||||
abm.session_id = abm.sender.user_id
|
||||
elif abm.type == MessageType.GROUP_MESSAGE:
|
||||
abm.session_id = f"{abm.sender.user_id}%{abm.group_id}" # 也保留群组id
|
||||
else:
|
||||
abm.session_id = abm.sender.user_id
|
||||
|
||||
|
||||
@@ -91,6 +91,8 @@ class MisskeyPlatformAdapter(Platform):
|
||||
except Exception:
|
||||
self.max_download_bytes = None
|
||||
|
||||
self.unique_session = platform_settings["unique_session"]
|
||||
|
||||
self.api: MisskeyAPI | None = None
|
||||
self._running = False
|
||||
self.client_self_id = ""
|
||||
@@ -639,6 +641,7 @@ class MisskeyPlatformAdapter(Platform):
|
||||
sender_info,
|
||||
self.client_self_id,
|
||||
is_chat=False,
|
||||
unique_session=self.unique_session,
|
||||
)
|
||||
cache_user_info(
|
||||
self._user_cache,
|
||||
@@ -687,6 +690,7 @@ class MisskeyPlatformAdapter(Platform):
|
||||
sender_info,
|
||||
self.client_self_id,
|
||||
is_chat=True,
|
||||
unique_session=self.unique_session,
|
||||
)
|
||||
cache_user_info(
|
||||
self._user_cache,
|
||||
@@ -716,6 +720,7 @@ class MisskeyPlatformAdapter(Platform):
|
||||
self.client_self_id,
|
||||
is_chat=False,
|
||||
room_id=room_id,
|
||||
unique_session=self.unique_session,
|
||||
)
|
||||
|
||||
cache_user_info(
|
||||
|
||||
@@ -338,6 +338,7 @@ def create_base_message(
|
||||
client_self_id: str,
|
||||
is_chat: bool = False,
|
||||
room_id: str | None = None,
|
||||
unique_session: bool = False,
|
||||
) -> AstrBotMessage:
|
||||
"""创建基础消息对象"""
|
||||
message = AstrBotMessage()
|
||||
@@ -352,6 +353,8 @@ def create_base_message(
|
||||
if room_id:
|
||||
session_prefix = "room"
|
||||
session_id = f"{session_prefix}%{room_id}"
|
||||
if unique_session:
|
||||
session_id += f"_{sender_info['sender_id']}"
|
||||
message.type = MessageType.GROUP_MESSAGE
|
||||
message.group_id = room_id
|
||||
elif is_chat:
|
||||
|
||||
@@ -44,8 +44,11 @@ class botClient(Client):
|
||||
message,
|
||||
MessageType.GROUP_MESSAGE,
|
||||
)
|
||||
abm.group_id = cast(str, message.group_openid)
|
||||
abm.session_id = abm.group_id
|
||||
abm.session_id = (
|
||||
abm.sender.user_id
|
||||
if self.platform.unique_session
|
||||
else cast(str, message.group_openid)
|
||||
)
|
||||
self._commit(abm)
|
||||
|
||||
# 收到频道消息
|
||||
@@ -54,8 +57,9 @@ class botClient(Client):
|
||||
message,
|
||||
MessageType.GROUP_MESSAGE,
|
||||
)
|
||||
abm.group_id = message.channel_id
|
||||
abm.session_id = abm.group_id
|
||||
abm.session_id = (
|
||||
abm.sender.user_id if self.platform.unique_session else message.channel_id
|
||||
)
|
||||
self._commit(abm)
|
||||
|
||||
# 收到私聊消息
|
||||
@@ -100,6 +104,7 @@ class QQOfficialPlatformAdapter(Platform):
|
||||
|
||||
self.appid = platform_config["appid"]
|
||||
self.secret = platform_config["secret"]
|
||||
self.unique_session: bool = platform_settings["unique_session"]
|
||||
qq_group = platform_config["enable_group_c2c"]
|
||||
guild_dm = platform_config["enable_guild_direct_message"]
|
||||
|
||||
|
||||
@@ -35,8 +35,11 @@ class botClient(Client):
|
||||
message,
|
||||
MessageType.GROUP_MESSAGE,
|
||||
)
|
||||
abm.group_id = cast(str, message.group_openid)
|
||||
abm.session_id = abm.group_id
|
||||
abm.session_id = (
|
||||
abm.sender.user_id
|
||||
if self.platform.unique_session
|
||||
else cast(str, message.group_openid)
|
||||
)
|
||||
self._commit(abm)
|
||||
|
||||
# 收到频道消息
|
||||
@@ -45,8 +48,9 @@ class botClient(Client):
|
||||
message,
|
||||
MessageType.GROUP_MESSAGE,
|
||||
)
|
||||
abm.group_id = message.channel_id
|
||||
abm.session_id = abm.group_id
|
||||
abm.session_id = (
|
||||
abm.sender.user_id if self.platform.unique_session else message.channel_id
|
||||
)
|
||||
self._commit(abm)
|
||||
|
||||
# 收到私聊消息
|
||||
@@ -91,6 +95,7 @@ class QQOfficialWebhookPlatformAdapter(Platform):
|
||||
|
||||
self.appid = platform_config["appid"]
|
||||
self.secret = platform_config["secret"]
|
||||
self.unique_session = platform_settings["unique_session"]
|
||||
self.unified_webhook_mode = platform_config.get("unified_webhook_mode", False)
|
||||
|
||||
intents = botpy.Intents(
|
||||
|
||||
@@ -142,12 +142,7 @@ class SatoriPlatformAdapter(Platform):
|
||||
raise ValueError(f"WebSocket URL必须以ws://或wss://开头: {self.endpoint}")
|
||||
|
||||
try:
|
||||
websocket = await connect(
|
||||
self.endpoint,
|
||||
additional_headers={},
|
||||
max_size=10 * 1024 * 1024, # 10MB
|
||||
)
|
||||
|
||||
websocket = await connect(self.endpoint, additional_headers={})
|
||||
self.ws = websocket
|
||||
|
||||
await asyncio.sleep(0.1)
|
||||
|
||||
@@ -41,6 +41,7 @@ class SlackAdapter(Platform):
|
||||
) -> None:
|
||||
super().__init__(platform_config, event_queue)
|
||||
self.settings = platform_settings
|
||||
self.unique_session = platform_settings.get("unique_session", False)
|
||||
|
||||
self.bot_token = platform_config.get("bot_token")
|
||||
self.app_token = platform_config.get("app_token")
|
||||
@@ -146,10 +147,12 @@ class SlackAdapter(Platform):
|
||||
abm.group_id = channel_id
|
||||
|
||||
# 设置会话ID
|
||||
if abm.type == MessageType.GROUP_MESSAGE:
|
||||
abm.session_id = abm.group_id
|
||||
if self.unique_session and abm.type == MessageType.GROUP_MESSAGE:
|
||||
abm.session_id = f"{user_id}_{channel_id}"
|
||||
else:
|
||||
abm.session_id = user_id
|
||||
abm.session_id = (
|
||||
channel_id if abm.type == MessageType.GROUP_MESSAGE else user_id
|
||||
)
|
||||
|
||||
abm.message_id = event.get("client_msg_id", uuid.uuid4().hex)
|
||||
abm.timestamp = int(float(event.get("ts", time.time())))
|
||||
|
||||
@@ -200,15 +200,6 @@ class TelegramPlatformEvent(AstrMessageEvent):
|
||||
if isinstance(chain, MessageChain):
|
||||
if chain.type == "break":
|
||||
# 分割符
|
||||
if message_id:
|
||||
try:
|
||||
await self.client.edit_message_text(
|
||||
text=delta,
|
||||
chat_id=payload["chat_id"],
|
||||
message_id=message_id,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"编辑消息失败(streaming-break): {e!s}")
|
||||
message_id = None # 重置消息 ID
|
||||
delta = "" # 重置 delta
|
||||
continue
|
||||
|
||||
@@ -79,6 +79,7 @@ class WebChatAdapter(Platform):
|
||||
super().__init__(platform_config, event_queue)
|
||||
|
||||
self.settings = platform_settings
|
||||
self.unique_session = platform_settings["unique_session"]
|
||||
self.imgs_dir = os.path.join(get_astrbot_data_path(), "webchat", "imgs")
|
||||
os.makedirs(self.imgs_dir, exist_ok=True)
|
||||
|
||||
|
||||
@@ -1,12 +1,11 @@
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import uuid
|
||||
|
||||
from astrbot.api import logger
|
||||
from astrbot.api.event import AstrMessageEvent, MessageChain
|
||||
from astrbot.api.message_components import File, Image, Json, Plain, Record
|
||||
from astrbot.api.message_components import File, Image, Plain, Record
|
||||
from astrbot.core.utils.astrbot_path import get_astrbot_data_path
|
||||
|
||||
from .webchat_queue_mgr import webchat_queue_mgr
|
||||
@@ -42,20 +41,12 @@ class WebChatMessageEvent(AstrMessageEvent):
|
||||
await web_chat_back_queue.put(
|
||||
{
|
||||
"type": "plain",
|
||||
"cid": cid,
|
||||
"data": data,
|
||||
"streaming": streaming,
|
||||
"chain_type": message.type,
|
||||
},
|
||||
)
|
||||
elif isinstance(comp, Json):
|
||||
await web_chat_back_queue.put(
|
||||
{
|
||||
"type": "plain",
|
||||
"data": json.dumps(comp.data, ensure_ascii=False),
|
||||
"streaming": streaming,
|
||||
"chain_type": message.type,
|
||||
},
|
||||
)
|
||||
elif isinstance(comp, Image):
|
||||
# save image to local
|
||||
filename = f"{str(uuid.uuid4())}.jpg"
|
||||
@@ -67,6 +58,7 @@ class WebChatMessageEvent(AstrMessageEvent):
|
||||
await web_chat_back_queue.put(
|
||||
{
|
||||
"type": "image",
|
||||
"cid": cid,
|
||||
"data": data,
|
||||
"streaming": streaming,
|
||||
},
|
||||
@@ -82,6 +74,7 @@ class WebChatMessageEvent(AstrMessageEvent):
|
||||
await web_chat_back_queue.put(
|
||||
{
|
||||
"type": "record",
|
||||
"cid": cid,
|
||||
"data": data,
|
||||
"streaming": streaming,
|
||||
},
|
||||
@@ -98,6 +91,7 @@ class WebChatMessageEvent(AstrMessageEvent):
|
||||
await web_chat_back_queue.put(
|
||||
{
|
||||
"type": "file",
|
||||
"cid": cid,
|
||||
"data": data,
|
||||
"streaming": streaming,
|
||||
},
|
||||
@@ -117,17 +111,18 @@ class WebChatMessageEvent(AstrMessageEvent):
|
||||
cid = self.session_id.split("!")[-1]
|
||||
web_chat_back_queue = webchat_queue_mgr.get_or_create_back_queue(cid)
|
||||
async for chain in generator:
|
||||
# if chain.type == "break" and final_data:
|
||||
# # 分割符
|
||||
# await web_chat_back_queue.put(
|
||||
# {
|
||||
# "type": "break", # break means a segment end
|
||||
# "data": final_data,
|
||||
# "streaming": True,
|
||||
# },
|
||||
# )
|
||||
# final_data = ""
|
||||
# continue
|
||||
if chain.type == "break" and final_data:
|
||||
# 分割符
|
||||
await web_chat_back_queue.put(
|
||||
{
|
||||
"type": "break", # break means a segment end
|
||||
"data": final_data,
|
||||
"streaming": True,
|
||||
"cid": cid,
|
||||
},
|
||||
)
|
||||
final_data = ""
|
||||
continue
|
||||
|
||||
r = await WebChatMessageEvent._send(
|
||||
chain,
|
||||
@@ -147,6 +142,7 @@ class WebChatMessageEvent(AstrMessageEvent):
|
||||
"data": final_data,
|
||||
"reasoning": reasoning_content,
|
||||
"streaming": True,
|
||||
"cid": cid,
|
||||
},
|
||||
)
|
||||
await super().send_streaming(generator, use_fallback)
|
||||
|
||||
@@ -47,6 +47,7 @@ class WeChatPadProAdapter(Platform):
|
||||
self._shutdown_event = None
|
||||
self.wxnewpass = None
|
||||
self.settings = platform_settings
|
||||
self.unique_session = platform_settings.get("unique_session", False)
|
||||
|
||||
self.metadata = PlatformMetadata(
|
||||
name="wechatpadpro",
|
||||
@@ -508,10 +509,11 @@ class WeChatPadProAdapter(Platform):
|
||||
if accurate_nickname:
|
||||
abm.sender.nickname = accurate_nickname
|
||||
|
||||
if abm.type == MessageType.GROUP_MESSAGE:
|
||||
abm.session_id = abm.group_id
|
||||
# 对于群聊,session_id 可以是群聊 ID 或发送者 ID + 群聊 ID (如果 unique_session 为 True)
|
||||
if self.unique_session:
|
||||
abm.session_id = f"{from_user_name}#{abm.sender.user_id}"
|
||||
else:
|
||||
abm.session_id = abm.sender.user_id
|
||||
abm.session_id = from_user_name
|
||||
|
||||
msg_source = raw_message.get("msg_source", "")
|
||||
if self.wxid in msg_source:
|
||||
|
||||
@@ -191,7 +191,7 @@ class WeixinOfficialAccountPlatformAdapter(Platform):
|
||||
if self.active_send_mode:
|
||||
await self.convert_message(msg, None)
|
||||
else:
|
||||
if str(msg.id) in self.wexin_event_workers:
|
||||
if msg.id in self.wexin_event_workers:
|
||||
future = self.wexin_event_workers[str(cast(str | int, msg.id))]
|
||||
logger.debug(f"duplicate message id checked: {msg.id}")
|
||||
else:
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import enum
|
||||
import json
|
||||
@@ -14,7 +12,6 @@ import astrbot.core.message.components as Comp
|
||||
from astrbot import logger
|
||||
from astrbot.core.agent.message import (
|
||||
AssistantMessageSegment,
|
||||
ContentPart,
|
||||
ToolCall,
|
||||
ToolCallMessageSegment,
|
||||
)
|
||||
@@ -93,8 +90,6 @@ class ProviderRequest:
|
||||
"""会话 ID"""
|
||||
image_urls: list[str] = field(default_factory=list)
|
||||
"""图片 URL 列表"""
|
||||
extra_user_content_parts: list[ContentPart] = field(default_factory=list)
|
||||
"""额外的用户消息内容部分列表,用于在用户消息后添加额外的内容块(如系统提醒、指令等)。支持 dict 或 ContentPart 对象"""
|
||||
func_tool: ToolSet | None = None
|
||||
"""可用的函数工具"""
|
||||
contexts: list[dict] = field(default_factory=list)
|
||||
@@ -169,23 +164,13 @@ class ProviderRequest:
|
||||
|
||||
async def assemble_context(self) -> dict:
|
||||
"""将请求(prompt 和 image_urls)包装成 OpenAI 的消息格式。"""
|
||||
# 构建内容块列表
|
||||
content_blocks = []
|
||||
|
||||
# 1. 用户原始发言(OpenAI 建议:用户发言在前)
|
||||
if self.prompt and self.prompt.strip():
|
||||
content_blocks.append({"type": "text", "text": self.prompt})
|
||||
elif self.image_urls:
|
||||
# 如果没有文本但有图片,添加占位文本
|
||||
content_blocks.append({"type": "text", "text": "[图片]"})
|
||||
|
||||
# 2. 额外的内容块(系统提醒、指令等)
|
||||
if self.extra_user_content_parts:
|
||||
for part in self.extra_user_content_parts:
|
||||
content_blocks.append(part.model_dump())
|
||||
|
||||
# 3. 图片内容
|
||||
if self.image_urls:
|
||||
user_content = {
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": self.prompt if self.prompt else "[图片]"},
|
||||
],
|
||||
}
|
||||
for image_url in self.image_urls:
|
||||
if image_url.startswith("http"):
|
||||
image_path = await download_image_by_url(image_url)
|
||||
@@ -198,21 +183,11 @@ class ProviderRequest:
|
||||
if not image_data:
|
||||
logger.warning(f"图片 {image_url} 得到的结果为空,将忽略。")
|
||||
continue
|
||||
content_blocks.append(
|
||||
user_content["content"].append(
|
||||
{"type": "image_url", "image_url": {"url": image_data}},
|
||||
)
|
||||
|
||||
# 只有当只有一个来自 prompt 的文本块且没有额外内容块时,才降级为简单格式以保持向后兼容
|
||||
if (
|
||||
len(content_blocks) == 1
|
||||
and content_blocks[0]["type"] == "text"
|
||||
and not self.extra_user_content_parts
|
||||
and not self.image_urls
|
||||
):
|
||||
return {"role": "user", "content": content_blocks[0]["text"]}
|
||||
|
||||
# 否则返回多模态格式
|
||||
return {"role": "user", "content": content_blocks}
|
||||
return user_content
|
||||
return {"role": "user", "content": self.prompt}
|
||||
|
||||
async def _encode_image_bs64(self, image_url: str) -> str:
|
||||
"""将图片转换为 base64"""
|
||||
@@ -224,38 +199,6 @@ class ProviderRequest:
|
||||
return ""
|
||||
|
||||
|
||||
@dataclass
|
||||
class TokenUsage:
|
||||
input_other: int = 0
|
||||
"""The number of input tokens, excluding cached tokens."""
|
||||
input_cached: int = 0
|
||||
"""The number of input cached tokens."""
|
||||
output: int = 0
|
||||
"""The number of output tokens."""
|
||||
|
||||
@property
|
||||
def total(self) -> int:
|
||||
return self.input_other + self.input_cached + self.output
|
||||
|
||||
@property
|
||||
def input(self) -> int:
|
||||
return self.input_other + self.input_cached
|
||||
|
||||
def __add__(self, other: TokenUsage) -> TokenUsage:
|
||||
return TokenUsage(
|
||||
input_other=self.input_other + other.input_other,
|
||||
input_cached=self.input_cached + other.input_cached,
|
||||
output=self.output + other.output,
|
||||
)
|
||||
|
||||
def __sub__(self, other: TokenUsage) -> TokenUsage:
|
||||
return TokenUsage(
|
||||
input_other=self.input_other - other.input_other,
|
||||
input_cached=self.input_cached - other.input_cached,
|
||||
output=self.output - other.output,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class LLMResponse:
|
||||
role: str
|
||||
@@ -272,8 +215,6 @@ class LLMResponse:
|
||||
"""Tool call extra content. tool_call_id -> extra_content dict"""
|
||||
reasoning_content: str = ""
|
||||
"""The reasoning content extracted from the LLM, if any."""
|
||||
reasoning_signature: str | None = None
|
||||
"""The signature of the reasoning content, if any."""
|
||||
|
||||
raw_completion: (
|
||||
ChatCompletion | GenerateContentResponse | AnthropicMessage | None
|
||||
@@ -286,29 +227,20 @@ class LLMResponse:
|
||||
is_chunk: bool = False
|
||||
"""Indicates if the response is a chunked response."""
|
||||
|
||||
id: str | None = None
|
||||
"""The ID of the response. For chunked responses, it's the ID of the chunk; for non-chunked responses, it's the ID of the response."""
|
||||
usage: TokenUsage | None = None
|
||||
"""The usage of the response. For chunked responses, it's the usage of the chunk; for non-chunked responses, it's the usage of the response."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
role: str,
|
||||
completion_text: str | None = None,
|
||||
completion_text: str = "",
|
||||
result_chain: MessageChain | None = None,
|
||||
tools_call_args: list[dict[str, Any]] | None = None,
|
||||
tools_call_name: list[str] | None = None,
|
||||
tools_call_ids: list[str] | None = None,
|
||||
tools_call_extra_content: dict[str, dict[str, Any]] | None = None,
|
||||
reasoning_content: str | None = None,
|
||||
reasoning_signature: str | None = None,
|
||||
raw_completion: ChatCompletion
|
||||
| GenerateContentResponse
|
||||
| AnthropicMessage
|
||||
| None = None,
|
||||
is_chunk: bool = False,
|
||||
id: str | None = None,
|
||||
usage: TokenUsage | None = None,
|
||||
):
|
||||
"""初始化 LLMResponse
|
||||
|
||||
@@ -321,8 +253,6 @@ class LLMResponse:
|
||||
raw_completion (ChatCompletion, optional): 原始响应, OpenAI 格式. Defaults to None.
|
||||
|
||||
"""
|
||||
if reasoning_content is None:
|
||||
reasoning_content = ""
|
||||
if tools_call_args is None:
|
||||
tools_call_args = []
|
||||
if tools_call_name is None:
|
||||
@@ -339,8 +269,6 @@ class LLMResponse:
|
||||
self.tools_call_name = tools_call_name
|
||||
self.tools_call_ids = tools_call_ids
|
||||
self.tools_call_extra_content = tools_call_extra_content
|
||||
self.reasoning_content = reasoning_content
|
||||
self.reasoning_signature = reasoning_signature
|
||||
self.raw_completion = raw_completion
|
||||
self.is_chunk = is_chunk
|
||||
|
||||
|
||||
+105
-230
@@ -1,5 +1,4 @@
|
||||
import asyncio
|
||||
import copy
|
||||
import traceback
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
@@ -33,12 +32,10 @@ class ProviderManager:
|
||||
persona_mgr: PersonaManager,
|
||||
):
|
||||
self.reload_lock = asyncio.Lock()
|
||||
self.resource_lock = asyncio.Lock()
|
||||
self.persona_mgr = persona_mgr
|
||||
self.acm = acm
|
||||
config = acm.confs["default"]
|
||||
self.providers_config: list = config["provider"]
|
||||
self.provider_sources_config: list = config.get("provider_sources", [])
|
||||
self.provider_settings: dict = config["provider_settings"]
|
||||
self.provider_stt_settings: dict = config.get("provider_stt_settings", {})
|
||||
self.provider_tts_settings: dict = config.get("provider_tts_settings", {})
|
||||
@@ -119,34 +116,19 @@ class ProviderManager:
|
||||
TTSProvider,
|
||||
):
|
||||
self.curr_tts_provider_inst = prov
|
||||
await sp.put_async(
|
||||
key="curr_provider_tts",
|
||||
value=provider_id,
|
||||
scope="global",
|
||||
scope_id="global",
|
||||
)
|
||||
sp.put("curr_provider_tts", provider_id, scope="global", scope_id="global")
|
||||
elif provider_type == ProviderType.SPEECH_TO_TEXT and isinstance(
|
||||
prov,
|
||||
STTProvider,
|
||||
):
|
||||
self.curr_stt_provider_inst = prov
|
||||
await sp.put_async(
|
||||
key="curr_provider_stt",
|
||||
value=provider_id,
|
||||
scope="global",
|
||||
scope_id="global",
|
||||
)
|
||||
sp.put("curr_provider_stt", provider_id, scope="global", scope_id="global")
|
||||
elif provider_type == ProviderType.CHAT_COMPLETION and isinstance(
|
||||
prov,
|
||||
Provider,
|
||||
):
|
||||
self.curr_provider_inst = prov
|
||||
await sp.put_async(
|
||||
key="curr_provider",
|
||||
value=provider_id,
|
||||
scope="global",
|
||||
scope_id="global",
|
||||
)
|
||||
sp.put("curr_provider", provider_id, scope="global", scope_id="global")
|
||||
|
||||
async def get_provider_by_id(self, provider_id: str) -> Providers | None:
|
||||
"""根据提供商 ID 获取提供商实例"""
|
||||
@@ -166,7 +148,6 @@ class ProviderManager:
|
||||
|
||||
"""
|
||||
provider = None
|
||||
provider_id = None
|
||||
if umo:
|
||||
provider_id = sp.get(
|
||||
f"provider_perf_{provider_type.value}",
|
||||
@@ -204,12 +185,6 @@ class ProviderManager:
|
||||
)
|
||||
else:
|
||||
raise ValueError(f"Unknown provider type: {provider_type}")
|
||||
|
||||
if not provider and provider_id:
|
||||
logger.warning(
|
||||
f"没有找到 ID 为 {provider_id} 的提供商,这可能是由于您修改了提供商(模型)ID 导致的。"
|
||||
)
|
||||
|
||||
return provider
|
||||
|
||||
async def initialize(self):
|
||||
@@ -221,21 +196,21 @@ class ProviderManager:
|
||||
logger.error(traceback.format_exc())
|
||||
logger.error(e)
|
||||
|
||||
selected_provider_id = await sp.get_async(
|
||||
key="curr_provider",
|
||||
default=self.provider_settings.get("default_provider_id"),
|
||||
selected_provider_id = sp.get(
|
||||
"curr_provider",
|
||||
self.provider_settings.get("default_provider_id"),
|
||||
scope="global",
|
||||
scope_id="global",
|
||||
)
|
||||
selected_stt_provider_id = await sp.get_async(
|
||||
key="curr_provider_stt",
|
||||
default=self.provider_stt_settings.get("provider_id"),
|
||||
selected_stt_provider_id = sp.get(
|
||||
"curr_provider_stt",
|
||||
self.provider_stt_settings.get("provider_id"),
|
||||
scope="global",
|
||||
scope_id="global",
|
||||
)
|
||||
selected_tts_provider_id = await sp.get_async(
|
||||
key="curr_provider_tts",
|
||||
default=self.provider_tts_settings.get("provider_id"),
|
||||
selected_tts_provider_id = sp.get(
|
||||
"curr_provider_tts",
|
||||
self.provider_tts_settings.get("provider_id"),
|
||||
scope="global",
|
||||
scope_id="global",
|
||||
)
|
||||
@@ -276,136 +251,7 @@ class ProviderManager:
|
||||
# 初始化 MCP Client 连接
|
||||
asyncio.create_task(self.llm_tools.init_mcp_clients(), name="init_mcp_clients")
|
||||
|
||||
def dynamic_import_provider(self, type: str):
|
||||
"""动态导入提供商适配器模块
|
||||
|
||||
Args:
|
||||
type (str): 提供商请求类型。
|
||||
|
||||
Raises:
|
||||
ImportError: 如果提供商类型未知或无法导入对应模块,则抛出异常。
|
||||
"""
|
||||
match type:
|
||||
case "openai_chat_completion":
|
||||
from .sources.openai_source import (
|
||||
ProviderOpenAIOfficial as ProviderOpenAIOfficial,
|
||||
)
|
||||
case "zhipu_chat_completion":
|
||||
from .sources.zhipu_source import ProviderZhipu as ProviderZhipu
|
||||
case "groq_chat_completion":
|
||||
from .sources.groq_source import ProviderGroq as ProviderGroq
|
||||
case "anthropic_chat_completion":
|
||||
from .sources.anthropic_source import (
|
||||
ProviderAnthropic as ProviderAnthropic,
|
||||
)
|
||||
case "googlegenai_chat_completion":
|
||||
from .sources.gemini_source import (
|
||||
ProviderGoogleGenAI as ProviderGoogleGenAI,
|
||||
)
|
||||
case "sensevoice_stt_selfhost":
|
||||
from .sources.sensevoice_selfhosted_source import (
|
||||
ProviderSenseVoiceSTTSelfHost as ProviderSenseVoiceSTTSelfHost,
|
||||
)
|
||||
case "openai_whisper_api":
|
||||
from .sources.whisper_api_source import (
|
||||
ProviderOpenAIWhisperAPI as ProviderOpenAIWhisperAPI,
|
||||
)
|
||||
case "openai_whisper_selfhost":
|
||||
from .sources.whisper_selfhosted_source import (
|
||||
ProviderOpenAIWhisperSelfHost as ProviderOpenAIWhisperSelfHost,
|
||||
)
|
||||
case "xinference_stt":
|
||||
from .sources.xinference_stt_provider import (
|
||||
ProviderXinferenceSTT as ProviderXinferenceSTT,
|
||||
)
|
||||
case "openai_tts_api":
|
||||
from .sources.openai_tts_api_source import (
|
||||
ProviderOpenAITTSAPI as ProviderOpenAITTSAPI,
|
||||
)
|
||||
case "edge_tts":
|
||||
from .sources.edge_tts_source import (
|
||||
ProviderEdgeTTS as ProviderEdgeTTS,
|
||||
)
|
||||
case "gsv_tts_selfhost":
|
||||
from .sources.gsv_selfhosted_source import (
|
||||
ProviderGSVTTS as ProviderGSVTTS,
|
||||
)
|
||||
case "gsvi_tts_api":
|
||||
from .sources.gsvi_tts_source import (
|
||||
ProviderGSVITTS as ProviderGSVITTS,
|
||||
)
|
||||
case "fishaudio_tts_api":
|
||||
from .sources.fishaudio_tts_api_source import (
|
||||
ProviderFishAudioTTSAPI as ProviderFishAudioTTSAPI,
|
||||
)
|
||||
case "dashscope_tts":
|
||||
from .sources.dashscope_tts import (
|
||||
ProviderDashscopeTTSAPI as ProviderDashscopeTTSAPI,
|
||||
)
|
||||
case "azure_tts":
|
||||
from .sources.azure_tts_source import (
|
||||
AzureTTSProvider as AzureTTSProvider,
|
||||
)
|
||||
case "minimax_tts_api":
|
||||
from .sources.minimax_tts_api_source import (
|
||||
ProviderMiniMaxTTSAPI as ProviderMiniMaxTTSAPI,
|
||||
)
|
||||
case "volcengine_tts":
|
||||
from .sources.volcengine_tts import (
|
||||
ProviderVolcengineTTS as ProviderVolcengineTTS,
|
||||
)
|
||||
case "gemini_tts":
|
||||
from .sources.gemini_tts_source import (
|
||||
ProviderGeminiTTSAPI as ProviderGeminiTTSAPI,
|
||||
)
|
||||
case "openai_embedding":
|
||||
from .sources.openai_embedding_source import (
|
||||
OpenAIEmbeddingProvider as OpenAIEmbeddingProvider,
|
||||
)
|
||||
case "gemini_embedding":
|
||||
from .sources.gemini_embedding_source import (
|
||||
GeminiEmbeddingProvider as GeminiEmbeddingProvider,
|
||||
)
|
||||
case "vllm_rerank":
|
||||
from .sources.vllm_rerank_source import (
|
||||
VLLMRerankProvider as VLLMRerankProvider,
|
||||
)
|
||||
case "xinference_rerank":
|
||||
from .sources.xinference_rerank_source import (
|
||||
XinferenceRerankProvider as XinferenceRerankProvider,
|
||||
)
|
||||
case "bailian_rerank":
|
||||
from .sources.bailian_rerank_source import (
|
||||
BailianRerankProvider as BailianRerankProvider,
|
||||
)
|
||||
|
||||
def get_merged_provider_config(self, provider_config: dict) -> dict:
|
||||
"""获取 provider 配置和 provider_source 配置合并后的结果
|
||||
|
||||
Returns:
|
||||
dict: 合并后的 provider 配置,key 为 provider id,value 为合并后的配置字典
|
||||
"""
|
||||
pc = copy.deepcopy(provider_config)
|
||||
provider_source_id = pc.get("provider_source_id", "")
|
||||
if provider_source_id:
|
||||
provider_source = None
|
||||
for ps in self.provider_sources_config:
|
||||
if ps.get("id") == provider_source_id:
|
||||
provider_source = ps
|
||||
break
|
||||
|
||||
if provider_source:
|
||||
# 合并配置,provider 的配置优先级更高
|
||||
merged_config = {**provider_source, **pc}
|
||||
# 保持 id 为 provider 的 id,而不是 source 的 id
|
||||
merged_config["id"] = pc["id"]
|
||||
pc = merged_config
|
||||
return pc
|
||||
|
||||
async def load_provider(self, provider_config: dict):
|
||||
# 如果 provider_source_id 存在且不为空,则从 provider_sources 中找到对应的配置并合并
|
||||
provider_config = self.get_merged_provider_config(provider_config)
|
||||
|
||||
if not provider_config["enable"]:
|
||||
logger.info(f"Provider {provider_config['id']} is disabled, skipping")
|
||||
return
|
||||
@@ -418,7 +264,99 @@ class ProviderManager:
|
||||
|
||||
# 动态导入
|
||||
try:
|
||||
self.dynamic_import_provider(provider_config["type"])
|
||||
match provider_config["type"]:
|
||||
case "openai_chat_completion":
|
||||
from .sources.openai_source import (
|
||||
ProviderOpenAIOfficial as ProviderOpenAIOfficial,
|
||||
)
|
||||
case "zhipu_chat_completion":
|
||||
from .sources.zhipu_source import ProviderZhipu as ProviderZhipu
|
||||
case "groq_chat_completion":
|
||||
from .sources.groq_source import ProviderGroq as ProviderGroq
|
||||
case "anthropic_chat_completion":
|
||||
from .sources.anthropic_source import (
|
||||
ProviderAnthropic as ProviderAnthropic,
|
||||
)
|
||||
case "googlegenai_chat_completion":
|
||||
from .sources.gemini_source import (
|
||||
ProviderGoogleGenAI as ProviderGoogleGenAI,
|
||||
)
|
||||
case "sensevoice_stt_selfhost":
|
||||
from .sources.sensevoice_selfhosted_source import (
|
||||
ProviderSenseVoiceSTTSelfHost as ProviderSenseVoiceSTTSelfHost,
|
||||
)
|
||||
case "openai_whisper_api":
|
||||
from .sources.whisper_api_source import (
|
||||
ProviderOpenAIWhisperAPI as ProviderOpenAIWhisperAPI,
|
||||
)
|
||||
case "openai_whisper_selfhost":
|
||||
from .sources.whisper_selfhosted_source import (
|
||||
ProviderOpenAIWhisperSelfHost as ProviderOpenAIWhisperSelfHost,
|
||||
)
|
||||
case "xinference_stt":
|
||||
from .sources.xinference_stt_provider import (
|
||||
ProviderXinferenceSTT as ProviderXinferenceSTT,
|
||||
)
|
||||
case "openai_tts_api":
|
||||
from .sources.openai_tts_api_source import (
|
||||
ProviderOpenAITTSAPI as ProviderOpenAITTSAPI,
|
||||
)
|
||||
case "edge_tts":
|
||||
from .sources.edge_tts_source import (
|
||||
ProviderEdgeTTS as ProviderEdgeTTS,
|
||||
)
|
||||
case "gsv_tts_selfhost":
|
||||
from .sources.gsv_selfhosted_source import (
|
||||
ProviderGSVTTS as ProviderGSVTTS,
|
||||
)
|
||||
case "gsvi_tts_api":
|
||||
from .sources.gsvi_tts_source import (
|
||||
ProviderGSVITTS as ProviderGSVITTS,
|
||||
)
|
||||
case "fishaudio_tts_api":
|
||||
from .sources.fishaudio_tts_api_source import (
|
||||
ProviderFishAudioTTSAPI as ProviderFishAudioTTSAPI,
|
||||
)
|
||||
case "dashscope_tts":
|
||||
from .sources.dashscope_tts import (
|
||||
ProviderDashscopeTTSAPI as ProviderDashscopeTTSAPI,
|
||||
)
|
||||
case "azure_tts":
|
||||
from .sources.azure_tts_source import (
|
||||
AzureTTSProvider as AzureTTSProvider,
|
||||
)
|
||||
case "minimax_tts_api":
|
||||
from .sources.minimax_tts_api_source import (
|
||||
ProviderMiniMaxTTSAPI as ProviderMiniMaxTTSAPI,
|
||||
)
|
||||
case "volcengine_tts":
|
||||
from .sources.volcengine_tts import (
|
||||
ProviderVolcengineTTS as ProviderVolcengineTTS,
|
||||
)
|
||||
case "gemini_tts":
|
||||
from .sources.gemini_tts_source import (
|
||||
ProviderGeminiTTSAPI as ProviderGeminiTTSAPI,
|
||||
)
|
||||
case "openai_embedding":
|
||||
from .sources.openai_embedding_source import (
|
||||
OpenAIEmbeddingProvider as OpenAIEmbeddingProvider,
|
||||
)
|
||||
case "gemini_embedding":
|
||||
from .sources.gemini_embedding_source import (
|
||||
GeminiEmbeddingProvider as GeminiEmbeddingProvider,
|
||||
)
|
||||
case "vllm_rerank":
|
||||
from .sources.vllm_rerank_source import (
|
||||
VLLMRerankProvider as VLLMRerankProvider,
|
||||
)
|
||||
case "xinference_rerank":
|
||||
from .sources.xinference_rerank_source import (
|
||||
XinferenceRerankProvider as XinferenceRerankProvider,
|
||||
)
|
||||
case "bailian_rerank":
|
||||
from .sources.bailian_rerank_source import (
|
||||
BailianRerankProvider as BailianRerankProvider,
|
||||
)
|
||||
except (ImportError, ModuleNotFoundError) as e:
|
||||
logger.critical(
|
||||
f"加载 {provider_config['type']}({provider_config['id']}) 提供商适配器失败:{e}。可能是因为有未安装的依赖。",
|
||||
@@ -561,7 +499,6 @@ class ProviderManager:
|
||||
|
||||
# 和配置文件保持同步
|
||||
self.providers_config = astrbot_config["provider"]
|
||||
self.provider_sources_config = astrbot_config.get("provider_sources", [])
|
||||
config_ids = [provider["id"] for provider in self.providers_config]
|
||||
logger.info(f"providers in user's config: {config_ids}")
|
||||
for key in list(self.inst_map.keys()):
|
||||
@@ -633,68 +570,6 @@ class ProviderManager:
|
||||
)
|
||||
del self.inst_map[provider_id]
|
||||
|
||||
async def delete_provider(
|
||||
self, provider_id: str | None = None, provider_source_id: str | None = None
|
||||
):
|
||||
"""Delete provider and/or provider source from config and terminate the instances. Config will be saved after deletion."""
|
||||
async with self.resource_lock:
|
||||
# delete from config
|
||||
target_prov_ids = []
|
||||
if provider_id:
|
||||
target_prov_ids.append(provider_id)
|
||||
else:
|
||||
for prov in self.providers_config:
|
||||
if prov.get("provider_source_id") == provider_source_id:
|
||||
target_prov_ids.append(prov.get("id"))
|
||||
config = self.acm.default_conf
|
||||
for tpid in target_prov_ids:
|
||||
await self.terminate_provider(tpid)
|
||||
config["provider"] = [
|
||||
prov for prov in config["provider"] if prov.get("id") != tpid
|
||||
]
|
||||
config.save_config()
|
||||
logger.info(f"Provider {target_prov_ids} 已从配置中删除。")
|
||||
|
||||
async def update_provider(self, origin_provider_id: str, new_config: dict):
|
||||
"""Update provider config and reload the instance. Config will be saved after update."""
|
||||
async with self.resource_lock:
|
||||
npid = new_config.get("id", None)
|
||||
if not npid:
|
||||
raise ValueError("New provider config must have an 'id' field")
|
||||
config = self.acm.default_conf
|
||||
for provider in config["provider"]:
|
||||
if (
|
||||
provider.get("id", None) == npid
|
||||
and provider.get("id", None) != origin_provider_id
|
||||
):
|
||||
raise ValueError(f"Provider ID {npid} already exists")
|
||||
# update config
|
||||
for idx, provider in enumerate(config["provider"]):
|
||||
if provider.get("id", None) == origin_provider_id:
|
||||
config["provider"][idx] = new_config
|
||||
break
|
||||
else:
|
||||
raise ValueError(f"Provider ID {origin_provider_id} not found")
|
||||
config.save_config()
|
||||
# reload instance
|
||||
await self.reload(new_config)
|
||||
|
||||
async def create_provider(self, new_config: dict):
|
||||
"""Add new provider config and load the instance. Config will be saved after addition."""
|
||||
async with self.resource_lock:
|
||||
npid = new_config.get("id", None)
|
||||
if not npid:
|
||||
raise ValueError("New provider config must have an 'id' field")
|
||||
config = self.acm.default_conf
|
||||
for provider in config["provider"]:
|
||||
if provider.get("id", None) == npid:
|
||||
raise ValueError(f"Provider ID {npid} already exists")
|
||||
# add to config
|
||||
config["provider"].append(new_config)
|
||||
config.save_config()
|
||||
# load instance
|
||||
await self.load_provider(new_config)
|
||||
|
||||
async def terminate(self):
|
||||
for provider_inst in self.provider_insts:
|
||||
if hasattr(provider_inst, "terminate"):
|
||||
|
||||
@@ -4,7 +4,7 @@ import os
|
||||
from collections.abc import AsyncGenerator
|
||||
from typing import TypeAlias, Union
|
||||
|
||||
from astrbot.core.agent.message import ContentPart, Message
|
||||
from astrbot.core.agent.message import Message
|
||||
from astrbot.core.agent.tool import ToolSet
|
||||
from astrbot.core.provider.entities import (
|
||||
LLMResponse,
|
||||
@@ -103,7 +103,6 @@ class Provider(AbstractProvider):
|
||||
system_prompt: str | None = None,
|
||||
tool_calls_result: ToolCallsResult | list[ToolCallsResult] | None = None,
|
||||
model: str | None = None,
|
||||
extra_user_content_parts: list[ContentPart] | None = None,
|
||||
**kwargs,
|
||||
) -> LLMResponse:
|
||||
"""获得 LLM 的文本对话结果。会使用当前的模型进行对话。
|
||||
@@ -115,7 +114,6 @@ class Provider(AbstractProvider):
|
||||
tools: tool set
|
||||
contexts: 上下文,和 prompt 二选一使用
|
||||
tool_calls_result: 回传给 LLM 的工具调用结果。参考: https://platform.openai.com/docs/guides/function-calling
|
||||
extra_user_content_parts: 额外的内容块列表,用于在用户消息后添加额外的文本块(如系统提醒、指令等)
|
||||
kwargs: 其他参数
|
||||
|
||||
Notes:
|
||||
|
||||
@@ -6,13 +6,10 @@ from mimetypes import guess_type
|
||||
import anthropic
|
||||
from anthropic import AsyncAnthropic
|
||||
from anthropic.types import Message
|
||||
from anthropic.types.message_delta_usage import MessageDeltaUsage
|
||||
from anthropic.types.usage import Usage
|
||||
|
||||
from astrbot import logger
|
||||
from astrbot.api.provider import Provider
|
||||
from astrbot.core.agent.message import ContentPart, ImageURLPart, TextPart
|
||||
from astrbot.core.provider.entities import LLMResponse, TokenUsage
|
||||
from astrbot.core.provider.entities import LLMResponse
|
||||
from astrbot.core.provider.func_tool_manager import ToolSet
|
||||
from astrbot.core.utils.io import download_image_by_url
|
||||
|
||||
@@ -48,9 +45,7 @@ class ProviderAnthropic(Provider):
|
||||
base_url=self.base_url,
|
||||
)
|
||||
|
||||
self.thinking_config = provider_config.get("anth_thinking_config", {})
|
||||
|
||||
self.set_model(provider_config.get("model", "unknown"))
|
||||
self.set_model(provider_config["model_config"]["model"])
|
||||
|
||||
def _prepare_payload(self, messages: list[dict]):
|
||||
"""准备 Anthropic API 的请求 payload
|
||||
@@ -66,33 +61,12 @@ class ProviderAnthropic(Provider):
|
||||
new_messages = []
|
||||
for message in messages:
|
||||
if message["role"] == "system":
|
||||
system_prompt = message["content"] or "<empty system prompt>"
|
||||
system_prompt = message["content"]
|
||||
elif message["role"] == "assistant":
|
||||
blocks = []
|
||||
reasoning_content = ""
|
||||
thinking_signature = ""
|
||||
if isinstance(message["content"], str) and message["content"].strip():
|
||||
if isinstance(message["content"], str):
|
||||
blocks.append({"type": "text", "text": message["content"]})
|
||||
elif isinstance(message["content"], list):
|
||||
for part in message["content"]:
|
||||
if part.get("type") == "think":
|
||||
# only pick the last think part for now
|
||||
reasoning_content = part.get("think")
|
||||
thinking_signature = part.get("encrypted")
|
||||
else:
|
||||
blocks.append(part)
|
||||
|
||||
if reasoning_content and thinking_signature:
|
||||
blocks.insert(
|
||||
0,
|
||||
{
|
||||
"type": "thinking",
|
||||
"thinking": reasoning_content,
|
||||
"signature": thinking_signature,
|
||||
},
|
||||
)
|
||||
|
||||
if "tool_calls" in message and isinstance(message["tool_calls"], list):
|
||||
if "tool_calls" in message:
|
||||
for tool_call in message["tool_calls"]:
|
||||
blocks.append( # noqa: PERF401
|
||||
{
|
||||
@@ -123,7 +97,7 @@ class ProviderAnthropic(Provider):
|
||||
{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": message["tool_call_id"],
|
||||
"content": message["content"] or "<empty response>",
|
||||
"content": message["content"],
|
||||
},
|
||||
],
|
||||
},
|
||||
@@ -133,40 +107,12 @@ class ProviderAnthropic(Provider):
|
||||
|
||||
return system_prompt, new_messages
|
||||
|
||||
def _extract_usage(self, usage: Usage) -> TokenUsage:
|
||||
# https://docs.claude.com/en/docs/build-with-claude/prompt-caching#tracking-cache-performance
|
||||
return TokenUsage(
|
||||
input_other=usage.input_tokens or 0,
|
||||
input_cached=usage.cache_read_input_tokens or 0,
|
||||
output=usage.output_tokens,
|
||||
)
|
||||
|
||||
def _update_usage(self, token_usage: TokenUsage, usage: MessageDeltaUsage) -> None:
|
||||
if usage.input_tokens is not None:
|
||||
token_usage.input_other = usage.input_tokens
|
||||
if usage.cache_read_input_tokens is not None:
|
||||
token_usage.input_cached = usage.cache_read_input_tokens
|
||||
if usage.output_tokens is not None:
|
||||
token_usage.output = usage.output_tokens
|
||||
|
||||
async def _query(self, payloads: dict, tools: ToolSet | None) -> LLMResponse:
|
||||
if tools:
|
||||
if tool_list := tools.get_func_desc_anthropic_style():
|
||||
payloads["tools"] = tool_list
|
||||
|
||||
extra_body = self.provider_config.get("custom_extra_body", {})
|
||||
|
||||
if "max_tokens" not in payloads:
|
||||
payloads["max_tokens"] = 1024
|
||||
if self.thinking_config.get("budget"):
|
||||
payloads["thinking"] = {
|
||||
"budget_tokens": self.thinking_config.get("budget"),
|
||||
"type": "enabled",
|
||||
}
|
||||
|
||||
completion = await self.client.messages.create(
|
||||
**payloads, stream=False, extra_body=extra_body
|
||||
)
|
||||
completion = await self.client.messages.create(**payloads, stream=False)
|
||||
|
||||
assert isinstance(completion, Message)
|
||||
logger.debug(f"completion: {completion}")
|
||||
@@ -181,19 +127,10 @@ class ProviderAnthropic(Provider):
|
||||
completion_text = str(content_block.text).strip()
|
||||
llm_response.completion_text = completion_text
|
||||
|
||||
if content_block.type == "thinking":
|
||||
reasoning_content = str(content_block.thinking).strip()
|
||||
llm_response.reasoning_content = reasoning_content
|
||||
llm_response.reasoning_signature = content_block.signature
|
||||
|
||||
if content_block.type == "tool_use":
|
||||
llm_response.tools_call_args.append(content_block.input)
|
||||
llm_response.tools_call_name.append(content_block.name)
|
||||
llm_response.tools_call_ids.append(content_block.id)
|
||||
|
||||
llm_response.id = completion.id
|
||||
llm_response.usage = self._extract_usage(completion.usage)
|
||||
|
||||
# TODO(Soulter): 处理 end_turn 情况
|
||||
if not llm_response.completion_text and not llm_response.tools_call_args:
|
||||
raise Exception(f"Anthropic API 返回的 completion 无法解析:{completion}。")
|
||||
@@ -214,29 +151,10 @@ class ProviderAnthropic(Provider):
|
||||
# 用于累积最终结果
|
||||
final_text = ""
|
||||
final_tool_calls = []
|
||||
id = None
|
||||
usage = TokenUsage()
|
||||
extra_body = self.provider_config.get("custom_extra_body", {})
|
||||
reasoning_content = ""
|
||||
reasoning_signature = ""
|
||||
|
||||
if "max_tokens" not in payloads:
|
||||
payloads["max_tokens"] = 1024
|
||||
if self.thinking_config.get("budget"):
|
||||
payloads["thinking"] = {
|
||||
"budget_tokens": self.thinking_config.get("budget"),
|
||||
"type": "enabled",
|
||||
}
|
||||
|
||||
async with self.client.messages.stream(
|
||||
**payloads, extra_body=extra_body
|
||||
) as stream:
|
||||
async with self.client.messages.stream(**payloads) as stream:
|
||||
assert isinstance(stream, anthropic.AsyncMessageStream)
|
||||
async for event in stream:
|
||||
if event.type == "message_start":
|
||||
# the usage contains input token usage
|
||||
id = event.message.id
|
||||
usage = self._extract_usage(event.message.usage)
|
||||
if event.type == "content_block_start":
|
||||
if event.content_block.type == "text":
|
||||
# 文本块开始
|
||||
@@ -244,8 +162,6 @@ class ProviderAnthropic(Provider):
|
||||
role="assistant",
|
||||
completion_text="",
|
||||
is_chunk=True,
|
||||
usage=usage,
|
||||
id=id,
|
||||
)
|
||||
elif event.content_block.type == "tool_use":
|
||||
# 工具使用块开始,初始化缓冲区
|
||||
@@ -263,24 +179,7 @@ class ProviderAnthropic(Provider):
|
||||
role="assistant",
|
||||
completion_text=event.delta.text,
|
||||
is_chunk=True,
|
||||
usage=usage,
|
||||
id=id,
|
||||
)
|
||||
elif event.delta.type == "thinking_delta":
|
||||
# 思考增量
|
||||
reasoning = event.delta.thinking
|
||||
if reasoning:
|
||||
yield LLMResponse(
|
||||
role="assistant",
|
||||
reasoning_content=reasoning,
|
||||
is_chunk=True,
|
||||
usage=usage,
|
||||
id=id,
|
||||
reasoning_signature=reasoning_signature or None,
|
||||
)
|
||||
reasoning_content += reasoning
|
||||
elif event.delta.type == "signature_delta":
|
||||
reasoning_signature = event.delta.signature
|
||||
elif event.delta.type == "input_json_delta":
|
||||
# 工具调用参数增量
|
||||
if event.index in tool_use_buffer:
|
||||
@@ -316,8 +215,6 @@ class ProviderAnthropic(Provider):
|
||||
tools_call_name=[tool_info["name"]],
|
||||
tools_call_ids=[tool_info["id"]],
|
||||
is_chunk=True,
|
||||
usage=usage,
|
||||
id=id,
|
||||
)
|
||||
except json.JSONDecodeError:
|
||||
# JSON 解析失败,跳过这个工具调用
|
||||
@@ -326,19 +223,11 @@ class ProviderAnthropic(Provider):
|
||||
# 清理缓冲区
|
||||
del tool_use_buffer[event.index]
|
||||
|
||||
elif event.type == "message_delta":
|
||||
if event.usage:
|
||||
self._update_usage(usage, event.usage)
|
||||
|
||||
# 返回最终的完整结果
|
||||
final_response = LLMResponse(
|
||||
role="assistant",
|
||||
completion_text=final_text,
|
||||
is_chunk=False,
|
||||
usage=usage,
|
||||
id=id,
|
||||
reasoning_content=reasoning_content,
|
||||
reasoning_signature=reasoning_signature or None,
|
||||
)
|
||||
|
||||
if final_tool_calls:
|
||||
@@ -360,16 +249,13 @@ class ProviderAnthropic(Provider):
|
||||
system_prompt=None,
|
||||
tool_calls_result=None,
|
||||
model=None,
|
||||
extra_user_content_parts=None,
|
||||
**kwargs,
|
||||
) -> LLMResponse:
|
||||
if contexts is None:
|
||||
contexts = []
|
||||
new_record = None
|
||||
if prompt is not None:
|
||||
new_record = await self.assemble_context(
|
||||
prompt, image_urls, extra_user_content_parts
|
||||
)
|
||||
new_record = await self.assemble_context(prompt, image_urls)
|
||||
context_query = self._ensure_message_to_dicts(contexts)
|
||||
if new_record:
|
||||
context_query.append(new_record)
|
||||
@@ -391,9 +277,10 @@ class ProviderAnthropic(Provider):
|
||||
|
||||
system_prompt, new_messages = self._prepare_payload(context_query)
|
||||
|
||||
model = model or self.get_model()
|
||||
model_config = self.provider_config.get("model_config", {})
|
||||
model_config["model"] = model or self.get_model()
|
||||
|
||||
payloads = {"messages": new_messages, "model": model}
|
||||
payloads = {"messages": new_messages, **model_config}
|
||||
|
||||
# Anthropic has a different way of handling system prompts
|
||||
if system_prompt:
|
||||
@@ -403,30 +290,28 @@ class ProviderAnthropic(Provider):
|
||||
try:
|
||||
llm_response = await self._query(payloads, func_tool)
|
||||
except Exception as e:
|
||||
# logger.error(f"发生了错误。Provider 配置如下: {model_config}")
|
||||
raise e
|
||||
|
||||
return llm_response
|
||||
|
||||
async def text_chat_stream(
|
||||
self,
|
||||
prompt=None,
|
||||
prompt,
|
||||
session_id=None,
|
||||
image_urls=None,
|
||||
image_urls=...,
|
||||
func_tool=None,
|
||||
contexts=None,
|
||||
contexts=...,
|
||||
system_prompt=None,
|
||||
tool_calls_result=None,
|
||||
model=None,
|
||||
extra_user_content_parts=None,
|
||||
**kwargs,
|
||||
):
|
||||
if contexts is None:
|
||||
contexts = []
|
||||
new_record = None
|
||||
if prompt is not None:
|
||||
new_record = await self.assemble_context(
|
||||
prompt, image_urls, extra_user_content_parts
|
||||
)
|
||||
new_record = await self.assemble_context(prompt, image_urls)
|
||||
context_query = self._ensure_message_to_dicts(contexts)
|
||||
if new_record:
|
||||
context_query.append(new_record)
|
||||
@@ -447,9 +332,10 @@ class ProviderAnthropic(Provider):
|
||||
|
||||
system_prompt, new_messages = self._prepare_payload(context_query)
|
||||
|
||||
model = model or self.get_model()
|
||||
model_config = self.provider_config.get("model_config", {})
|
||||
model_config["model"] = model or self.get_model()
|
||||
|
||||
payloads = {"messages": new_messages, "model": model}
|
||||
payloads = {"messages": new_messages, **model_config}
|
||||
|
||||
# Anthropic has a different way of handling system prompts
|
||||
if system_prompt:
|
||||
@@ -458,15 +344,15 @@ class ProviderAnthropic(Provider):
|
||||
async for llm_response in self._query_stream(payloads, func_tool):
|
||||
yield llm_response
|
||||
|
||||
async def assemble_context(
|
||||
self,
|
||||
text: str,
|
||||
image_urls: list[str] | None = None,
|
||||
extra_user_content_parts: list[ContentPart] | None = None,
|
||||
):
|
||||
async def assemble_context(self, text: str, image_urls: list[str] | None = None):
|
||||
"""组装上下文,支持文本和图片"""
|
||||
if not image_urls:
|
||||
return {"role": "user", "content": text}
|
||||
|
||||
async def resolve_image_url(image_url: str) -> dict | None:
|
||||
content = []
|
||||
content.append({"type": "text", "text": text})
|
||||
|
||||
for image_url in image_urls:
|
||||
if image_url.startswith("http"):
|
||||
image_path = await download_image_by_url(image_url)
|
||||
image_data = await self.encode_image_bs64(image_path)
|
||||
@@ -478,68 +364,28 @@ class ProviderAnthropic(Provider):
|
||||
|
||||
if not image_data:
|
||||
logger.warning(f"图片 {image_url} 得到的结果为空,将忽略。")
|
||||
return None
|
||||
continue
|
||||
|
||||
# Get mime type for the image
|
||||
mime_type, _ = guess_type(image_url)
|
||||
if not mime_type:
|
||||
mime_type = "image/jpeg" # Default to JPEG if can't determine
|
||||
|
||||
return {
|
||||
"type": "image",
|
||||
"source": {
|
||||
"type": "base64",
|
||||
"media_type": mime_type,
|
||||
"data": (
|
||||
image_data.split("base64,")[1]
|
||||
if "base64," in image_data
|
||||
else image_data
|
||||
),
|
||||
content.append(
|
||||
{
|
||||
"type": "image",
|
||||
"source": {
|
||||
"type": "base64",
|
||||
"media_type": mime_type,
|
||||
"data": (
|
||||
image_data.split("base64,")[1]
|
||||
if "base64," in image_data
|
||||
else image_data
|
||||
),
|
||||
},
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
content = []
|
||||
|
||||
# 1. 用户原始发言(OpenAI 建议:用户发言在前)
|
||||
if text:
|
||||
content.append({"type": "text", "text": text})
|
||||
elif image_urls:
|
||||
# 如果没有文本但有图片,添加占位文本
|
||||
content.append({"type": "text", "text": "[图片]"})
|
||||
elif extra_user_content_parts:
|
||||
# 如果只有额外内容块,也需要添加占位文本
|
||||
content.append({"type": "text", "text": " "})
|
||||
|
||||
# 2. 额外的内容块(系统提醒、指令等)
|
||||
if extra_user_content_parts:
|
||||
for block in extra_user_content_parts:
|
||||
if isinstance(block, TextPart):
|
||||
content.append({"type": "text", "text": block.text})
|
||||
elif isinstance(block, ImageURLPart):
|
||||
image_dict = await resolve_image_url(block.image_url.url)
|
||||
if image_dict:
|
||||
content.append(image_dict)
|
||||
else:
|
||||
raise ValueError(f"不支持的额外内容块类型: {type(block)}")
|
||||
|
||||
# 3. 图片内容
|
||||
if image_urls:
|
||||
for image_url in image_urls:
|
||||
image_dict = await resolve_image_url(image_url)
|
||||
if image_dict:
|
||||
content.append(image_dict)
|
||||
|
||||
# 如果只有主文本且没有额外内容块和图片,返回简单格式以保持向后兼容
|
||||
if (
|
||||
text
|
||||
and not extra_user_content_parts
|
||||
and not image_urls
|
||||
and len(content) == 1
|
||||
and content[0]["type"] == "text"
|
||||
):
|
||||
return {"role": "user", "content": content[0]["text"]}
|
||||
|
||||
# 否则返回多模态格式
|
||||
return {"role": "user", "content": content}
|
||||
|
||||
async def encode_image_bs64(self, image_url: str) -> str:
|
||||
|
||||
@@ -56,14 +56,10 @@ class ProviderFishAudioTTSAPI(TTSProvider):
|
||||
"api_base",
|
||||
"https://api.fish-audio.cn/v1",
|
||||
)
|
||||
try:
|
||||
self.timeout: int = int(provider_config.get("timeout", 20))
|
||||
except ValueError:
|
||||
self.timeout = 20
|
||||
self.headers = {
|
||||
"Authorization": f"Bearer {self.chosen_api_key}",
|
||||
}
|
||||
self.set_model(provider_config.get("model", None))
|
||||
self.set_model(provider_config["model"])
|
||||
|
||||
async def _get_reference_id_by_character(self, character: str) -> str | None:
|
||||
"""获取角色的reference_id
|
||||
@@ -139,21 +135,17 @@ class ProviderFishAudioTTSAPI(TTSProvider):
|
||||
path = os.path.join(temp_dir, f"fishaudio_tts_api_{uuid.uuid4()}.wav")
|
||||
self.headers["content-type"] = "application/msgpack"
|
||||
request = await self._generate_request(text)
|
||||
async with AsyncClient(base_url=self.api_base, timeout=self.timeout).stream(
|
||||
async with AsyncClient(base_url=self.api_base).stream(
|
||||
"POST",
|
||||
"/tts",
|
||||
headers=self.headers,
|
||||
content=ormsgpack.packb(request, option=ormsgpack.OPT_SERIALIZE_PYDANTIC),
|
||||
) as response:
|
||||
if response.status_code == 200 and response.headers.get(
|
||||
"content-type", ""
|
||||
).startswith("audio/"):
|
||||
if response.headers["content-type"] == "audio/wav":
|
||||
with open(path, "wb") as f:
|
||||
async for chunk in response.aiter_bytes():
|
||||
f.write(chunk)
|
||||
return path
|
||||
error_bytes = await response.aread()
|
||||
error_text = error_bytes.decode("utf-8", errors="replace")[:1024]
|
||||
raise Exception(
|
||||
f"Fish Audio API请求失败: 状态码 {response.status_code}, 响应内容: {error_text}"
|
||||
)
|
||||
body = await response.aread()
|
||||
text = body.decode("utf-8", errors="replace")
|
||||
raise Exception(f"Fish Audio API请求失败: {text}")
|
||||
|
||||
@@ -13,9 +13,8 @@ from google.genai.errors import APIError
|
||||
import astrbot.core.message.components as Comp
|
||||
from astrbot import logger
|
||||
from astrbot.api.provider import Provider
|
||||
from astrbot.core.agent.message import ContentPart, ImageURLPart, TextPart
|
||||
from astrbot.core.message.message_event_result import MessageChain
|
||||
from astrbot.core.provider.entities import LLMResponse, TokenUsage
|
||||
from astrbot.core.provider.entities import LLMResponse
|
||||
from astrbot.core.provider.func_tool_manager import ToolSet
|
||||
from astrbot.core.utils.io import download_image_by_url
|
||||
|
||||
@@ -69,7 +68,7 @@ class ProviderGoogleGenAI(Provider):
|
||||
self.api_base = self.api_base[:-1]
|
||||
|
||||
self._init_client()
|
||||
self.set_model(provider_config.get("model", "unknown"))
|
||||
self.set_model(provider_config["model_config"]["model"])
|
||||
self._init_safety_settings()
|
||||
|
||||
def _init_client(self) -> None:
|
||||
@@ -139,7 +138,7 @@ class ProviderGoogleGenAI(Provider):
|
||||
modalities = ["TEXT"]
|
||||
|
||||
tool_list: list[types.Tool] | None = []
|
||||
model_name = cast(str, payloads.get("model", self.get_model()))
|
||||
model_name = self.get_model()
|
||||
native_coderunner = self.provider_config.get("gm_native_coderunner", False)
|
||||
native_search = self.provider_config.get("gm_native_search", False)
|
||||
url_context = self.provider_config.get("gm_url_context", False)
|
||||
@@ -198,53 +197,6 @@ class ProviderGoogleGenAI(Provider):
|
||||
types.Tool(function_declarations=func_desc["function_declarations"]),
|
||||
]
|
||||
|
||||
# oper thinking config
|
||||
thinking_config = None
|
||||
if model_name in [
|
||||
"gemini-2.5-pro",
|
||||
"gemini-2.5-pro-preview",
|
||||
"gemini-2.5-flash",
|
||||
"gemini-2.5-flash-preview",
|
||||
"gemini-2.5-flash-lite",
|
||||
"gemini-2.5-flash-lite-preview",
|
||||
"gemini-robotics-er-1.5-preview",
|
||||
"gemini-live-2.5-flash-preview-native-audio-09-2025",
|
||||
]:
|
||||
# The thinkingBudget parameter, introduced with the Gemini 2.5 series
|
||||
thinking_budget = self.provider_config.get("gm_thinking_config", {}).get(
|
||||
"budget", 0
|
||||
)
|
||||
if thinking_budget is not None:
|
||||
thinking_config = types.ThinkingConfig(
|
||||
thinking_budget=thinking_budget,
|
||||
)
|
||||
elif model_name in [
|
||||
"gemini-3-pro",
|
||||
"gemini-3-pro-preview",
|
||||
"gemini-3-flash",
|
||||
"gemini-3-flash-preview",
|
||||
"gemini-3-flash-lite",
|
||||
"gemini-3-flash-lite-preview",
|
||||
]:
|
||||
# The thinkingLevel parameter, recommended for Gemini 3 models and onwards
|
||||
# Gemini 2.5 series models don't support thinkingLevel; use thinkingBudget instead.
|
||||
thinking_level = self.provider_config.get("gm_thinking_config", {}).get(
|
||||
"level", "HIGH"
|
||||
)
|
||||
if thinking_level and isinstance(thinking_level, str):
|
||||
thinking_level = thinking_level.upper()
|
||||
if thinking_level not in ["MINIMAL", "LOW", "MEDIUM", "HIGH"]:
|
||||
logger.warning(
|
||||
f"Invalid thinking level: {thinking_level}, using HIGH"
|
||||
)
|
||||
thinking_level = "HIGH"
|
||||
level = types.ThinkingLevel(thinking_level)
|
||||
thinking_config = types.ThinkingConfig()
|
||||
if not hasattr(types.ThinkingConfig, "thinking_level"):
|
||||
setattr(types.ThinkingConfig, "thinking_level", level)
|
||||
else:
|
||||
thinking_config.thinking_level = level
|
||||
|
||||
return types.GenerateContentConfig(
|
||||
system_instruction=system_instruction,
|
||||
temperature=temperature,
|
||||
@@ -264,7 +216,22 @@ class ProviderGoogleGenAI(Provider):
|
||||
response_modalities=modalities,
|
||||
tools=cast(types.ToolListUnion | None, tool_list),
|
||||
safety_settings=self.safety_settings if self.safety_settings else None,
|
||||
thinking_config=thinking_config,
|
||||
thinking_config=(
|
||||
types.ThinkingConfig(
|
||||
thinking_budget=min(
|
||||
int(
|
||||
self.provider_config.get("gm_thinking_config", {}).get(
|
||||
"budget",
|
||||
0,
|
||||
),
|
||||
),
|
||||
24576,
|
||||
),
|
||||
)
|
||||
if "gemini-2.5-flash" in self.get_model()
|
||||
and hasattr(types.ThinkingConfig, "thinking_budget")
|
||||
else None
|
||||
),
|
||||
automatic_function_calling=types.AutomaticFunctionCallingConfig(
|
||||
disable=True,
|
||||
),
|
||||
@@ -321,37 +288,9 @@ class ProviderGoogleGenAI(Provider):
|
||||
append_or_extend(gemini_contents, parts, types.UserContent)
|
||||
|
||||
elif role == "assistant":
|
||||
if isinstance(content, str):
|
||||
if content:
|
||||
parts = [types.Part.from_text(text=content)]
|
||||
append_or_extend(gemini_contents, parts, types.ModelContent)
|
||||
elif isinstance(content, list):
|
||||
parts = []
|
||||
thinking_signature = None
|
||||
text = ""
|
||||
for part in content:
|
||||
# for most cases, assistant content only contains two parts: think and text
|
||||
if part.get("type") == "think":
|
||||
thinking_signature = part.get("encrypted") or None
|
||||
else:
|
||||
text += str(part.get("text"))
|
||||
|
||||
if thinking_signature and isinstance(thinking_signature, str):
|
||||
try:
|
||||
thinking_signature = base64.b64decode(thinking_signature)
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"Failed to decode google gemini thinking signature: {e}",
|
||||
exc_info=True,
|
||||
)
|
||||
thinking_signature = None
|
||||
parts.append(
|
||||
types.Part(
|
||||
text=text,
|
||||
thought_signature=thinking_signature,
|
||||
)
|
||||
)
|
||||
append_or_extend(gemini_contents, parts, types.ModelContent)
|
||||
|
||||
elif not native_tool_enabled and "tool_calls" in message:
|
||||
parts = []
|
||||
for tool in message["tool_calls"]:
|
||||
@@ -408,16 +347,6 @@ class ProviderGoogleGenAI(Provider):
|
||||
]
|
||||
return "".join(thought_buf).strip()
|
||||
|
||||
def _extract_usage(
|
||||
self, usage_metadata: types.GenerateContentResponseUsageMetadata
|
||||
) -> TokenUsage:
|
||||
"""Extract usage from candidate"""
|
||||
return TokenUsage(
|
||||
input_other=usage_metadata.prompt_token_count or 0,
|
||||
input_cached=usage_metadata.cached_content_token_count or 0,
|
||||
output=usage_metadata.candidates_token_count or 0,
|
||||
)
|
||||
|
||||
def _process_content_parts(
|
||||
self,
|
||||
candidate: types.Candidate,
|
||||
@@ -469,8 +398,7 @@ class ProviderGoogleGenAI(Provider):
|
||||
for part in result_parts:
|
||||
if part.text:
|
||||
chain.append(Comp.Plain(part.text))
|
||||
|
||||
if (
|
||||
elif (
|
||||
part.function_call
|
||||
and part.function_call.name is not None
|
||||
and part.function_call.args is not None
|
||||
@@ -487,18 +415,13 @@ class ProviderGoogleGenAI(Provider):
|
||||
llm_response.tools_call_extra_content[tool_call_id] = {
|
||||
"google": {"thought_signature": ts_bs64}
|
||||
}
|
||||
|
||||
if (
|
||||
elif (
|
||||
part.inline_data
|
||||
and part.inline_data.mime_type
|
||||
and part.inline_data.mime_type.startswith("image/")
|
||||
and part.inline_data.data
|
||||
):
|
||||
chain.append(Comp.Image.fromBytes(part.inline_data.data))
|
||||
|
||||
if ts := part.thought_signature:
|
||||
# only keep the last thinking signature
|
||||
llm_response.reasoning_signature = base64.b64encode(ts).decode("utf-8")
|
||||
return MessageChain(chain=chain)
|
||||
|
||||
async def _query(self, payloads: dict, tools: ToolSet | None) -> LLMResponse:
|
||||
@@ -508,8 +431,6 @@ class ProviderGoogleGenAI(Provider):
|
||||
None,
|
||||
)
|
||||
|
||||
model = payloads.get("model", self.get_model())
|
||||
|
||||
modalities = ["TEXT"]
|
||||
if self.provider_config.get("gm_resp_image_modal", False):
|
||||
modalities.append("IMAGE")
|
||||
@@ -528,7 +449,7 @@ class ProviderGoogleGenAI(Provider):
|
||||
temperature,
|
||||
)
|
||||
result = await self.client.models.generate_content(
|
||||
model=model,
|
||||
model=self.get_model(),
|
||||
contents=cast(types.ContentListUnion, conversation),
|
||||
config=config,
|
||||
)
|
||||
@@ -554,11 +475,11 @@ class ProviderGoogleGenAI(Provider):
|
||||
e.message = ""
|
||||
if "Developer instruction is not enabled" in e.message:
|
||||
logger.warning(
|
||||
f"{model} 不支持 system prompt,已自动去除(影响人格设置)",
|
||||
f"{self.get_model()} 不支持 system prompt,已自动去除(影响人格设置)",
|
||||
)
|
||||
system_instruction = None
|
||||
elif "Function calling is not enabled" in e.message:
|
||||
logger.warning(f"{model} 不支持函数调用,已自动去除")
|
||||
logger.warning(f"{self.get_model()} 不支持函数调用,已自动去除")
|
||||
tools = None
|
||||
elif (
|
||||
"Multi-modal output is not supported" in e.message
|
||||
@@ -567,7 +488,7 @@ class ProviderGoogleGenAI(Provider):
|
||||
or "only supports text output" in e.message
|
||||
):
|
||||
logger.warning(
|
||||
f"{model} 不支持多模态输出,降级为文本模态",
|
||||
f"{self.get_model()} 不支持多模态输出,降级为文本模态",
|
||||
)
|
||||
modalities = ["TEXT"]
|
||||
else:
|
||||
@@ -580,9 +501,6 @@ class ProviderGoogleGenAI(Provider):
|
||||
result.candidates[0],
|
||||
llm_response,
|
||||
)
|
||||
llm_response.id = result.response_id
|
||||
if result.usage_metadata:
|
||||
llm_response.usage = self._extract_usage(result.usage_metadata)
|
||||
return llm_response
|
||||
|
||||
async def _query_stream(
|
||||
@@ -595,7 +513,7 @@ class ProviderGoogleGenAI(Provider):
|
||||
(msg["content"] for msg in payloads["messages"] if msg["role"] == "system"),
|
||||
None,
|
||||
)
|
||||
model = payloads.get("model", self.get_model())
|
||||
|
||||
conversation = self._prepare_conversation(payloads)
|
||||
|
||||
result = None
|
||||
@@ -607,7 +525,7 @@ class ProviderGoogleGenAI(Provider):
|
||||
system_instruction,
|
||||
)
|
||||
result = await self.client.models.generate_content_stream(
|
||||
model=model,
|
||||
model=self.get_model(),
|
||||
contents=cast(types.ContentListUnion, conversation),
|
||||
config=config,
|
||||
)
|
||||
@@ -617,11 +535,11 @@ class ProviderGoogleGenAI(Provider):
|
||||
e.message = ""
|
||||
if "Developer instruction is not enabled" in e.message:
|
||||
logger.warning(
|
||||
f"{model} 不支持 system prompt,已自动去除(影响人格设置)",
|
||||
f"{self.get_model()} 不支持 system prompt,已自动去除(影响人格设置)",
|
||||
)
|
||||
system_instruction = None
|
||||
elif "Function calling is not enabled" in e.message:
|
||||
logger.warning(f"{model} 不支持函数调用,已自动去除")
|
||||
logger.warning(f"{self.get_model()} 不支持函数调用,已自动去除")
|
||||
tools = None
|
||||
else:
|
||||
raise
|
||||
@@ -651,9 +569,6 @@ class ProviderGoogleGenAI(Provider):
|
||||
chunk.candidates[0],
|
||||
llm_response,
|
||||
)
|
||||
llm_response.id = chunk.response_id
|
||||
if chunk.usage_metadata:
|
||||
llm_response.usage = self._extract_usage(chunk.usage_metadata)
|
||||
yield llm_response
|
||||
return
|
||||
|
||||
@@ -681,9 +596,6 @@ class ProviderGoogleGenAI(Provider):
|
||||
chunk.candidates[0],
|
||||
final_response,
|
||||
)
|
||||
final_response.id = chunk.response_id
|
||||
if chunk.usage_metadata:
|
||||
final_response.usage = self._extract_usage(chunk.usage_metadata)
|
||||
break
|
||||
|
||||
# Yield final complete response with accumulated text
|
||||
@@ -715,16 +627,13 @@ class ProviderGoogleGenAI(Provider):
|
||||
system_prompt=None,
|
||||
tool_calls_result=None,
|
||||
model=None,
|
||||
extra_user_content_parts=None,
|
||||
**kwargs,
|
||||
) -> LLMResponse:
|
||||
if contexts is None:
|
||||
contexts = []
|
||||
new_record = None
|
||||
if prompt is not None:
|
||||
new_record = await self.assemble_context(
|
||||
prompt, image_urls, extra_user_content_parts
|
||||
)
|
||||
new_record = await self.assemble_context(prompt, image_urls)
|
||||
context_query = self._ensure_message_to_dicts(contexts)
|
||||
if new_record:
|
||||
context_query.append(new_record)
|
||||
@@ -743,9 +652,10 @@ class ProviderGoogleGenAI(Provider):
|
||||
for tcr in tool_calls_result:
|
||||
context_query.extend(tcr.to_openai_messages())
|
||||
|
||||
model = model or self.get_model()
|
||||
model_config = self.provider_config.get("model_config", {})
|
||||
model_config["model"] = model or self.get_model()
|
||||
|
||||
payloads = {"messages": context_query, "model": model}
|
||||
payloads = {"messages": context_query, **model_config}
|
||||
|
||||
retry = 10
|
||||
keys = self.api_keys.copy()
|
||||
@@ -770,16 +680,13 @@ class ProviderGoogleGenAI(Provider):
|
||||
system_prompt=None,
|
||||
tool_calls_result=None,
|
||||
model=None,
|
||||
extra_user_content_parts=None,
|
||||
**kwargs,
|
||||
) -> AsyncGenerator[LLMResponse, None]:
|
||||
if contexts is None:
|
||||
contexts = []
|
||||
new_record = None
|
||||
if prompt is not None:
|
||||
new_record = await self.assemble_context(
|
||||
prompt, image_urls, extra_user_content_parts
|
||||
)
|
||||
new_record = await self.assemble_context(prompt, image_urls)
|
||||
context_query = self._ensure_message_to_dicts(contexts)
|
||||
if new_record:
|
||||
context_query.append(new_record)
|
||||
@@ -798,9 +705,10 @@ class ProviderGoogleGenAI(Provider):
|
||||
for tcr in tool_calls_result:
|
||||
context_query.extend(tcr.to_openai_messages())
|
||||
|
||||
model = model or self.get_model()
|
||||
model_config = self.provider_config.get("model_config", {})
|
||||
model_config["model"] = model or self.get_model()
|
||||
|
||||
payloads = {"messages": context_query, "model": model}
|
||||
payloads = {"messages": context_query, **model_config}
|
||||
|
||||
retry = 10
|
||||
keys = self.api_keys.copy()
|
||||
@@ -838,75 +746,33 @@ class ProviderGoogleGenAI(Provider):
|
||||
self.chosen_api_key = key
|
||||
self._init_client()
|
||||
|
||||
async def assemble_context(
|
||||
self,
|
||||
text: str,
|
||||
image_urls: list[str] | None = None,
|
||||
extra_user_content_parts: list[ContentPart] | None = None,
|
||||
):
|
||||
async def assemble_context(self, text: str, image_urls: list[str] | None = None):
|
||||
"""组装上下文。"""
|
||||
|
||||
async def resolve_image_part(image_url: str) -> dict | None:
|
||||
if image_url.startswith("http"):
|
||||
image_path = await download_image_by_url(image_url)
|
||||
image_data = await self.encode_image_bs64(image_path)
|
||||
elif image_url.startswith("file:///"):
|
||||
image_path = image_url.replace("file:///", "")
|
||||
image_data = await self.encode_image_bs64(image_path)
|
||||
else:
|
||||
image_data = await self.encode_image_bs64(image_url)
|
||||
if not image_data:
|
||||
logger.warning(f"图片 {image_url} 得到的结果为空,将忽略。")
|
||||
return None
|
||||
return {
|
||||
"type": "image_url",
|
||||
"image_url": {"url": image_data},
|
||||
}
|
||||
|
||||
# 构建内容块列表
|
||||
content_blocks = []
|
||||
|
||||
# 1. 用户原始发言(OpenAI 建议:用户发言在前)
|
||||
if text:
|
||||
content_blocks.append({"type": "text", "text": text})
|
||||
elif image_urls:
|
||||
# 如果没有文本但有图片,添加占位文本
|
||||
content_blocks.append({"type": "text", "text": "[图片]"})
|
||||
elif extra_user_content_parts:
|
||||
# 如果只有额外内容块,也需要添加占位文本
|
||||
content_blocks.append({"type": "text", "text": " "})
|
||||
|
||||
# 2. 额外的内容块(系统提醒、指令等)
|
||||
if extra_user_content_parts:
|
||||
for part in extra_user_content_parts:
|
||||
if isinstance(part, TextPart):
|
||||
content_blocks.append({"type": "text", "text": part.text})
|
||||
elif isinstance(part, ImageURLPart):
|
||||
image_part = await resolve_image_part(part.image_url.url)
|
||||
if image_part:
|
||||
content_blocks.append(image_part)
|
||||
else:
|
||||
raise ValueError(f"不支持的额外内容块类型: {type(part)}")
|
||||
|
||||
# 3. 图片内容
|
||||
if image_urls:
|
||||
user_content = {
|
||||
"role": "user",
|
||||
"content": [{"type": "text", "text": text if text else "[图片]"}],
|
||||
}
|
||||
for image_url in image_urls:
|
||||
image_part = await resolve_image_part(image_url)
|
||||
if image_part:
|
||||
content_blocks.append(image_part)
|
||||
|
||||
# 如果只有主文本且没有额外内容块和图片,返回简单格式以保持向后兼容
|
||||
if (
|
||||
text
|
||||
and not extra_user_content_parts
|
||||
and not image_urls
|
||||
and len(content_blocks) == 1
|
||||
and content_blocks[0]["type"] == "text"
|
||||
):
|
||||
return {"role": "user", "content": content_blocks[0]["text"]}
|
||||
|
||||
# 否则返回多模态格式
|
||||
return {"role": "user", "content": content_blocks}
|
||||
if image_url.startswith("http"):
|
||||
image_path = await download_image_by_url(image_url)
|
||||
image_data = await self.encode_image_bs64(image_path)
|
||||
elif image_url.startswith("file:///"):
|
||||
image_path = image_url.replace("file:///", "")
|
||||
image_data = await self.encode_image_bs64(image_path)
|
||||
else:
|
||||
image_data = await self.encode_image_bs64(image_url)
|
||||
if not image_data:
|
||||
logger.warning(f"图片 {image_url} 得到的结果为空,将忽略。")
|
||||
continue
|
||||
user_content["content"].append(
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": image_data},
|
||||
},
|
||||
)
|
||||
return user_content
|
||||
return {"role": "user", "content": text}
|
||||
|
||||
async def encode_image_bs64(self, image_url: str) -> str:
|
||||
"""将图片转换为 base64"""
|
||||
|
||||
@@ -51,7 +51,7 @@ class ProviderMiniMaxTTSAPI(TTSProvider):
|
||||
"voice_id": ""
|
||||
if self.is_timber_weight
|
||||
else provider_config.get("minimax-voice-id", ""),
|
||||
"emotion": provider_config.get("minimax-voice-emotion", "auto"),
|
||||
"emotion": provider_config.get("minimax-voice-emotion", "neutral"),
|
||||
"latex_read": provider_config.get("minimax-voice-latex", False),
|
||||
"english_normalization": provider_config.get(
|
||||
"minimax-voice-english-normalization",
|
||||
@@ -59,9 +59,6 @@ class ProviderMiniMaxTTSAPI(TTSProvider):
|
||||
),
|
||||
}
|
||||
|
||||
if self.voice_setting["emotion"] == "auto":
|
||||
self.voice_setting.pop("emotion", None)
|
||||
|
||||
self.audio_setting: dict = {
|
||||
"sample_rate": 32000,
|
||||
"bitrate": 128000,
|
||||
|
||||
@@ -12,15 +12,14 @@ from openai._exceptions import NotFoundError
|
||||
from openai.lib.streaming.chat._completions import ChatCompletionStreamState
|
||||
from openai.types.chat.chat_completion import ChatCompletion
|
||||
from openai.types.chat.chat_completion_chunk import ChatCompletionChunk
|
||||
from openai.types.completion_usage import CompletionUsage
|
||||
|
||||
import astrbot.core.message.components as Comp
|
||||
from astrbot import logger
|
||||
from astrbot.api.provider import Provider
|
||||
from astrbot.core.agent.message import ContentPart, ImageURLPart, Message, TextPart
|
||||
from astrbot.core.agent.message import Message
|
||||
from astrbot.core.agent.tool import ToolSet
|
||||
from astrbot.core.message.message_event_result import MessageChain
|
||||
from astrbot.core.provider.entities import LLMResponse, TokenUsage, ToolCallsResult
|
||||
from astrbot.core.provider.entities import LLMResponse, ToolCallsResult
|
||||
from astrbot.core.utils.io import download_image_by_url
|
||||
|
||||
from ..register import register_provider_adapter
|
||||
@@ -69,11 +68,34 @@ class ProviderOpenAIOfficial(Provider):
|
||||
self.client.chat.completions.create,
|
||||
).parameters.keys()
|
||||
|
||||
model = provider_config.get("model", "unknown")
|
||||
model_config = provider_config.get("model_config", {})
|
||||
model = model_config.get("model", "unknown")
|
||||
self.set_model(model)
|
||||
|
||||
self.reasoning_key = "reasoning_content"
|
||||
|
||||
def _maybe_inject_xai_search(self, payloads: dict, **kwargs):
|
||||
"""当开启 xAI 原生搜索时,向请求体注入 Live Search 参数。
|
||||
|
||||
- 仅在 provider_config.xai_native_search 为 True 时生效
|
||||
- 默认注入 {"mode": "auto"}
|
||||
- 允许通过 kwargs 使用 xai_search_mode 覆盖(on/auto/off)
|
||||
"""
|
||||
if not bool(self.provider_config.get("xai_native_search", False)):
|
||||
return
|
||||
|
||||
mode = kwargs.get("xai_search_mode", "auto")
|
||||
mode = str(mode).lower()
|
||||
if mode not in ("auto", "on", "off"):
|
||||
mode = "auto"
|
||||
|
||||
# off 时不注入,保持与未开启一致
|
||||
if mode == "off":
|
||||
return
|
||||
|
||||
# OpenAI SDK 不识别的字段会在 _query/_query_stream 中放入 extra_body
|
||||
payloads["search_parameters"] = {"mode": mode}
|
||||
|
||||
async def get_models(self):
|
||||
try:
|
||||
models_str = []
|
||||
@@ -112,6 +134,10 @@ class ProviderOpenAIOfficial(Provider):
|
||||
|
||||
model = payloads.get("model", "").lower()
|
||||
|
||||
# 针对 deepseek 模型的特殊处理:deepseek-reasoner调用必须移除 tools ,否则将被切换至 deepseek-chat
|
||||
if model == "deepseek-reasoner" and "tools" in payloads:
|
||||
del payloads["tools"]
|
||||
|
||||
completion = await self.client.chat.completions.create(
|
||||
**payloads,
|
||||
stream=False,
|
||||
@@ -182,7 +208,6 @@ class ProviderOpenAIOfficial(Provider):
|
||||
# handle the content delta
|
||||
reasoning = self._extract_reasoning_content(chunk)
|
||||
_y = False
|
||||
llm_response.id = chunk.id
|
||||
if reasoning:
|
||||
llm_response.reasoning_content = reasoning
|
||||
_y = True
|
||||
@@ -192,8 +217,6 @@ class ProviderOpenAIOfficial(Provider):
|
||||
chain=[Comp.Plain(completion_text)],
|
||||
)
|
||||
_y = True
|
||||
if chunk.usage:
|
||||
llm_response.usage = self._extract_usage(chunk.usage)
|
||||
if _y:
|
||||
yield llm_response
|
||||
|
||||
@@ -222,19 +245,6 @@ class ProviderOpenAIOfficial(Provider):
|
||||
reasoning_text = str(reasoning_attr)
|
||||
return reasoning_text
|
||||
|
||||
def _extract_usage(self, usage: CompletionUsage) -> TokenUsage:
|
||||
ptd = usage.prompt_tokens_details
|
||||
cached = ptd.cached_tokens if ptd and ptd.cached_tokens else 0
|
||||
prompt_tokens = 0 if usage.prompt_tokens is None else usage.prompt_tokens
|
||||
completion_tokens = (
|
||||
0 if usage.completion_tokens is None else usage.completion_tokens
|
||||
)
|
||||
return TokenUsage(
|
||||
input_other=prompt_tokens - cached,
|
||||
input_cached=cached,
|
||||
output=completion_tokens,
|
||||
)
|
||||
|
||||
async def _parse_openai_completion(
|
||||
self, completion: ChatCompletion, tools: ToolSet | None
|
||||
) -> LLMResponse:
|
||||
@@ -311,10 +321,6 @@ class ProviderOpenAIOfficial(Provider):
|
||||
raise Exception(f"API 返回的 completion 无法解析:{completion}。")
|
||||
|
||||
llm_response.raw_completion = completion
|
||||
llm_response.id = completion.id
|
||||
|
||||
if completion.usage:
|
||||
llm_response.usage = self._extract_usage(completion.usage)
|
||||
|
||||
return llm_response
|
||||
|
||||
@@ -326,7 +332,6 @@ class ProviderOpenAIOfficial(Provider):
|
||||
system_prompt: str | None = None,
|
||||
tool_calls_result: ToolCallsResult | list[ToolCallsResult] | None = None,
|
||||
model: str | None = None,
|
||||
extra_user_content_parts: list[ContentPart] | None = None,
|
||||
**kwargs,
|
||||
) -> tuple:
|
||||
"""准备聊天所需的有效载荷和上下文"""
|
||||
@@ -334,9 +339,7 @@ class ProviderOpenAIOfficial(Provider):
|
||||
contexts = []
|
||||
new_record = None
|
||||
if prompt is not None:
|
||||
new_record = await self.assemble_context(
|
||||
prompt, image_urls, extra_user_content_parts
|
||||
)
|
||||
new_record = await self.assemble_context(prompt, image_urls)
|
||||
context_query = self._ensure_message_to_dicts(contexts)
|
||||
if new_record:
|
||||
context_query.append(new_record)
|
||||
@@ -355,32 +358,16 @@ class ProviderOpenAIOfficial(Provider):
|
||||
for tcr in tool_calls_result:
|
||||
context_query.extend(tcr.to_openai_messages())
|
||||
|
||||
model = model or self.get_model()
|
||||
model_config = self.provider_config.get("model_config", {})
|
||||
model_config["model"] = model or self.get_model()
|
||||
|
||||
payloads = {"messages": context_query, "model": model}
|
||||
payloads = {"messages": context_query, **model_config}
|
||||
|
||||
self._finally_convert_payload(payloads)
|
||||
# xAI origin search tool inject
|
||||
self._maybe_inject_xai_search(payloads, **kwargs)
|
||||
|
||||
return payloads, context_query
|
||||
|
||||
def _finally_convert_payload(self, payloads: dict):
|
||||
"""Finally convert the payload. Such as think part conversion, tool inject."""
|
||||
for message in payloads.get("messages", []):
|
||||
if message.get("role") == "assistant" and isinstance(
|
||||
message.get("content"), list
|
||||
):
|
||||
reasoning_content = ""
|
||||
new_content = [] # not including think part
|
||||
for part in message["content"]:
|
||||
if part.get("type") == "think":
|
||||
reasoning_content += str(part.get("think"))
|
||||
else:
|
||||
new_content.append(part)
|
||||
message["content"] = new_content
|
||||
# reasoning key is "reasoning_content"
|
||||
if reasoning_content:
|
||||
message["reasoning_content"] = reasoning_content
|
||||
|
||||
async def _handle_api_error(
|
||||
self,
|
||||
e: Exception,
|
||||
@@ -474,7 +461,6 @@ class ProviderOpenAIOfficial(Provider):
|
||||
system_prompt=None,
|
||||
tool_calls_result=None,
|
||||
model=None,
|
||||
extra_user_content_parts=None,
|
||||
**kwargs,
|
||||
) -> LLMResponse:
|
||||
payloads, context_query = await self._prepare_chat_payload(
|
||||
@@ -484,7 +470,6 @@ class ProviderOpenAIOfficial(Provider):
|
||||
system_prompt,
|
||||
tool_calls_result,
|
||||
model=model,
|
||||
extra_user_content_parts=extra_user_content_parts,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
@@ -624,71 +609,33 @@ class ProviderOpenAIOfficial(Provider):
|
||||
self,
|
||||
text: str,
|
||||
image_urls: list[str] | None = None,
|
||||
extra_user_content_parts: list[ContentPart] | None = None,
|
||||
) -> dict:
|
||||
"""组装成符合 OpenAI 格式的 role 为 user 的消息段"""
|
||||
|
||||
async def resolve_image_part(image_url: str) -> dict | None:
|
||||
if image_url.startswith("http"):
|
||||
image_path = await download_image_by_url(image_url)
|
||||
image_data = await self.encode_image_bs64(image_path)
|
||||
elif image_url.startswith("file:///"):
|
||||
image_path = image_url.replace("file:///", "")
|
||||
image_data = await self.encode_image_bs64(image_path)
|
||||
else:
|
||||
image_data = await self.encode_image_bs64(image_url)
|
||||
if not image_data:
|
||||
logger.warning(f"图片 {image_url} 得到的结果为空,将忽略。")
|
||||
return None
|
||||
return {
|
||||
"type": "image_url",
|
||||
"image_url": {"url": image_data},
|
||||
}
|
||||
|
||||
# 构建内容块列表
|
||||
content_blocks = []
|
||||
|
||||
# 1. 用户原始发言(OpenAI 建议:用户发言在前)
|
||||
if text:
|
||||
content_blocks.append({"type": "text", "text": text})
|
||||
elif image_urls:
|
||||
# 如果没有文本但有图片,添加占位文本
|
||||
content_blocks.append({"type": "text", "text": "[图片]"})
|
||||
elif extra_user_content_parts:
|
||||
# 如果只有额外内容块,也需要添加占位文本
|
||||
content_blocks.append({"type": "text", "text": " "})
|
||||
|
||||
# 2. 额外的内容块(系统提醒、指令等)
|
||||
if extra_user_content_parts:
|
||||
for part in extra_user_content_parts:
|
||||
if isinstance(part, TextPart):
|
||||
content_blocks.append({"type": "text", "text": part.text})
|
||||
elif isinstance(part, ImageURLPart):
|
||||
image_part = await resolve_image_part(part.image_url.url)
|
||||
if image_part:
|
||||
content_blocks.append(image_part)
|
||||
else:
|
||||
raise ValueError(f"不支持的额外内容块类型: {type(part)}")
|
||||
|
||||
# 3. 图片内容
|
||||
if image_urls:
|
||||
user_content = {
|
||||
"role": "user",
|
||||
"content": [{"type": "text", "text": text if text else "[图片]"}],
|
||||
}
|
||||
for image_url in image_urls:
|
||||
image_part = await resolve_image_part(image_url)
|
||||
if image_part:
|
||||
content_blocks.append(image_part)
|
||||
|
||||
# 如果只有主文本且没有额外内容块和图片,返回简单格式以保持向后兼容
|
||||
if (
|
||||
text
|
||||
and not extra_user_content_parts
|
||||
and not image_urls
|
||||
and len(content_blocks) == 1
|
||||
and content_blocks[0]["type"] == "text"
|
||||
):
|
||||
return {"role": "user", "content": content_blocks[0]["text"]}
|
||||
|
||||
# 否则返回多模态格式
|
||||
return {"role": "user", "content": content_blocks}
|
||||
if image_url.startswith("http"):
|
||||
image_path = await download_image_by_url(image_url)
|
||||
image_data = await self.encode_image_bs64(image_path)
|
||||
elif image_url.startswith("file:///"):
|
||||
image_path = image_url.replace("file:///", "")
|
||||
image_data = await self.encode_image_bs64(image_path)
|
||||
else:
|
||||
image_data = await self.encode_image_bs64(image_url)
|
||||
if not image_data:
|
||||
logger.warning(f"图片 {image_url} 得到的结果为空,将忽略。")
|
||||
continue
|
||||
user_content["content"].append(
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": image_data},
|
||||
},
|
||||
)
|
||||
return user_content
|
||||
return {"role": "user", "content": text}
|
||||
|
||||
async def encode_image_bs64(self, image_url: str) -> str:
|
||||
"""将图片转换为 base64"""
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
from ..register import register_provider_adapter
|
||||
from .openai_source import ProviderOpenAIOfficial
|
||||
|
||||
|
||||
@register_provider_adapter(
|
||||
"xai_chat_completion", "xAI Chat Completion Provider Adapter"
|
||||
)
|
||||
class ProviderXAI(ProviderOpenAIOfficial):
|
||||
def __init__(
|
||||
self,
|
||||
provider_config: dict,
|
||||
provider_settings: dict,
|
||||
) -> None:
|
||||
super().__init__(provider_config, provider_settings)
|
||||
|
||||
def _maybe_inject_xai_search(self, payloads: dict):
|
||||
"""当开启 xAI 原生搜索时,向请求体注入 Live Search 参数。
|
||||
|
||||
- 仅在 provider_config.xai_native_search 为 True 时生效
|
||||
- 默认注入 {"mode": "auto"}
|
||||
"""
|
||||
if not bool(self.provider_config.get("xai_native_search", False)):
|
||||
return
|
||||
# OpenAI SDK 不识别的字段会在 _query/_query_stream 中放入 extra_body
|
||||
payloads["search_parameters"] = {"mode": "auto"}
|
||||
|
||||
def _finally_convert_payload(self, payloads: dict):
|
||||
self._maybe_inject_xai_search(payloads)
|
||||
super()._finally_convert_payload(payloads)
|
||||
@@ -8,10 +8,7 @@ from xinference_client.client.restful.async_restful_client import (
|
||||
|
||||
from astrbot.core import logger
|
||||
from astrbot.core.utils.astrbot_path import get_astrbot_data_path
|
||||
from astrbot.core.utils.tencent_record_helper import (
|
||||
convert_to_pcm_wav,
|
||||
tencent_silk_to_wav,
|
||||
)
|
||||
from astrbot.core.utils.tencent_record_helper import tencent_silk_to_wav
|
||||
|
||||
from ..entities import ProviderType
|
||||
from ..provider import STTProvider
|
||||
@@ -114,22 +111,17 @@ class ProviderXinferenceSTT(STTProvider):
|
||||
return ""
|
||||
|
||||
# 2. Check for conversion
|
||||
conversion_type = None
|
||||
|
||||
if b"SILK" in audio_bytes[:8]:
|
||||
conversion_type = "silk"
|
||||
elif b"#!AMR" in audio_bytes[:6]:
|
||||
conversion_type = "amr"
|
||||
elif audio_url.endswith(".silk") or is_tencent:
|
||||
conversion_type = "silk"
|
||||
elif audio_url.endswith(".amr"):
|
||||
conversion_type = "amr"
|
||||
needs_conversion = False
|
||||
if (
|
||||
audio_url.endswith((".amr", ".silk"))
|
||||
or is_tencent
|
||||
or b"SILK" in audio_bytes[:8]
|
||||
):
|
||||
needs_conversion = True
|
||||
|
||||
# 3. Perform conversion if needed
|
||||
if conversion_type:
|
||||
logger.info(
|
||||
f"Audio requires conversion ({conversion_type}), using temporary files..."
|
||||
)
|
||||
if needs_conversion:
|
||||
logger.info("Audio requires conversion, using temporary files...")
|
||||
temp_dir = os.path.join(get_astrbot_data_path(), "temp")
|
||||
os.makedirs(temp_dir, exist_ok=True)
|
||||
|
||||
@@ -140,12 +132,8 @@ class ProviderXinferenceSTT(STTProvider):
|
||||
with open(input_path, "wb") as f:
|
||||
f.write(audio_bytes)
|
||||
|
||||
if conversion_type == "silk":
|
||||
logger.info("Converting silk to wav ...")
|
||||
await tencent_silk_to_wav(input_path, output_path)
|
||||
elif conversion_type == "amr":
|
||||
logger.info("Converting amr to wav ...")
|
||||
await convert_to_pcm_wav(input_path, output_path)
|
||||
logger.info("Converting silk/amr file to wav ...")
|
||||
await tencent_silk_to_wav(input_path, output_path)
|
||||
|
||||
with open(output_path, "rb") as f:
|
||||
audio_bytes = f.read()
|
||||
|
||||
@@ -2,19 +2,15 @@ from astrbot.core import html_renderer
|
||||
from astrbot.core.provider import Provider
|
||||
from astrbot.core.star.star_tools import StarTools
|
||||
from astrbot.core.utils.command_parser import CommandParserMixin
|
||||
from astrbot.core.utils.plugin_kv_store import PluginKVStoreMixin
|
||||
|
||||
from .context import Context
|
||||
from .star import StarMetadata, star_map, star_registry
|
||||
from .star_manager import PluginManager
|
||||
|
||||
|
||||
class Star(CommandParserMixin, PluginKVStoreMixin):
|
||||
class Star(CommandParserMixin):
|
||||
"""所有插件(Star)的父类,所有插件都应该继承于这个类"""
|
||||
|
||||
author: str
|
||||
name: str
|
||||
|
||||
def __init__(self, context: Context, config: dict | None = None):
|
||||
StarTools.initialize(context)
|
||||
self.context = context
|
||||
|
||||
@@ -1,496 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import defaultdict
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from astrbot.core import db_helper, logger
|
||||
from astrbot.core.db.po import CommandConfig
|
||||
from astrbot.core.star.filter.command import CommandFilter
|
||||
from astrbot.core.star.filter.command_group import CommandGroupFilter
|
||||
from astrbot.core.star.filter.permission import PermissionType, PermissionTypeFilter
|
||||
from astrbot.core.star.star import star_map
|
||||
from astrbot.core.star.star_handler import StarHandlerMetadata, star_handlers_registry
|
||||
|
||||
|
||||
@dataclass
|
||||
class CommandDescriptor:
|
||||
handler: StarHandlerMetadata = field(repr=False)
|
||||
filter_ref: CommandFilter | CommandGroupFilter | None = field(
|
||||
default=None,
|
||||
repr=False,
|
||||
)
|
||||
handler_full_name: str = ""
|
||||
handler_name: str = ""
|
||||
plugin_name: str = ""
|
||||
plugin_display_name: str | None = None
|
||||
module_path: str = ""
|
||||
description: str = ""
|
||||
command_type: str = "command" # "command" | "group" | "sub_command"
|
||||
raw_command_name: str | None = None
|
||||
current_fragment: str | None = None
|
||||
parent_signature: str = ""
|
||||
parent_group_handler: str = ""
|
||||
original_command: str | None = None
|
||||
effective_command: str | None = None
|
||||
aliases: list[str] = field(default_factory=list)
|
||||
permission: str = "everyone"
|
||||
enabled: bool = True
|
||||
is_group: bool = False
|
||||
is_sub_command: bool = False
|
||||
reserved: bool = False
|
||||
config: CommandConfig | None = None
|
||||
has_conflict: bool = False
|
||||
sub_commands: list[CommandDescriptor] = field(default_factory=list)
|
||||
|
||||
|
||||
async def sync_command_configs() -> None:
|
||||
"""同步指令配置,清理过期配置。"""
|
||||
descriptors = _collect_descriptors(include_sub_commands=False)
|
||||
config_records = await db_helper.get_command_configs()
|
||||
config_map = _bind_configs_to_descriptors(descriptors, config_records)
|
||||
live_handlers = {desc.handler_full_name for desc in descriptors}
|
||||
|
||||
stale_configs = [key for key in config_map if key not in live_handlers]
|
||||
if stale_configs:
|
||||
await db_helper.delete_command_configs(stale_configs)
|
||||
|
||||
|
||||
async def toggle_command(handler_full_name: str, enabled: bool) -> CommandDescriptor:
|
||||
descriptor = _build_descriptor_by_full_name(handler_full_name)
|
||||
if not descriptor:
|
||||
raise ValueError("指定的处理函数不存在或不是指令。")
|
||||
|
||||
existing_cfg = await db_helper.get_command_config(handler_full_name)
|
||||
config = await db_helper.upsert_command_config(
|
||||
handler_full_name=handler_full_name,
|
||||
plugin_name=descriptor.plugin_name or "",
|
||||
module_path=descriptor.module_path,
|
||||
original_command=descriptor.original_command or descriptor.handler_name,
|
||||
resolved_command=(
|
||||
existing_cfg.resolved_command
|
||||
if existing_cfg
|
||||
else descriptor.current_fragment
|
||||
),
|
||||
enabled=enabled,
|
||||
keep_original_alias=False,
|
||||
conflict_key=existing_cfg.conflict_key
|
||||
if existing_cfg and existing_cfg.conflict_key
|
||||
else descriptor.original_command,
|
||||
resolution_strategy=existing_cfg.resolution_strategy if existing_cfg else None,
|
||||
note=existing_cfg.note if existing_cfg else None,
|
||||
extra_data=existing_cfg.extra_data if existing_cfg else None,
|
||||
auto_managed=False,
|
||||
)
|
||||
_bind_descriptor_with_config(descriptor, config)
|
||||
await sync_command_configs()
|
||||
return descriptor
|
||||
|
||||
|
||||
async def rename_command(
|
||||
handler_full_name: str,
|
||||
new_fragment: str,
|
||||
aliases: list[str] | None = None,
|
||||
) -> CommandDescriptor:
|
||||
descriptor = _build_descriptor_by_full_name(handler_full_name)
|
||||
if not descriptor:
|
||||
raise ValueError("指定的处理函数不存在或不是指令。")
|
||||
|
||||
new_fragment = new_fragment.strip()
|
||||
if not new_fragment:
|
||||
raise ValueError("指令名不能为空。")
|
||||
|
||||
# 校验主指令名
|
||||
candidate_full = _compose_command(descriptor.parent_signature, new_fragment)
|
||||
if _is_command_in_use(handler_full_name, candidate_full):
|
||||
raise ValueError(f"指令名 '{candidate_full}' 已被其他指令占用。")
|
||||
|
||||
# 校验别名
|
||||
if aliases:
|
||||
for alias in aliases:
|
||||
alias = alias.strip()
|
||||
if not alias:
|
||||
continue
|
||||
alias_full = _compose_command(descriptor.parent_signature, alias)
|
||||
if _is_command_in_use(handler_full_name, alias_full):
|
||||
raise ValueError(f"别名 '{alias_full}' 已被其他指令占用。")
|
||||
|
||||
existing_cfg = await db_helper.get_command_config(handler_full_name)
|
||||
merged_extra = dict(existing_cfg.extra_data or {}) if existing_cfg else {}
|
||||
merged_extra["resolved_aliases"] = aliases or []
|
||||
|
||||
config = await db_helper.upsert_command_config(
|
||||
handler_full_name=handler_full_name,
|
||||
plugin_name=descriptor.plugin_name or "",
|
||||
module_path=descriptor.module_path,
|
||||
original_command=descriptor.original_command or descriptor.handler_name,
|
||||
resolved_command=new_fragment,
|
||||
enabled=True if descriptor.enabled else False,
|
||||
keep_original_alias=False,
|
||||
conflict_key=descriptor.original_command,
|
||||
resolution_strategy="manual_rename",
|
||||
note=None,
|
||||
extra_data=merged_extra,
|
||||
auto_managed=False,
|
||||
)
|
||||
_bind_descriptor_with_config(descriptor, config)
|
||||
|
||||
await sync_command_configs()
|
||||
return descriptor
|
||||
|
||||
|
||||
async def list_commands() -> list[dict[str, Any]]:
|
||||
descriptors = _collect_descriptors(include_sub_commands=True)
|
||||
config_records = await db_helper.get_command_configs()
|
||||
_bind_configs_to_descriptors(descriptors, config_records)
|
||||
|
||||
conflict_groups = _group_conflicts(descriptors)
|
||||
conflict_handler_names: set[str] = {
|
||||
d.handler_full_name for group in conflict_groups.values() for d in group
|
||||
}
|
||||
|
||||
# 分类,设置冲突标志,将子指令挂载到父指令组
|
||||
group_map: dict[str, CommandDescriptor] = {}
|
||||
sub_commands: list[CommandDescriptor] = []
|
||||
root_commands: list[CommandDescriptor] = []
|
||||
|
||||
for desc in descriptors:
|
||||
desc.has_conflict = desc.handler_full_name in conflict_handler_names
|
||||
if desc.is_group:
|
||||
group_map[desc.handler_full_name] = desc
|
||||
elif desc.is_sub_command:
|
||||
sub_commands.append(desc)
|
||||
else:
|
||||
root_commands.append(desc)
|
||||
|
||||
for sub in sub_commands:
|
||||
if sub.parent_group_handler and sub.parent_group_handler in group_map:
|
||||
group_map[sub.parent_group_handler].sub_commands.append(sub)
|
||||
else:
|
||||
root_commands.append(sub)
|
||||
|
||||
# 指令组 + 普通指令,按 effective_command 字母排序
|
||||
all_commands = list(group_map.values()) + root_commands
|
||||
all_commands.sort(key=lambda d: (d.effective_command or "").lower())
|
||||
|
||||
result = [_descriptor_to_dict(desc) for desc in all_commands]
|
||||
return result
|
||||
|
||||
|
||||
async def list_command_conflicts() -> list[dict[str, Any]]:
|
||||
"""列出所有冲突的指令组。"""
|
||||
descriptors = _collect_descriptors(include_sub_commands=False)
|
||||
config_records = await db_helper.get_command_configs()
|
||||
_bind_configs_to_descriptors(descriptors, config_records)
|
||||
|
||||
conflict_groups = _group_conflicts(descriptors)
|
||||
details = [
|
||||
{
|
||||
"conflict_key": key,
|
||||
"handlers": [
|
||||
{
|
||||
"handler_full_name": item.handler_full_name,
|
||||
"plugin": item.plugin_name,
|
||||
"current_name": item.effective_command,
|
||||
}
|
||||
for item in group
|
||||
],
|
||||
}
|
||||
for key, group in conflict_groups.items()
|
||||
]
|
||||
return details
|
||||
|
||||
|
||||
# Internal helpers ----------------------------------------------------------
|
||||
|
||||
|
||||
def _collect_descriptors(include_sub_commands: bool) -> list[CommandDescriptor]:
|
||||
"""收集指令,按需包含子指令。"""
|
||||
descriptors: list[CommandDescriptor] = []
|
||||
for handler in star_handlers_registry:
|
||||
try:
|
||||
desc = _build_descriptor(handler)
|
||||
if not desc:
|
||||
continue
|
||||
if not include_sub_commands and desc.is_sub_command:
|
||||
continue
|
||||
descriptors.append(desc)
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"解析指令处理函数 {handler.handler_full_name} 失败,跳过该指令。原因: {e!s}"
|
||||
)
|
||||
continue
|
||||
return descriptors
|
||||
|
||||
|
||||
def _build_descriptor(handler: StarHandlerMetadata) -> CommandDescriptor | None:
|
||||
filter_ref = _locate_primary_filter(handler)
|
||||
if filter_ref is None:
|
||||
return None
|
||||
|
||||
plugin_meta = star_map.get(handler.handler_module_path)
|
||||
plugin_name = (
|
||||
plugin_meta.name if plugin_meta else None
|
||||
) or handler.handler_module_path
|
||||
plugin_display = plugin_meta.display_name if plugin_meta else None
|
||||
|
||||
is_sub_command = bool(handler.extras_configs.get("sub_command"))
|
||||
parent_group_handler = ""
|
||||
|
||||
if isinstance(filter_ref, CommandFilter):
|
||||
raw_fragment = getattr(
|
||||
filter_ref, "_original_command_name", filter_ref.command_name
|
||||
)
|
||||
current_fragment = filter_ref.command_name
|
||||
parent_signature = (filter_ref.parent_command_names or [""])[0].strip()
|
||||
# 如果是子指令,尝试找到父指令组的 handler_full_name
|
||||
if is_sub_command and parent_signature:
|
||||
parent_group_handler = _find_parent_group_handler(
|
||||
handler.handler_module_path, parent_signature
|
||||
)
|
||||
else:
|
||||
raw_fragment = getattr(
|
||||
filter_ref, "_original_group_name", filter_ref.group_name
|
||||
)
|
||||
current_fragment = filter_ref.group_name
|
||||
parent_signature = _resolve_group_parent_signature(filter_ref)
|
||||
|
||||
original_command = _compose_command(parent_signature, raw_fragment)
|
||||
effective_command = _compose_command(parent_signature, current_fragment)
|
||||
|
||||
# 确定 command_type
|
||||
if isinstance(filter_ref, CommandGroupFilter):
|
||||
command_type = "group"
|
||||
elif is_sub_command:
|
||||
command_type = "sub_command"
|
||||
else:
|
||||
command_type = "command"
|
||||
|
||||
descriptor = CommandDescriptor(
|
||||
handler=handler,
|
||||
filter_ref=filter_ref,
|
||||
handler_full_name=handler.handler_full_name,
|
||||
handler_name=handler.handler_name,
|
||||
plugin_name=plugin_name,
|
||||
plugin_display_name=plugin_display,
|
||||
module_path=handler.handler_module_path,
|
||||
description=handler.desc or "",
|
||||
command_type=command_type,
|
||||
raw_command_name=raw_fragment,
|
||||
current_fragment=current_fragment,
|
||||
parent_signature=parent_signature,
|
||||
parent_group_handler=parent_group_handler,
|
||||
original_command=original_command,
|
||||
effective_command=effective_command,
|
||||
aliases=sorted(getattr(filter_ref, "alias", set())),
|
||||
permission=_determine_permission(handler),
|
||||
enabled=handler.enabled,
|
||||
is_group=isinstance(filter_ref, CommandGroupFilter),
|
||||
is_sub_command=is_sub_command,
|
||||
reserved=plugin_meta.reserved if plugin_meta else False,
|
||||
)
|
||||
return descriptor
|
||||
|
||||
|
||||
def _build_descriptor_by_full_name(full_name: str) -> CommandDescriptor | None:
|
||||
handler = star_handlers_registry.get_handler_by_full_name(full_name)
|
||||
if not handler:
|
||||
return None
|
||||
return _build_descriptor(handler)
|
||||
|
||||
|
||||
def _locate_primary_filter(
|
||||
handler: StarHandlerMetadata,
|
||||
) -> CommandFilter | CommandGroupFilter | None:
|
||||
for filter_ref in handler.event_filters:
|
||||
if isinstance(filter_ref, (CommandFilter, CommandGroupFilter)):
|
||||
return filter_ref
|
||||
return None
|
||||
|
||||
|
||||
def _determine_permission(handler: StarHandlerMetadata) -> str:
|
||||
for filter_ref in handler.event_filters:
|
||||
if isinstance(filter_ref, PermissionTypeFilter):
|
||||
return (
|
||||
"admin"
|
||||
if filter_ref.permission_type == PermissionType.ADMIN
|
||||
else "member"
|
||||
)
|
||||
return "everyone"
|
||||
|
||||
|
||||
def _resolve_group_parent_signature(group_filter: CommandGroupFilter) -> str:
|
||||
signatures: list[str] = []
|
||||
parent = group_filter.parent_group
|
||||
while parent:
|
||||
signatures.append(getattr(parent, "_original_group_name", parent.group_name))
|
||||
parent = parent.parent_group
|
||||
return " ".join(reversed(signatures)).strip()
|
||||
|
||||
|
||||
def _find_parent_group_handler(module_path: str, parent_signature: str) -> str:
|
||||
"""根据模块路径和父级签名,找到对应的指令组 handler_full_name。"""
|
||||
parent_sig_normalized = parent_signature.strip()
|
||||
for handler in star_handlers_registry:
|
||||
if handler.handler_module_path != module_path:
|
||||
continue
|
||||
filter_ref = _locate_primary_filter(handler)
|
||||
if not isinstance(filter_ref, CommandGroupFilter):
|
||||
continue
|
||||
# 检查该指令组的完整指令名是否匹配 parent_signature
|
||||
group_names = filter_ref.get_complete_command_names()
|
||||
if parent_sig_normalized in group_names:
|
||||
return handler.handler_full_name
|
||||
return ""
|
||||
|
||||
|
||||
def _compose_command(parent_signature: str, fragment: str | None) -> str:
|
||||
fragment = (fragment or "").strip()
|
||||
parent_signature = parent_signature.strip()
|
||||
if not parent_signature:
|
||||
return fragment
|
||||
if not fragment:
|
||||
return parent_signature
|
||||
return f"{parent_signature} {fragment}"
|
||||
|
||||
|
||||
def _bind_descriptor_with_config(
|
||||
descriptor: CommandDescriptor,
|
||||
config: CommandConfig,
|
||||
) -> None:
|
||||
_apply_config_to_descriptor(descriptor, config)
|
||||
_apply_config_to_runtime(descriptor, config)
|
||||
|
||||
|
||||
def _apply_config_to_descriptor(
|
||||
descriptor: CommandDescriptor,
|
||||
config: CommandConfig,
|
||||
) -> None:
|
||||
descriptor.config = config
|
||||
descriptor.enabled = config.enabled
|
||||
|
||||
if config.original_command:
|
||||
descriptor.original_command = config.original_command
|
||||
|
||||
new_fragment = config.resolved_command or descriptor.current_fragment
|
||||
descriptor.current_fragment = new_fragment
|
||||
descriptor.effective_command = _compose_command(
|
||||
descriptor.parent_signature,
|
||||
new_fragment,
|
||||
)
|
||||
|
||||
extra = config.extra_data or {}
|
||||
resolved_aliases = extra.get("resolved_aliases")
|
||||
if isinstance(resolved_aliases, list):
|
||||
descriptor.aliases = [str(x) for x in resolved_aliases if str(x).strip()]
|
||||
|
||||
|
||||
def _apply_config_to_runtime(
|
||||
descriptor: CommandDescriptor,
|
||||
config: CommandConfig,
|
||||
) -> None:
|
||||
descriptor.handler.enabled = config.enabled
|
||||
if descriptor.filter_ref:
|
||||
if descriptor.current_fragment:
|
||||
_set_filter_fragment(descriptor.filter_ref, descriptor.current_fragment)
|
||||
extra = config.extra_data or {}
|
||||
resolved_aliases = extra.get("resolved_aliases")
|
||||
if isinstance(resolved_aliases, list):
|
||||
_set_filter_aliases(
|
||||
descriptor.filter_ref,
|
||||
[str(x) for x in resolved_aliases if str(x).strip()],
|
||||
)
|
||||
|
||||
|
||||
def _bind_configs_to_descriptors(
|
||||
descriptors: list[CommandDescriptor],
|
||||
config_records: list[CommandConfig],
|
||||
) -> dict[str, CommandConfig]:
|
||||
config_map = {cfg.handler_full_name: cfg for cfg in config_records}
|
||||
for desc in descriptors:
|
||||
if cfg := config_map.get(desc.handler_full_name):
|
||||
_bind_descriptor_with_config(desc, cfg)
|
||||
return config_map
|
||||
|
||||
|
||||
def _group_conflicts(
|
||||
descriptors: list[CommandDescriptor],
|
||||
) -> dict[str, list[CommandDescriptor]]:
|
||||
conflicts: dict[str, list[CommandDescriptor]] = defaultdict(list)
|
||||
for desc in descriptors:
|
||||
if desc.effective_command and desc.enabled:
|
||||
conflicts[desc.effective_command].append(desc)
|
||||
return {k: v for k, v in conflicts.items() if len(v) > 1}
|
||||
|
||||
|
||||
def _set_filter_fragment(
|
||||
filter_ref: CommandFilter | CommandGroupFilter,
|
||||
fragment: str,
|
||||
) -> None:
|
||||
attr = (
|
||||
"group_name" if isinstance(filter_ref, CommandGroupFilter) else "command_name"
|
||||
)
|
||||
current_value = getattr(filter_ref, attr)
|
||||
if fragment == current_value:
|
||||
return
|
||||
setattr(filter_ref, attr, fragment)
|
||||
if hasattr(filter_ref, "_cmpl_cmd_names"):
|
||||
filter_ref._cmpl_cmd_names = None
|
||||
|
||||
|
||||
def _set_filter_aliases(
|
||||
filter_ref: CommandFilter | CommandGroupFilter,
|
||||
aliases: list[str],
|
||||
) -> None:
|
||||
current_aliases = getattr(filter_ref, "alias", set())
|
||||
if set(aliases) == current_aliases:
|
||||
return
|
||||
setattr(filter_ref, "alias", set(aliases))
|
||||
if hasattr(filter_ref, "_cmpl_cmd_names"):
|
||||
filter_ref._cmpl_cmd_names = None
|
||||
|
||||
|
||||
def _is_command_in_use(
|
||||
target_handler_full_name: str,
|
||||
candidate_full_command: str,
|
||||
) -> bool:
|
||||
candidate = candidate_full_command.strip()
|
||||
for handler in star_handlers_registry:
|
||||
if handler.handler_full_name == target_handler_full_name:
|
||||
continue
|
||||
filter_ref = _locate_primary_filter(handler)
|
||||
if not filter_ref:
|
||||
continue
|
||||
names = {name.strip() for name in filter_ref.get_complete_command_names()}
|
||||
if candidate in names:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _descriptor_to_dict(desc: CommandDescriptor) -> dict[str, Any]:
|
||||
result = {
|
||||
"handler_full_name": desc.handler_full_name,
|
||||
"handler_name": desc.handler_name,
|
||||
"plugin": desc.plugin_name,
|
||||
"plugin_display_name": desc.plugin_display_name,
|
||||
"module_path": desc.module_path,
|
||||
"description": desc.description,
|
||||
"type": desc.command_type,
|
||||
"parent_signature": desc.parent_signature,
|
||||
"parent_group_handler": desc.parent_group_handler,
|
||||
"original_command": desc.original_command,
|
||||
"current_fragment": desc.current_fragment,
|
||||
"effective_command": desc.effective_command,
|
||||
"aliases": desc.aliases,
|
||||
"permission": desc.permission,
|
||||
"enabled": desc.enabled,
|
||||
"is_group": desc.is_group,
|
||||
"has_conflict": desc.has_conflict,
|
||||
"reserved": desc.reserved,
|
||||
}
|
||||
# 如果是指令组,包含子指令列表
|
||||
if desc.is_group and desc.sub_commands:
|
||||
result["sub_commands"] = [_descriptor_to_dict(sub) for sub in desc.sub_commands]
|
||||
else:
|
||||
result["sub_commands"] = []
|
||||
return result
|
||||
@@ -149,12 +149,9 @@ class Context:
|
||||
contexts: context messages for the LLM
|
||||
max_steps: Maximum number of tool calls before stopping the loop
|
||||
**kwargs: Additional keyword arguments. The kwargs will not be passed to the LLM directly for now, but can include:
|
||||
stream: bool - whether to stream the LLM response
|
||||
agent_hooks: BaseAgentRunHooks[AstrAgentContext] - hooks to run during agent execution
|
||||
agent_context: AstrAgentContext - context to use for the agent
|
||||
|
||||
other kwargs will be DIRECTLY passed to the runner.reset() method
|
||||
|
||||
Returns:
|
||||
The final LLMResponse after tool calls are completed.
|
||||
|
||||
@@ -197,15 +194,6 @@ class Context:
|
||||
)
|
||||
agent_runner = ToolLoopAgentRunner()
|
||||
tool_executor = FunctionToolExecutor()
|
||||
|
||||
streaming = kwargs.get("stream", False)
|
||||
|
||||
other_kwargs = {
|
||||
k: v
|
||||
for k, v in kwargs.items()
|
||||
if k not in ["stream", "agent_hooks", "agent_context"]
|
||||
}
|
||||
|
||||
await agent_runner.reset(
|
||||
provider=prov,
|
||||
request=request,
|
||||
@@ -215,8 +203,7 @@ class Context:
|
||||
),
|
||||
tool_executor=tool_executor,
|
||||
agent_hooks=agent_hooks,
|
||||
streaming=streaming,
|
||||
**other_kwargs,
|
||||
streaming=kwargs.get("stream", False),
|
||||
)
|
||||
async for _ in agent_runner.step_until_done(max_steps):
|
||||
pass
|
||||
@@ -280,10 +267,6 @@ class Context:
|
||||
):
|
||||
"""通过 ID 获取对应的 LLM Provider。"""
|
||||
prov = self.provider_manager.inst_map.get(provider_id)
|
||||
if provider_id and not prov:
|
||||
logger.warning(
|
||||
f"没有找到 ID 为 {provider_id} 的提供商,这可能是由于您修改了提供商(模型)ID 导致的。"
|
||||
)
|
||||
return prov
|
||||
|
||||
def get_all_providers(self) -> list[Provider]:
|
||||
@@ -390,7 +373,7 @@ class Context:
|
||||
if not module_path:
|
||||
_parts = []
|
||||
module_part = tool.__module__.split(".")
|
||||
flags = ["builtin_stars", "plugins"]
|
||||
flags = ["packages", "plugins"]
|
||||
for i, part in enumerate(module_part):
|
||||
_parts.append(part)
|
||||
if part in flags and i + 1 < len(module_part):
|
||||
|
||||
@@ -40,7 +40,6 @@ class CommandFilter(HandlerFilter):
|
||||
):
|
||||
self.command_name = command_name
|
||||
self.alias = alias if alias else set()
|
||||
self._original_command_name = command_name
|
||||
self.parent_command_names = (
|
||||
parent_command_names if parent_command_names is not None else [""]
|
||||
)
|
||||
|
||||
@@ -18,7 +18,6 @@ class CommandGroupFilter(HandlerFilter):
|
||||
):
|
||||
self.group_name = group_name
|
||||
self.alias = alias if alias else set()
|
||||
self._original_group_name = group_name
|
||||
self.sub_command_filters: list[CommandFilter | CommandGroupFilter] = []
|
||||
self.custom_filter_list: list[CustomFilter] = []
|
||||
self.parent_group = parent_group
|
||||
|
||||
@@ -12,7 +12,6 @@ from .star_handler import (
|
||||
register_on_llm_request,
|
||||
register_on_llm_response,
|
||||
register_on_platform_loaded,
|
||||
register_on_waiting_llm_request,
|
||||
register_permission_type,
|
||||
register_platform_adapter_type,
|
||||
register_regex,
|
||||
@@ -31,7 +30,6 @@ __all__ = [
|
||||
"register_on_llm_request",
|
||||
"register_on_llm_response",
|
||||
"register_on_platform_loaded",
|
||||
"register_on_waiting_llm_request",
|
||||
"register_permission_type",
|
||||
"register_platform_adapter_type",
|
||||
"register_regex",
|
||||
|
||||
@@ -339,30 +339,6 @@ def register_on_platform_loaded(**kwargs):
|
||||
return decorator
|
||||
|
||||
|
||||
def register_on_waiting_llm_request(**kwargs):
|
||||
"""当等待调用 LLM 时的通知事件(在获取锁之前)
|
||||
|
||||
此钩子在消息确定要调用 LLM 但还未开始排队等锁时触发,
|
||||
适合用于发送"正在思考中..."等用户反馈提示。
|
||||
|
||||
Examples:
|
||||
```py
|
||||
@on_waiting_llm_request()
|
||||
async def on_waiting_llm(self, event: AstrMessageEvent) -> None:
|
||||
await event.send("🤔 正在思考中...")
|
||||
```
|
||||
|
||||
"""
|
||||
|
||||
def decorator(awaitable):
|
||||
_ = get_handler_or_create(
|
||||
awaitable, EventType.OnWaitingLLMRequestEvent, **kwargs
|
||||
)
|
||||
return awaitable
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def register_on_llm_request(**kwargs):
|
||||
"""当有 LLM 请求时的事件
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ class SessionServiceManager:
|
||||
# =============================================================================
|
||||
|
||||
@staticmethod
|
||||
async def is_llm_enabled_for_session(session_id: str) -> bool:
|
||||
def is_llm_enabled_for_session(session_id: str) -> bool:
|
||||
"""检查LLM是否在指定会话中启用
|
||||
|
||||
Args:
|
||||
@@ -23,11 +23,11 @@ class SessionServiceManager:
|
||||
|
||||
"""
|
||||
# 获取会话服务配置
|
||||
session_services = await sp.get_async(
|
||||
session_services = sp.get(
|
||||
"session_service_config",
|
||||
{},
|
||||
scope="umo",
|
||||
scope_id=session_id,
|
||||
key="session_service_config",
|
||||
default={},
|
||||
)
|
||||
|
||||
# 如果配置了该会话的LLM状态,返回该状态
|
||||
@@ -39,7 +39,7 @@ class SessionServiceManager:
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
async def set_llm_status_for_session(session_id: str, enabled: bool) -> None:
|
||||
def set_llm_status_for_session(session_id: str, enabled: bool) -> None:
|
||||
"""设置LLM在指定会话中的启停状态
|
||||
|
||||
Args:
|
||||
@@ -48,24 +48,18 @@ class SessionServiceManager:
|
||||
|
||||
"""
|
||||
session_config = (
|
||||
await sp.get_async(
|
||||
scope="umo",
|
||||
scope_id=session_id,
|
||||
key="session_service_config",
|
||||
default={},
|
||||
)
|
||||
or {}
|
||||
sp.get("session_service_config", {}, scope="umo", scope_id=session_id) or {}
|
||||
)
|
||||
session_config["llm_enabled"] = enabled
|
||||
await sp.put_async(
|
||||
sp.put(
|
||||
"session_service_config",
|
||||
session_config,
|
||||
scope="umo",
|
||||
scope_id=session_id,
|
||||
key="session_service_config",
|
||||
value=session_config,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
async def should_process_llm_request(event: AstrMessageEvent) -> bool:
|
||||
def should_process_llm_request(event: AstrMessageEvent) -> bool:
|
||||
"""检查是否应该处理LLM请求
|
||||
|
||||
Args:
|
||||
@@ -76,14 +70,14 @@ class SessionServiceManager:
|
||||
|
||||
"""
|
||||
session_id = event.unified_msg_origin
|
||||
return await SessionServiceManager.is_llm_enabled_for_session(session_id)
|
||||
return SessionServiceManager.is_llm_enabled_for_session(session_id)
|
||||
|
||||
# =============================================================================
|
||||
# TTS 相关方法
|
||||
# =============================================================================
|
||||
|
||||
@staticmethod
|
||||
async def is_tts_enabled_for_session(session_id: str) -> bool:
|
||||
def is_tts_enabled_for_session(session_id: str) -> bool:
|
||||
"""检查TTS是否在指定会话中启用
|
||||
|
||||
Args:
|
||||
@@ -94,11 +88,11 @@ class SessionServiceManager:
|
||||
|
||||
"""
|
||||
# 获取会话服务配置
|
||||
session_services = await sp.get_async(
|
||||
session_services = sp.get(
|
||||
"session_service_config",
|
||||
{},
|
||||
scope="umo",
|
||||
scope_id=session_id,
|
||||
key="session_service_config",
|
||||
default={},
|
||||
)
|
||||
|
||||
# 如果配置了该会话的TTS状态,返回该状态
|
||||
@@ -110,7 +104,7 @@ class SessionServiceManager:
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
async def set_tts_status_for_session(session_id: str, enabled: bool) -> None:
|
||||
def set_tts_status_for_session(session_id: str, enabled: bool) -> None:
|
||||
"""设置TTS在指定会话中的启停状态
|
||||
|
||||
Args:
|
||||
@@ -119,20 +113,14 @@ class SessionServiceManager:
|
||||
|
||||
"""
|
||||
session_config = (
|
||||
await sp.get_async(
|
||||
scope="umo",
|
||||
scope_id=session_id,
|
||||
key="session_service_config",
|
||||
default={},
|
||||
)
|
||||
or {}
|
||||
sp.get("session_service_config", {}, scope="umo", scope_id=session_id) or {}
|
||||
)
|
||||
session_config["tts_enabled"] = enabled
|
||||
await sp.put_async(
|
||||
sp.put(
|
||||
"session_service_config",
|
||||
session_config,
|
||||
scope="umo",
|
||||
scope_id=session_id,
|
||||
key="session_service_config",
|
||||
value=session_config,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
@@ -140,7 +128,7 @@ class SessionServiceManager:
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
async def should_process_tts_request(event: AstrMessageEvent) -> bool:
|
||||
def should_process_tts_request(event: AstrMessageEvent) -> bool:
|
||||
"""检查是否应该处理TTS请求
|
||||
|
||||
Args:
|
||||
@@ -151,14 +139,14 @@ class SessionServiceManager:
|
||||
|
||||
"""
|
||||
session_id = event.unified_msg_origin
|
||||
return await SessionServiceManager.is_tts_enabled_for_session(session_id)
|
||||
return SessionServiceManager.is_tts_enabled_for_session(session_id)
|
||||
|
||||
# =============================================================================
|
||||
# 会话整体启停相关方法
|
||||
# =============================================================================
|
||||
|
||||
@staticmethod
|
||||
async def is_session_enabled(session_id: str) -> bool:
|
||||
def is_session_enabled(session_id: str) -> bool:
|
||||
"""检查会话是否整体启用
|
||||
|
||||
Args:
|
||||
@@ -169,11 +157,11 @@ class SessionServiceManager:
|
||||
|
||||
"""
|
||||
# 获取会话服务配置
|
||||
session_services = await sp.get_async(
|
||||
session_services = sp.get(
|
||||
"session_service_config",
|
||||
{},
|
||||
scope="umo",
|
||||
scope_id=session_id,
|
||||
key="session_service_config",
|
||||
default={},
|
||||
)
|
||||
|
||||
# 如果配置了该会话的整体状态,返回该状态
|
||||
|
||||
@@ -8,10 +8,7 @@ class SessionPluginManager:
|
||||
"""管理会话级别的插件启停状态"""
|
||||
|
||||
@staticmethod
|
||||
async def is_plugin_enabled_for_session(
|
||||
session_id: str,
|
||||
plugin_name: str,
|
||||
) -> bool:
|
||||
def is_plugin_enabled_for_session(session_id: str, plugin_name: str) -> bool:
|
||||
"""检查插件是否在指定会话中启用
|
||||
|
||||
Args:
|
||||
@@ -23,11 +20,11 @@ class SessionPluginManager:
|
||||
|
||||
"""
|
||||
# 获取会话插件配置
|
||||
session_plugin_config = await sp.get_async(
|
||||
session_plugin_config = sp.get(
|
||||
"session_plugin_config",
|
||||
{},
|
||||
scope="umo",
|
||||
scope_id=session_id,
|
||||
key="session_plugin_config",
|
||||
default={},
|
||||
)
|
||||
session_config = session_plugin_config.get(session_id, {})
|
||||
|
||||
@@ -46,10 +43,7 @@ class SessionPluginManager:
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
async def filter_handlers_by_session(
|
||||
event: AstrMessageEvent,
|
||||
handlers: list,
|
||||
) -> list:
|
||||
def filter_handlers_by_session(event: AstrMessageEvent, handlers: list) -> list:
|
||||
"""根据会话配置过滤处理器列表
|
||||
|
||||
Args:
|
||||
@@ -65,15 +59,6 @@ class SessionPluginManager:
|
||||
session_id = event.unified_msg_origin
|
||||
filtered_handlers = []
|
||||
|
||||
session_plugin_config = await sp.get_async(
|
||||
scope="umo",
|
||||
scope_id=session_id,
|
||||
key="session_plugin_config",
|
||||
default={},
|
||||
)
|
||||
session_config = session_plugin_config.get(session_id, {})
|
||||
disabled_plugins = session_config.get("disabled_plugins", [])
|
||||
|
||||
for handler in handlers:
|
||||
# 获取处理器对应的插件
|
||||
plugin = star_map.get(handler.handler_module_path)
|
||||
@@ -91,11 +76,14 @@ class SessionPluginManager:
|
||||
continue
|
||||
|
||||
# 检查插件是否在当前会话中启用
|
||||
if plugin.name in disabled_plugins:
|
||||
if SessionPluginManager.is_plugin_enabled_for_session(
|
||||
session_id,
|
||||
plugin.name,
|
||||
):
|
||||
filtered_handlers.append(handler)
|
||||
else:
|
||||
logger.debug(
|
||||
f"插件 {plugin.name} 在会话 {session_id} 中被禁用,跳过处理器 {handler.handler_name}",
|
||||
)
|
||||
else:
|
||||
filtered_handlers.append(handler)
|
||||
|
||||
return filtered_handlers
|
||||
|
||||
@@ -118,8 +118,6 @@ class StarHandlerRegistry(Generic[T]):
|
||||
# 过滤事件类型
|
||||
if handler.event_type != event_type:
|
||||
continue
|
||||
if not handler.enabled:
|
||||
continue
|
||||
# 过滤启用状态
|
||||
if only_activated:
|
||||
plugin = star_map.get(handler.handler_module_path)
|
||||
@@ -184,7 +182,6 @@ class EventType(enum.Enum):
|
||||
OnPlatformLoadedEvent = enum.auto() # 平台加载完成
|
||||
|
||||
AdapterMessageEvent = enum.auto() # 收到适配器发来的消息
|
||||
OnWaitingLLMRequestEvent = enum.auto() # 等待调用 LLM(在获取锁之前,仅通知)
|
||||
OnLLMRequestEvent = enum.auto() # 收到 LLM 请求(可以是用户也可以是插件)
|
||||
OnLLMResponseEvent = enum.auto() # LLM 响应后
|
||||
OnDecoratingResultEvent = enum.auto() # 发送消息前
|
||||
@@ -223,8 +220,6 @@ class StarHandlerMetadata(Generic[H]):
|
||||
extras_configs: dict = field(default_factory=dict)
|
||||
"""插件注册的一些其他的信息, 如 priority 等"""
|
||||
|
||||
enabled: bool = True
|
||||
|
||||
def __lt__(self, other: StarHandlerMetadata):
|
||||
"""定义小于运算符以支持优先队列"""
|
||||
return self.extras_configs.get("priority", 0) < other.extras_configs.get(
|
||||
|
||||
@@ -18,13 +18,11 @@ from astrbot.core.config.astrbot_config import AstrBotConfig
|
||||
from astrbot.core.provider.register import llm_tools
|
||||
from astrbot.core.utils.astrbot_path import (
|
||||
get_astrbot_config_path,
|
||||
get_astrbot_path,
|
||||
get_astrbot_plugin_path,
|
||||
)
|
||||
from astrbot.core.utils.io import remove_dir
|
||||
|
||||
from . import StarMetadata
|
||||
from .command_management import sync_command_configs
|
||||
from .context import Context
|
||||
from .filter.permission import PermissionType, PermissionTypeFilter
|
||||
from .star import star_map, star_registry
|
||||
@@ -50,10 +48,13 @@ class PluginManager:
|
||||
"""存储插件的路径。即 data/plugins"""
|
||||
self.plugin_config_path = get_astrbot_config_path()
|
||||
"""存储插件配置的路径。data/config"""
|
||||
self.reserved_plugin_path = os.path.join(
|
||||
get_astrbot_path(), "astrbot", "builtin_stars"
|
||||
self.reserved_plugin_path = os.path.abspath(
|
||||
os.path.join(
|
||||
os.path.dirname(os.path.abspath(__file__)),
|
||||
"../../../packages",
|
||||
),
|
||||
)
|
||||
"""保留插件的路径。在 astrbot/builtin_stars 目录下"""
|
||||
"""保留插件的路径。在 packages 目录下"""
|
||||
self.conf_schema_fname = "_conf_schema.json"
|
||||
self.logo_fname = "logo.png"
|
||||
"""插件配置 Schema 文件名"""
|
||||
@@ -250,7 +251,7 @@ class PluginManager:
|
||||
list[str]: 与该插件相关的模块名列表
|
||||
|
||||
"""
|
||||
prefix = "astrbot.builtin_stars." if is_reserved else "data.plugins."
|
||||
prefix = "packages." if is_reserved else "data.plugins."
|
||||
return [
|
||||
key
|
||||
for key in list(sys.modules.keys())
|
||||
@@ -268,7 +269,7 @@ class PluginManager:
|
||||
可以基于模块名模式或插件目录名移除模块,用于清理插件相关的模块缓存
|
||||
|
||||
Args:
|
||||
module_patterns: 要移除的模块名模式列表(例如 ["data.plugins", "astrbot.builtin_stars"])
|
||||
module_patterns: 要移除的模块名模式列表(例如 ["data.plugins", "packages"])
|
||||
root_dir_name: 插件根目录名,用于移除与该插件相关的所有模块
|
||||
is_reserved: 插件是否为保留插件(影响模块路径前缀)
|
||||
|
||||
@@ -380,9 +381,9 @@ class PluginManager:
|
||||
reserved = plugin_module.get(
|
||||
"reserved",
|
||||
False,
|
||||
) # 是否是保留插件。目前在 astrbot/builtin_stars 目录下的都是保留插件。保留插件不可以卸载。
|
||||
) # 是否是保留插件。目前在 packages/ 目录下的都是保留插件。保留插件不可以卸载。
|
||||
|
||||
path = "data.plugins." if not reserved else "astrbot.builtin_stars."
|
||||
path = "data.plugins." if not reserved else "packages."
|
||||
path += root_dir_name + "." + module_str
|
||||
|
||||
# 检查是否需要载入指定的插件
|
||||
@@ -466,18 +467,6 @@ class PluginManager:
|
||||
metadata.star_cls = metadata.star_cls_type(
|
||||
context=self.context,
|
||||
)
|
||||
|
||||
p_name = (metadata.name or "unknown").lower().replace("/", "_")
|
||||
p_author = (
|
||||
(metadata.author or "unknown").lower().replace("/", "_")
|
||||
)
|
||||
setattr(metadata.star_cls, "name", p_name)
|
||||
setattr(metadata.star_cls, "author", p_author)
|
||||
setattr(
|
||||
metadata.star_cls,
|
||||
"plugin_id",
|
||||
f"{p_author}/{p_name}",
|
||||
)
|
||||
else:
|
||||
logger.info(f"插件 {metadata.name} 已被禁用。")
|
||||
|
||||
@@ -629,11 +618,6 @@ class PluginManager:
|
||||
# 清除 pip.main 导致的多余的 logging handlers
|
||||
for handler in logging.root.handlers[:]:
|
||||
logging.root.removeHandler(handler)
|
||||
try:
|
||||
await sync_command_configs()
|
||||
except Exception as e:
|
||||
logger.error(f"同步指令配置失败: {e!s}")
|
||||
logger.error(traceback.format_exc())
|
||||
|
||||
if not fail_rec:
|
||||
return True, None
|
||||
@@ -827,7 +811,7 @@ class PluginManager:
|
||||
if (
|
||||
mp
|
||||
and mp.startswith(plugin_module_path)
|
||||
and not mp.endswith(("astrbot.builtin_stars", "data.plugins"))
|
||||
and not mp.endswith(("packages", "data.plugins"))
|
||||
):
|
||||
to_remove.append(func_tool)
|
||||
for func_tool in to_remove:
|
||||
@@ -882,7 +866,7 @@ class PluginManager:
|
||||
plugin.module_path
|
||||
and mp
|
||||
and plugin.module_path.startswith(mp)
|
||||
and not mp.endswith(("astrbot.builtin_stars", "data.plugins"))
|
||||
and not mp.endswith(("packages", "data.plugins"))
|
||||
):
|
||||
func_tool.active = False
|
||||
if func_tool.name not in inactivated_llm_tools:
|
||||
@@ -931,7 +915,7 @@ class PluginManager:
|
||||
plugin.module_path
|
||||
and mp
|
||||
and plugin.module_path.startswith(mp)
|
||||
and not mp.endswith(("astrbot.builtin_stars", "data.plugins"))
|
||||
and not mp.endswith(("packages", "data.plugins"))
|
||||
and func_tool.name in inactivated_llm_tools
|
||||
):
|
||||
inactivated_llm_tools.remove(func_tool.name)
|
||||
@@ -944,49 +928,8 @@ class PluginManager:
|
||||
dir_name = os.path.basename(zip_file_path).replace(".zip", "")
|
||||
dir_name = dir_name.removesuffix("-master").removesuffix("-main").lower()
|
||||
desti_dir = os.path.join(self.plugin_store_path, dir_name)
|
||||
|
||||
# 第一步:检查是否已安装同目录名的插件,先终止旧插件
|
||||
existing_plugin = None
|
||||
for star in self.context.get_all_stars():
|
||||
if star.root_dir_name == dir_name:
|
||||
existing_plugin = star
|
||||
break
|
||||
|
||||
if existing_plugin:
|
||||
logger.info(f"检测到插件 {existing_plugin.name} 已安装,正在终止旧插件...")
|
||||
try:
|
||||
await self._terminate_plugin(existing_plugin)
|
||||
except Exception:
|
||||
logger.warning(traceback.format_exc())
|
||||
if existing_plugin.name and existing_plugin.module_path:
|
||||
await self._unbind_plugin(
|
||||
existing_plugin.name, existing_plugin.module_path
|
||||
)
|
||||
|
||||
self.updator.unzip_file(zip_file_path, desti_dir)
|
||||
|
||||
# 第二步:解压后,读取新插件的 metadata.yaml,检查是否存在同名但不同目录的插件
|
||||
try:
|
||||
new_metadata = self._load_plugin_metadata(desti_dir)
|
||||
if new_metadata and new_metadata.name:
|
||||
for star in self.context.get_all_stars():
|
||||
if (
|
||||
star.name == new_metadata.name
|
||||
and star.root_dir_name != dir_name
|
||||
):
|
||||
logger.warning(
|
||||
f"检测到同名插件 {star.name} 存在于不同目录 {star.root_dir_name},正在终止..."
|
||||
)
|
||||
try:
|
||||
await self._terminate_plugin(star)
|
||||
except Exception:
|
||||
logger.warning(traceback.format_exc())
|
||||
if star.name and star.module_path:
|
||||
await self._unbind_plugin(star.name, star.module_path)
|
||||
break # 只处理第一个匹配的
|
||||
except Exception as e:
|
||||
logger.debug(f"读取新插件 metadata.yaml 失败,跳过同名检查: {e!s}")
|
||||
|
||||
# remove the zip
|
||||
try:
|
||||
os.remove(zip_file_path)
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
import fnmatch
|
||||
|
||||
from astrbot.core.utils.shared_preferences import SharedPreferences
|
||||
|
||||
|
||||
@@ -11,15 +9,14 @@ class UmopConfigRouter:
|
||||
"""UMOP 到配置文件 ID 的映射"""
|
||||
self.sp = sp
|
||||
|
||||
async def initialize(self):
|
||||
await self._load_routing_table()
|
||||
self._load_routing_table()
|
||||
|
||||
async def _load_routing_table(self):
|
||||
def _load_routing_table(self):
|
||||
"""加载路由表"""
|
||||
# 从 SharedPreferences 中加载 umop_to_conf_id 映射
|
||||
sp_data = await self.sp.get_async(
|
||||
key="umop_config_routing",
|
||||
default={},
|
||||
sp_data = self.sp.get(
|
||||
"umop_config_routing",
|
||||
{},
|
||||
scope="global",
|
||||
scope_id="global",
|
||||
)
|
||||
@@ -33,7 +30,7 @@ class UmopConfigRouter:
|
||||
if len(p1_ls) != 3 or len(p2_ls) != 3:
|
||||
return False # 非法格式
|
||||
|
||||
return all(p == "" or fnmatch.fnmatchcase(t, p) for p, t in zip(p1_ls, p2_ls))
|
||||
return all(p == "" or p == "*" or p == t for p, t in zip(p1_ls, p2_ls))
|
||||
|
||||
def get_conf_id_for_umop(self, umo: str) -> str | None:
|
||||
"""根据 UMO 获取对应的配置文件 ID
|
||||
|
||||
@@ -5,10 +5,6 @@
|
||||
数据目录路径:固定为根目录下的 data 目录
|
||||
配置文件路径:固定为数据目录下的 config 目录
|
||||
插件目录路径:固定为数据目录下的 plugins 目录
|
||||
插件数据目录路径:固定为数据目录下的 plugin_data 目录
|
||||
T2I 模板目录路径:固定为数据目录下的 t2i_templates 目录
|
||||
WebChat 数据目录路径:固定为数据目录下的 webchat 目录
|
||||
临时文件目录路径:固定为数据目录下的 temp 目录
|
||||
"""
|
||||
|
||||
import os
|
||||
@@ -41,33 +37,3 @@ def get_astrbot_config_path() -> str:
|
||||
def get_astrbot_plugin_path() -> str:
|
||||
"""获取Astrbot插件目录路径"""
|
||||
return os.path.realpath(os.path.join(get_astrbot_data_path(), "plugins"))
|
||||
|
||||
|
||||
def get_astrbot_plugin_data_path() -> str:
|
||||
"""获取Astrbot插件数据目录路径"""
|
||||
return os.path.realpath(os.path.join(get_astrbot_data_path(), "plugin_data"))
|
||||
|
||||
|
||||
def get_astrbot_t2i_templates_path() -> str:
|
||||
"""获取Astrbot T2I 模板目录路径"""
|
||||
return os.path.realpath(os.path.join(get_astrbot_data_path(), "t2i_templates"))
|
||||
|
||||
|
||||
def get_astrbot_webchat_path() -> str:
|
||||
"""获取Astrbot WebChat 数据目录路径"""
|
||||
return os.path.realpath(os.path.join(get_astrbot_data_path(), "webchat"))
|
||||
|
||||
|
||||
def get_astrbot_temp_path() -> str:
|
||||
"""获取Astrbot临时文件目录路径"""
|
||||
return os.path.realpath(os.path.join(get_astrbot_data_path(), "temp"))
|
||||
|
||||
|
||||
def get_astrbot_knowledge_base_path() -> str:
|
||||
"""获取Astrbot知识库根目录路径"""
|
||||
return os.path.realpath(os.path.join(get_astrbot_data_path(), "knowledge_base"))
|
||||
|
||||
|
||||
def get_astrbot_backups_path() -> str:
|
||||
"""获取Astrbot备份目录路径"""
|
||||
return os.path.realpath(os.path.join(get_astrbot_data_path(), "backups"))
|
||||
|
||||
@@ -1,63 +0,0 @@
|
||||
from typing import Literal, TypedDict
|
||||
|
||||
import aiohttp
|
||||
|
||||
from astrbot.core import logger
|
||||
|
||||
|
||||
class LLMModalities(TypedDict):
|
||||
input: list[Literal["text", "image", "audio", "video"]]
|
||||
output: list[Literal["text", "image", "audio", "video"]]
|
||||
|
||||
|
||||
class LLMLimit(TypedDict):
|
||||
context: int
|
||||
output: int
|
||||
|
||||
|
||||
class LLMMetadata(TypedDict):
|
||||
id: str
|
||||
reasoning: bool
|
||||
tool_call: bool
|
||||
knowledge: str
|
||||
release_date: str
|
||||
modalities: LLMModalities
|
||||
open_weights: bool
|
||||
limit: LLMLimit
|
||||
|
||||
|
||||
LLM_METADATAS: dict[str, LLMMetadata] = {}
|
||||
|
||||
|
||||
async def update_llm_metadata():
|
||||
url = "https://models.dev/api.json"
|
||||
try:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.get(url) as response:
|
||||
data = await response.json()
|
||||
global LLM_METADATAS
|
||||
models = {}
|
||||
for info in data.values():
|
||||
for model in info.get("models", {}).values():
|
||||
model_id = model.get("id")
|
||||
if not model_id:
|
||||
continue
|
||||
models[model_id] = LLMMetadata(
|
||||
id=model_id,
|
||||
reasoning=model.get("reasoning", False),
|
||||
tool_call=model.get("tool_call", False),
|
||||
knowledge=model.get("knowledge", "none"),
|
||||
release_date=model.get("release_date", ""),
|
||||
modalities=model.get(
|
||||
"modalities", {"input": [], "output": []}
|
||||
),
|
||||
open_weights=model.get("open_weights", False),
|
||||
limit=model.get("limit", {"context": 0, "output": 0}),
|
||||
)
|
||||
# Replace the global cache in-place so references remain valid
|
||||
LLM_METADATAS.clear()
|
||||
LLM_METADATAS.update(models)
|
||||
logger.info(f"Successfully fetched metadata for {len(models)} LLMs.")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to fetch LLM metadata: {e}")
|
||||
return
|
||||
@@ -3,7 +3,6 @@ import traceback
|
||||
from astrbot.core import astrbot_config, logger
|
||||
from astrbot.core.astrbot_config_mgr import AstrBotConfig, AstrBotConfigManager
|
||||
from astrbot.core.db.migration.migra_45_to_46 import migrate_45_to_46
|
||||
from astrbot.core.db.migration.migra_token_usage import migrate_token_usage
|
||||
from astrbot.core.db.migration.migra_webchat_session import migrate_webchat_session
|
||||
|
||||
|
||||
@@ -33,92 +32,6 @@ def _migra_agent_runner_configs(conf: AstrBotConfig, ids_map: dict) -> None:
|
||||
logger.error(traceback.format_exc())
|
||||
|
||||
|
||||
def _migra_provider_to_source_structure(conf: AstrBotConfig) -> None:
|
||||
"""
|
||||
Migrate old provider structure to new provider-source separation.
|
||||
Provider only keeps: id, provider_source_id, model, modalities, custom_extra_body
|
||||
All other fields move to provider_sources.
|
||||
"""
|
||||
providers = conf.get("provider", [])
|
||||
provider_sources = conf.get("provider_sources", [])
|
||||
|
||||
# Track if any migration happened
|
||||
migrated = False
|
||||
|
||||
# Provider-only fields that should stay in provider
|
||||
provider_only_fields = {
|
||||
"id",
|
||||
"provider_source_id",
|
||||
"model",
|
||||
"modalities",
|
||||
"custom_extra_body",
|
||||
"enable",
|
||||
}
|
||||
|
||||
# Fields that should not go to source
|
||||
source_exclude_fields = provider_only_fields | {"model_config"}
|
||||
|
||||
for provider in providers:
|
||||
# Skip if already has provider_source_id
|
||||
if provider.get("provider_source_id"):
|
||||
continue
|
||||
|
||||
# Skip non-chat-completion types (they don't need source separation)
|
||||
provider_type = provider.get("provider_type", "")
|
||||
if provider_type != "chat_completion":
|
||||
# For old types without provider_type, check type field
|
||||
old_type = provider.get("type", "")
|
||||
if "chat_completion" not in old_type:
|
||||
continue
|
||||
|
||||
migrated = True
|
||||
logger.info(f"Migrating provider {provider.get('id')} to new structure")
|
||||
|
||||
# Extract source fields from provider
|
||||
source_fields = {}
|
||||
for key, value in list(provider.items()):
|
||||
if key not in source_exclude_fields:
|
||||
source_fields[key] = value
|
||||
|
||||
# Create new provider_source
|
||||
source_id = provider.get("id", "") + "_source"
|
||||
new_source = {"id": source_id, **source_fields}
|
||||
|
||||
# Update provider to only keep necessary fields
|
||||
provider["provider_source_id"] = source_id
|
||||
|
||||
# Extract model from model_config if exists
|
||||
if "model_config" in provider and isinstance(provider["model_config"], dict):
|
||||
model_config = provider["model_config"]
|
||||
provider["model"] = model_config.get("model", "")
|
||||
|
||||
# Put other model_config fields into custom_extra_body
|
||||
extra_body_fields = {k: v for k, v in model_config.items() if k != "model"}
|
||||
if extra_body_fields:
|
||||
if "custom_extra_body" not in provider:
|
||||
provider["custom_extra_body"] = {}
|
||||
provider["custom_extra_body"].update(extra_body_fields)
|
||||
|
||||
# Initialize new fields if not present
|
||||
if "modalities" not in provider:
|
||||
provider["modalities"] = []
|
||||
if "custom_extra_body" not in provider:
|
||||
provider["custom_extra_body"] = {}
|
||||
|
||||
# Remove fields that should be in source
|
||||
keys_to_remove = [k for k in provider.keys() if k not in provider_only_fields]
|
||||
for key in keys_to_remove:
|
||||
del provider[key]
|
||||
|
||||
# Add source to provider_sources
|
||||
provider_sources.append(new_source)
|
||||
|
||||
if migrated:
|
||||
conf["provider_sources"] = provider_sources
|
||||
conf.save_config()
|
||||
logger.info("Provider-source structure migration completed")
|
||||
|
||||
|
||||
async def migra(
|
||||
db, astrbot_config_mgr, umop_config_router, acm: AstrBotConfigManager
|
||||
) -> None:
|
||||
@@ -140,13 +53,6 @@ async def migra(
|
||||
logger.error(f"Migration for webchat session failed: {e!s}")
|
||||
logger.error(traceback.format_exc())
|
||||
|
||||
# migration for token_usage column
|
||||
try:
|
||||
await migrate_token_usage(db)
|
||||
except Exception as e:
|
||||
logger.error(f"Migration for token_usage column failed: {e!s}")
|
||||
logger.error(traceback.format_exc())
|
||||
|
||||
# migra third party agent runner configs
|
||||
_c = False
|
||||
providers = astrbot_config["provider"]
|
||||
@@ -165,10 +71,3 @@ async def migra(
|
||||
|
||||
for conf in acm.confs.values():
|
||||
_migra_agent_runner_configs(conf, ids_map)
|
||||
|
||||
# Migrate providers to new structure: extract source fields to provider_sources
|
||||
try:
|
||||
_migra_provider_to_source_structure(astrbot_config)
|
||||
except Exception as e:
|
||||
logger.error(f"Migration for provider-source structure failed: {e!s}")
|
||||
logger.error(traceback.format_exc())
|
||||
|
||||
@@ -1,29 +1,10 @@
|
||||
import asyncio
|
||||
import locale
|
||||
import logging
|
||||
import sys
|
||||
|
||||
logger = logging.getLogger("astrbot")
|
||||
|
||||
|
||||
def _robust_decode(line: bytes) -> str:
|
||||
"""解码字节流,兼容不同平台的编码"""
|
||||
try:
|
||||
return line.decode("utf-8").strip()
|
||||
except UnicodeDecodeError:
|
||||
pass
|
||||
try:
|
||||
return line.decode(locale.getpreferredencoding(False)).strip()
|
||||
except UnicodeDecodeError:
|
||||
pass
|
||||
if sys.platform.startswith("win"):
|
||||
try:
|
||||
return line.decode("gbk").strip()
|
||||
except UnicodeDecodeError:
|
||||
pass
|
||||
return line.decode("utf-8", errors="replace").strip()
|
||||
|
||||
|
||||
class PipInstaller:
|
||||
def __init__(self, pip_install_arg: str, pypi_index_url: str | None = None):
|
||||
self.pip_install_arg = pip_install_arg
|
||||
@@ -61,7 +42,7 @@ class PipInstaller:
|
||||
|
||||
assert process.stdout is not None
|
||||
async for line in process.stdout:
|
||||
logger.info(_robust_decode(line))
|
||||
logger.info(line.decode().strip())
|
||||
|
||||
await process.wait()
|
||||
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
from typing import TypeVar
|
||||
|
||||
from astrbot.core import sp
|
||||
|
||||
SUPPORTED_VALUE_TYPES = int | float | str | bytes | bool | dict | list | None
|
||||
_VT = TypeVar("_VT")
|
||||
|
||||
|
||||
class PluginKVStoreMixin:
|
||||
"""为插件提供键值存储功能的 Mixin 类"""
|
||||
|
||||
plugin_id: str
|
||||
|
||||
async def put_kv_data(
|
||||
self,
|
||||
key: str,
|
||||
value: SUPPORTED_VALUE_TYPES,
|
||||
) -> None:
|
||||
"""为指定插件存储一个键值对"""
|
||||
await sp.put_async("plugin", self.plugin_id, key, value)
|
||||
|
||||
async def get_kv_data(self, key: str, default: _VT) -> _VT | None:
|
||||
"""获取指定插件存储的键值对"""
|
||||
return await sp.get_async("plugin", self.plugin_id, key, default)
|
||||
|
||||
async def delete_kv_data(self, key: str) -> None:
|
||||
"""删除指定插件存储的键值对"""
|
||||
await sp.remove_async("plugin", self.plugin_id, key)
|
||||
@@ -1,7 +1,5 @@
|
||||
from .auth import AuthRoute
|
||||
from .backup import BackupRoute
|
||||
from .chat import ChatRoute
|
||||
from .command import CommandRoute
|
||||
from .config import ConfigRoute
|
||||
from .conversation import ConversationRoute
|
||||
from .file import FileRoute
|
||||
@@ -18,9 +16,7 @@ from .update import UpdateRoute
|
||||
|
||||
__all__ = [
|
||||
"AuthRoute",
|
||||
"BackupRoute",
|
||||
"ChatRoute",
|
||||
"CommandRoute",
|
||||
"ConfigRoute",
|
||||
"ConversationRoute",
|
||||
"FileRoute",
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -227,19 +227,16 @@ class ChatRoute(Route):
|
||||
text: str,
|
||||
media_parts: list,
|
||||
reasoning: str,
|
||||
agent_stats: dict,
|
||||
):
|
||||
"""保存 bot 消息到历史记录,返回保存的记录"""
|
||||
bot_message_parts = []
|
||||
bot_message_parts.extend(media_parts)
|
||||
if text:
|
||||
bot_message_parts.append({"type": "plain", "text": text})
|
||||
bot_message_parts.extend(media_parts)
|
||||
|
||||
new_his = {"type": "bot", "message": bot_message_parts}
|
||||
if reasoning:
|
||||
new_his["reasoning"] = reasoning
|
||||
if agent_stats:
|
||||
new_his["agent_stats"] = agent_stats
|
||||
|
||||
record = await self.platform_history_mgr.insert(
|
||||
platform_id="webchat",
|
||||
@@ -297,8 +294,7 @@ class ChatRoute(Route):
|
||||
accumulated_parts = []
|
||||
accumulated_text = ""
|
||||
accumulated_reasoning = ""
|
||||
tool_calls = {}
|
||||
agent_stats = {}
|
||||
|
||||
try:
|
||||
async with track_conversation(self.running_convs, webchat_conv_id):
|
||||
while True:
|
||||
@@ -318,16 +314,6 @@ class ChatRoute(Route):
|
||||
result_text = result["data"]
|
||||
msg_type = result.get("type")
|
||||
streaming = result.get("streaming", False)
|
||||
chain_type = result.get("chain_type")
|
||||
|
||||
if chain_type == "agent_stats":
|
||||
stats_info = {
|
||||
"type": "agent_stats",
|
||||
"data": json.loads(result_text),
|
||||
}
|
||||
yield f"data: {json.dumps(stats_info, ensure_ascii=False)}\n\n"
|
||||
agent_stats = stats_info["data"]
|
||||
continue
|
||||
|
||||
# 发送 SSE 数据
|
||||
try:
|
||||
@@ -349,35 +335,11 @@ class ChatRoute(Route):
|
||||
|
||||
# 累积消息部分
|
||||
if msg_type == "plain":
|
||||
chain_type = result.get("chain_type")
|
||||
if chain_type == "tool_call":
|
||||
tool_call = json.loads(result_text)
|
||||
tool_calls[tool_call.get("id")] = tool_call
|
||||
if accumulated_text:
|
||||
# 如果累积了文本,则先保存文本
|
||||
accumulated_parts.append(
|
||||
{"type": "plain", "text": accumulated_text}
|
||||
)
|
||||
accumulated_text = ""
|
||||
elif chain_type == "tool_call_result":
|
||||
tcr = json.loads(result_text)
|
||||
tc_id = tcr.get("id")
|
||||
if tc_id in tool_calls:
|
||||
tool_calls[tc_id]["result"] = tcr.get("result")
|
||||
tool_calls[tc_id]["finished_ts"] = tcr.get("ts")
|
||||
accumulated_parts.append(
|
||||
{
|
||||
"type": "tool_call",
|
||||
"tool_calls": [tool_calls[tc_id]],
|
||||
}
|
||||
)
|
||||
tool_calls.pop(tc_id, None)
|
||||
elif chain_type == "reasoning":
|
||||
chain_type = result.get("chain_type", "normal")
|
||||
if chain_type == "reasoning":
|
||||
accumulated_reasoning += result_text
|
||||
elif streaming:
|
||||
accumulated_text += result_text
|
||||
else:
|
||||
accumulated_text = result_text
|
||||
accumulated_text += result_text
|
||||
elif msg_type == "image":
|
||||
filename = result_text.replace("[IMAGE]", "")
|
||||
part = await self._create_attachment_from_file(
|
||||
@@ -405,20 +367,15 @@ class ChatRoute(Route):
|
||||
if msg_type == "end":
|
||||
break
|
||||
elif (
|
||||
(streaming and msg_type == "complete") or not streaming
|
||||
# or msg_type == "break"
|
||||
(streaming and msg_type == "complete")
|
||||
or not streaming
|
||||
or msg_type == "break"
|
||||
):
|
||||
if (
|
||||
chain_type == "tool_call"
|
||||
or chain_type == "tool_call_result"
|
||||
):
|
||||
continue
|
||||
saved_record = await self._save_bot_message(
|
||||
webchat_conv_id,
|
||||
accumulated_text,
|
||||
accumulated_parts,
|
||||
accumulated_reasoning,
|
||||
agent_stats,
|
||||
)
|
||||
# 发送保存的消息信息给前端
|
||||
if saved_record and not client_disconnected:
|
||||
@@ -433,11 +390,11 @@ class ChatRoute(Route):
|
||||
yield f"data: {json.dumps(saved_info, ensure_ascii=False)}\n\n"
|
||||
except Exception:
|
||||
pass
|
||||
accumulated_parts = []
|
||||
accumulated_text = ""
|
||||
accumulated_reasoning = ""
|
||||
# tool_calls = {}
|
||||
agent_stats = {}
|
||||
# 重置累积变量 (对于 break 后的下一段消息)
|
||||
if msg_type == "break":
|
||||
accumulated_parts = []
|
||||
accumulated_text = ""
|
||||
accumulated_reasoning = ""
|
||||
except BaseException as e:
|
||||
logger.exception(f"WebChat stream unexpected error: {e}", exc_info=True)
|
||||
|
||||
|
||||
@@ -1,83 +0,0 @@
|
||||
from quart import request
|
||||
|
||||
from astrbot.core.star.command_management import (
|
||||
list_command_conflicts,
|
||||
list_commands,
|
||||
)
|
||||
from astrbot.core.star.command_management import (
|
||||
rename_command as rename_command_service,
|
||||
)
|
||||
from astrbot.core.star.command_management import (
|
||||
toggle_command as toggle_command_service,
|
||||
)
|
||||
|
||||
from .route import Response, Route, RouteContext
|
||||
|
||||
|
||||
class CommandRoute(Route):
|
||||
def __init__(self, context: RouteContext) -> None:
|
||||
super().__init__(context)
|
||||
self.routes = {
|
||||
"/commands": ("GET", self.get_commands),
|
||||
"/commands/conflicts": ("GET", self.get_conflicts),
|
||||
"/commands/toggle": ("POST", self.toggle_command),
|
||||
"/commands/rename": ("POST", self.rename_command),
|
||||
}
|
||||
self.register_routes()
|
||||
|
||||
async def get_commands(self):
|
||||
commands = await list_commands()
|
||||
summary = {
|
||||
"total": len(commands),
|
||||
"disabled": len([cmd for cmd in commands if not cmd["enabled"]]),
|
||||
"conflicts": len([cmd for cmd in commands if cmd.get("has_conflict")]),
|
||||
}
|
||||
return Response().ok({"items": commands, "summary": summary}).__dict__
|
||||
|
||||
async def get_conflicts(self):
|
||||
conflicts = await list_command_conflicts()
|
||||
return Response().ok(conflicts).__dict__
|
||||
|
||||
async def toggle_command(self):
|
||||
data = await request.get_json()
|
||||
handler_full_name = data.get("handler_full_name")
|
||||
enabled = data.get("enabled")
|
||||
|
||||
if handler_full_name is None or enabled is None:
|
||||
return Response().error("handler_full_name 与 enabled 均为必填。").__dict__
|
||||
|
||||
if isinstance(enabled, str):
|
||||
enabled = enabled.lower() in ("1", "true", "yes", "on")
|
||||
|
||||
try:
|
||||
await toggle_command_service(handler_full_name, bool(enabled))
|
||||
except ValueError as exc:
|
||||
return Response().error(str(exc)).__dict__
|
||||
|
||||
payload = await _get_command_payload(handler_full_name)
|
||||
return Response().ok(payload).__dict__
|
||||
|
||||
async def rename_command(self):
|
||||
data = await request.get_json()
|
||||
handler_full_name = data.get("handler_full_name")
|
||||
new_name = data.get("new_name")
|
||||
aliases = data.get("aliases")
|
||||
|
||||
if not handler_full_name or not new_name:
|
||||
return Response().error("handler_full_name 与 new_name 均为必填。").__dict__
|
||||
|
||||
try:
|
||||
await rename_command_service(handler_full_name, new_name, aliases=aliases)
|
||||
except ValueError as exc:
|
||||
return Response().error(str(exc)).__dict__
|
||||
|
||||
payload = await _get_command_payload(handler_full_name)
|
||||
return Response().ok(payload).__dict__
|
||||
|
||||
|
||||
async def _get_command_payload(handler_full_name: str):
|
||||
commands = await list_commands()
|
||||
for cmd in commands:
|
||||
if cmd["handler_full_name"] == handler_full_name:
|
||||
return cmd
|
||||
return {}
|
||||
@@ -6,7 +6,7 @@ from typing import Any
|
||||
|
||||
from quart import request
|
||||
|
||||
from astrbot.core import astrbot_config, file_token_service, logger
|
||||
from astrbot.core import file_token_service, logger
|
||||
from astrbot.core.config.astrbot_config import AstrBotConfig
|
||||
from astrbot.core.config.default import (
|
||||
CONFIG_METADATA_2,
|
||||
@@ -21,7 +21,6 @@ from astrbot.core.platform.register import platform_cls_map, platform_registry
|
||||
from astrbot.core.provider import Provider
|
||||
from astrbot.core.provider.register import provider_registry
|
||||
from astrbot.core.star.star import star_registry
|
||||
from astrbot.core.utils.llm_metadata import LLM_METADATAS
|
||||
from astrbot.core.utils.webhook_utils import ensure_platform_webhook_config
|
||||
|
||||
from .route import Response, Route, RouteContext
|
||||
@@ -46,46 +45,6 @@ def try_cast(value: Any, type_: str):
|
||||
return None
|
||||
|
||||
|
||||
def _expect_type(value, expected_type, path_key, errors, expected_name=None):
|
||||
if not isinstance(value, expected_type):
|
||||
errors.append(
|
||||
f"错误的类型 {path_key}: 期望是 {expected_name or expected_type.__name__}, "
|
||||
f"得到了 {type(value).__name__}"
|
||||
)
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _validate_template_list(value, meta, path_key, errors, validate_fn):
|
||||
if not _expect_type(value, list, path_key, errors, "list"):
|
||||
return
|
||||
|
||||
templates = meta.get("templates")
|
||||
if not isinstance(templates, dict):
|
||||
templates = {}
|
||||
|
||||
for idx, item in enumerate(value):
|
||||
item_path = f"{path_key}[{idx}]"
|
||||
if not _expect_type(item, dict, item_path, errors, "dict"):
|
||||
continue
|
||||
|
||||
template_key = item.get("__template_key") or item.get("template")
|
||||
if not template_key:
|
||||
errors.append(f"缺少模板选择 {item_path}: 需要 __template_key")
|
||||
continue
|
||||
|
||||
template_meta = templates.get(template_key)
|
||||
if not template_meta:
|
||||
errors.append(f"未知模板 {item_path}: {template_key}")
|
||||
continue
|
||||
|
||||
validate_fn(
|
||||
item,
|
||||
template_meta.get("items", {}),
|
||||
path=f"{item_path}.",
|
||||
)
|
||||
|
||||
|
||||
def validate_config(data, schema: dict, is_core: bool) -> tuple[list[str], dict]:
|
||||
errors = []
|
||||
|
||||
@@ -101,11 +60,6 @@ def validate_config(data, schema: dict, is_core: bool) -> tuple[list[str], dict]
|
||||
if value is None:
|
||||
data[key] = DEFAULT_VALUE_MAP[meta["type"]]
|
||||
continue
|
||||
|
||||
if meta["type"] == "template_list":
|
||||
_validate_template_list(value, meta, f"{path}{key}", errors, validate)
|
||||
continue
|
||||
|
||||
if meta["type"] == "list" and not isinstance(value, list):
|
||||
errors.append(
|
||||
f"错误的类型 {path}{key}: 期望是 list, 得到了 {type(value).__name__}",
|
||||
@@ -225,157 +179,13 @@ class ConfigRoute(Route):
|
||||
"/config/provider/new": ("POST", self.post_new_provider),
|
||||
"/config/provider/update": ("POST", self.post_update_provider),
|
||||
"/config/provider/delete": ("POST", self.post_delete_provider),
|
||||
"/config/provider/template": ("GET", self.get_provider_template),
|
||||
"/config/provider/check_one": ("GET", self.check_one_provider_status),
|
||||
"/config/provider/list": ("GET", self.get_provider_config_list),
|
||||
"/config/provider/model_list": ("GET", self.get_provider_model_list),
|
||||
"/config/provider/get_embedding_dim": ("POST", self.get_embedding_dim),
|
||||
"/config/provider_sources/models": (
|
||||
"GET",
|
||||
self.get_provider_source_models,
|
||||
),
|
||||
"/config/provider_sources/update": (
|
||||
"POST",
|
||||
self.update_provider_source,
|
||||
),
|
||||
"/config/provider_sources/delete": (
|
||||
"POST",
|
||||
self.delete_provider_source,
|
||||
),
|
||||
}
|
||||
self.register_routes()
|
||||
|
||||
async def delete_provider_source(self):
|
||||
"""删除 provider_source,并更新关联的 providers"""
|
||||
post_data = await request.json
|
||||
if not post_data:
|
||||
return Response().error("缺少配置数据").__dict__
|
||||
|
||||
provider_source_id = post_data.get("id")
|
||||
if not provider_source_id:
|
||||
return Response().error("缺少 provider_source_id").__dict__
|
||||
|
||||
provider_sources = self.config.get("provider_sources", [])
|
||||
target_idx = next(
|
||||
(
|
||||
i
|
||||
for i, ps in enumerate(provider_sources)
|
||||
if ps.get("id") == provider_source_id
|
||||
),
|
||||
-1,
|
||||
)
|
||||
|
||||
if target_idx == -1:
|
||||
return Response().error("未找到对应的 provider source").__dict__
|
||||
|
||||
# 删除 provider_source
|
||||
del provider_sources[target_idx]
|
||||
|
||||
# 写回配置
|
||||
self.config["provider_sources"] = provider_sources
|
||||
|
||||
# 删除引用了该 provider_source 的 providers
|
||||
await self.core_lifecycle.provider_manager.delete_provider(
|
||||
provider_source_id=provider_source_id
|
||||
)
|
||||
|
||||
try:
|
||||
save_config(self.config, self.config, is_core=True)
|
||||
except Exception as e:
|
||||
logger.error(traceback.format_exc())
|
||||
return Response().error(str(e)).__dict__
|
||||
|
||||
return Response().ok(message="删除 provider source 成功").__dict__
|
||||
|
||||
async def update_provider_source(self):
|
||||
"""更新或新增 provider_source,并重载关联的 providers"""
|
||||
post_data = await request.json
|
||||
if not post_data:
|
||||
return Response().error("缺少配置数据").__dict__
|
||||
|
||||
new_source_config = post_data.get("config") or post_data
|
||||
original_id = post_data.get("original_id")
|
||||
if not original_id:
|
||||
return Response().error("缺少 original_id").__dict__
|
||||
|
||||
if not isinstance(new_source_config, dict):
|
||||
return Response().error("缺少或错误的配置数据").__dict__
|
||||
|
||||
# 确保配置中有 id 字段
|
||||
if not new_source_config.get("id"):
|
||||
new_source_config["id"] = original_id
|
||||
|
||||
provider_sources = self.config.get("provider_sources", [])
|
||||
|
||||
for ps in provider_sources:
|
||||
if ps.get("id") == new_source_config["id"] and ps.get("id") != original_id:
|
||||
return (
|
||||
Response()
|
||||
.error(
|
||||
f"Provider source ID '{new_source_config['id']}' exists already, please try another ID.",
|
||||
)
|
||||
.__dict__
|
||||
)
|
||||
|
||||
# 查找旧的 provider_source,若不存在则追加为新配置
|
||||
target_idx = next(
|
||||
(i for i, ps in enumerate(provider_sources) if ps.get("id") == original_id),
|
||||
-1,
|
||||
)
|
||||
|
||||
old_id = original_id
|
||||
if target_idx == -1:
|
||||
provider_sources.append(new_source_config)
|
||||
else:
|
||||
old_id = provider_sources[target_idx].get("id")
|
||||
provider_sources[target_idx] = new_source_config
|
||||
|
||||
# 更新引用了该 provider_source 的 providers
|
||||
affected_providers = []
|
||||
for provider in self.config.get("provider", []):
|
||||
if provider.get("provider_source_id") == old_id:
|
||||
provider["provider_source_id"] = new_source_config["id"]
|
||||
affected_providers.append(provider)
|
||||
|
||||
# 写回配置
|
||||
self.config["provider_sources"] = provider_sources
|
||||
|
||||
try:
|
||||
save_config(self.config, self.config, is_core=True)
|
||||
except Exception as e:
|
||||
logger.error(traceback.format_exc())
|
||||
return Response().error(str(e)).__dict__
|
||||
|
||||
# 重载受影响的 providers,使新的 source 配置生效
|
||||
reload_errors = []
|
||||
prov_mgr = self.core_lifecycle.provider_manager
|
||||
for provider in affected_providers:
|
||||
try:
|
||||
await prov_mgr.reload(provider)
|
||||
except Exception as e:
|
||||
logger.error(traceback.format_exc())
|
||||
reload_errors.append(f"{provider.get('id')}: {e}")
|
||||
|
||||
if reload_errors:
|
||||
return (
|
||||
Response()
|
||||
.error("更新成功,但部分提供商重载失败: " + ", ".join(reload_errors))
|
||||
.__dict__
|
||||
)
|
||||
|
||||
return Response().ok(message="更新 provider source 成功").__dict__
|
||||
|
||||
async def get_provider_template(self):
|
||||
config_schema = {
|
||||
"provider": CONFIG_METADATA_2["provider_group"]["metadata"]["provider"]
|
||||
}
|
||||
data = {
|
||||
"config_schema": config_schema,
|
||||
"providers": astrbot_config["provider"],
|
||||
"provider_sources": astrbot_config["provider_sources"],
|
||||
}
|
||||
return Response().ok(data=data).__dict__
|
||||
|
||||
async def get_uc_table(self):
|
||||
"""获取 UMOP 配置路由表"""
|
||||
return Response().ok({"routing": self.ucr.umop_to_conf_id}).__dict__
|
||||
@@ -623,25 +433,9 @@ class ConfigRoute(Route):
|
||||
return Response().error("缺少参数 provider_type").__dict__
|
||||
provider_type_ls = provider_type.split(",")
|
||||
provider_list = []
|
||||
ps = self.core_lifecycle.provider_manager.providers_config
|
||||
p_source_pt = {
|
||||
psrc["id"]: psrc["provider_type"]
|
||||
for psrc in self.core_lifecycle.provider_manager.provider_sources_config
|
||||
}
|
||||
for provider in ps:
|
||||
ps_id = provider.get("provider_source_id", None)
|
||||
if (
|
||||
ps_id
|
||||
and ps_id in p_source_pt
|
||||
and p_source_pt[ps_id] in provider_type_ls
|
||||
):
|
||||
# chat
|
||||
prov = self.core_lifecycle.provider_manager.get_merged_provider_config(
|
||||
provider
|
||||
)
|
||||
provider_list.append(prov)
|
||||
elif not ps_id and provider.get("provider_type", None) in provider_type_ls:
|
||||
# agent runner, embedding, etc
|
||||
astrbot_config = self.core_lifecycle.astrbot_config
|
||||
for provider in astrbot_config["provider"]:
|
||||
if provider.get("provider_type", None) in provider_type_ls:
|
||||
provider_list.append(provider)
|
||||
return Response().ok(provider_list).__dict__
|
||||
|
||||
@@ -664,18 +458,9 @@ class ConfigRoute(Route):
|
||||
|
||||
try:
|
||||
models = await provider.get_models()
|
||||
models = models or []
|
||||
|
||||
metadata_map = {}
|
||||
for model_id in models:
|
||||
meta = LLM_METADATAS.get(model_id)
|
||||
if meta:
|
||||
metadata_map[model_id] = meta
|
||||
|
||||
ret = {
|
||||
"models": models,
|
||||
"provider_id": provider_id,
|
||||
"model_metadata": metadata_map,
|
||||
}
|
||||
return Response().ok(ret).__dict__
|
||||
except Exception as e:
|
||||
@@ -737,104 +522,6 @@ class ConfigRoute(Route):
|
||||
logger.error(traceback.format_exc())
|
||||
return Response().error(f"获取嵌入维度失败: {e!s}").__dict__
|
||||
|
||||
async def get_provider_source_models(self):
|
||||
"""获取指定 provider_source 支持的模型列表
|
||||
|
||||
本质上会临时初始化一个 Provider 实例,调用 get_models() 获取模型列表,然后销毁实例
|
||||
"""
|
||||
provider_source_id = request.args.get("source_id")
|
||||
if not provider_source_id:
|
||||
return Response().error("缺少参数 source_id").__dict__
|
||||
|
||||
try:
|
||||
from astrbot.core.provider.register import provider_cls_map
|
||||
|
||||
# 从配置中查找对应的 provider_source
|
||||
provider_sources = self.config.get("provider_sources", [])
|
||||
provider_source = None
|
||||
for ps in provider_sources:
|
||||
if ps.get("id") == provider_source_id:
|
||||
provider_source = ps
|
||||
break
|
||||
|
||||
if not provider_source:
|
||||
return (
|
||||
Response()
|
||||
.error(f"未找到 ID 为 {provider_source_id} 的 provider_source")
|
||||
.__dict__
|
||||
)
|
||||
|
||||
# 获取 provider 类型
|
||||
provider_type = provider_source.get("type", None)
|
||||
if not provider_type:
|
||||
return Response().error("provider_source 缺少 type 字段").__dict__
|
||||
|
||||
try:
|
||||
self.core_lifecycle.provider_manager.dynamic_import_provider(
|
||||
provider_type
|
||||
)
|
||||
except ImportError as e:
|
||||
logger.error(traceback.format_exc())
|
||||
return Response().error(f"动态导入提供商适配器失败: {e!s}").__dict__
|
||||
|
||||
# 获取对应的 provider 类
|
||||
if provider_type not in provider_cls_map:
|
||||
return (
|
||||
Response()
|
||||
.error(f"未找到适用于 {provider_type} 的提供商适配器")
|
||||
.__dict__
|
||||
)
|
||||
|
||||
provider_metadata = provider_cls_map[provider_type]
|
||||
cls_type = provider_metadata.cls_type
|
||||
|
||||
if not cls_type:
|
||||
return Response().error(f"无法找到 {provider_type} 的类").__dict__
|
||||
|
||||
# 检查是否是 Provider 类型
|
||||
if not issubclass(cls_type, Provider):
|
||||
return (
|
||||
Response()
|
||||
.error(f"提供商 {provider_type} 不支持获取模型列表")
|
||||
.__dict__
|
||||
)
|
||||
|
||||
# 临时实例化 provider
|
||||
inst = cls_type(provider_source, {})
|
||||
|
||||
# 如果有 initialize 方法,调用它
|
||||
init_fn = getattr(inst, "initialize", None)
|
||||
if inspect.iscoroutinefunction(init_fn):
|
||||
await init_fn()
|
||||
|
||||
# 获取模型列表
|
||||
models = await inst.get_models()
|
||||
models = models or []
|
||||
|
||||
metadata_map = {}
|
||||
for model_id in models:
|
||||
meta = LLM_METADATAS.get(model_id)
|
||||
if meta:
|
||||
metadata_map[model_id] = meta
|
||||
|
||||
# 销毁实例(如果有 terminate 方法)
|
||||
terminate_fn = getattr(inst, "terminate", None)
|
||||
if inspect.iscoroutinefunction(terminate_fn):
|
||||
await terminate_fn()
|
||||
|
||||
logger.info(
|
||||
f"获取到 provider_source {provider_source_id} 的模型列表: {models}",
|
||||
)
|
||||
|
||||
return (
|
||||
Response()
|
||||
.ok({"models": models, "model_metadata": metadata_map})
|
||||
.__dict__
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(traceback.format_exc())
|
||||
return Response().error(f"获取模型列表失败: {e!s}").__dict__
|
||||
|
||||
async def get_platform_list(self):
|
||||
"""获取所有平台的列表"""
|
||||
platform_list = []
|
||||
@@ -846,15 +533,7 @@ class ConfigRoute(Route):
|
||||
data = await request.json
|
||||
config = data.get("config", None)
|
||||
conf_id = data.get("conf_id", None)
|
||||
|
||||
try:
|
||||
# 不更新 provider_sources, provider, platform
|
||||
# 这些配置有单独的接口进行更新
|
||||
if conf_id == "default":
|
||||
no_update_keys = ["provider_sources", "provider", "platform"]
|
||||
for key in no_update_keys:
|
||||
config[key] = self.acm.default_conf[key]
|
||||
|
||||
await self._save_astrbot_configs(config, conf_id)
|
||||
await self.core_lifecycle.reload_pipeline_scheduler(conf_id)
|
||||
return Response().ok(None, "保存成功~").__dict__
|
||||
@@ -894,30 +573,28 @@ class ConfigRoute(Route):
|
||||
|
||||
async def post_new_provider(self):
|
||||
new_provider_config = await request.json
|
||||
|
||||
self.config["provider"].append(new_provider_config)
|
||||
try:
|
||||
await self.core_lifecycle.provider_manager.create_provider(
|
||||
new_provider_config
|
||||
save_config(self.config, self.config, is_core=True)
|
||||
await self.core_lifecycle.provider_manager.load_provider(
|
||||
new_provider_config,
|
||||
)
|
||||
except Exception as e:
|
||||
return Response().error(str(e)).__dict__
|
||||
return Response().ok(None, "新增服务提供商配置成功").__dict__
|
||||
return Response().ok(None, "新增服务提供商配置成功~").__dict__
|
||||
|
||||
async def post_update_platform(self):
|
||||
update_platform_config = await request.json
|
||||
origin_platform_id = update_platform_config.get("id", None)
|
||||
platform_id = update_platform_config.get("id", None)
|
||||
new_config = update_platform_config.get("config", None)
|
||||
if not origin_platform_id or not new_config:
|
||||
if not platform_id or not new_config:
|
||||
return Response().error("参数错误").__dict__
|
||||
|
||||
if origin_platform_id != new_config.get("id", None):
|
||||
return Response().error("机器人名称不允许修改").__dict__
|
||||
|
||||
# 如果是支持统一 webhook 模式的平台,且启用了统一 webhook 模式,确保有 webhook_uuid
|
||||
ensure_platform_webhook_config(new_config)
|
||||
|
||||
for i, platform in enumerate(self.config["platform"]):
|
||||
if platform["id"] == origin_platform_id:
|
||||
if platform["id"] == platform_id:
|
||||
self.config["platform"][i] = new_config
|
||||
break
|
||||
else:
|
||||
@@ -932,15 +609,21 @@ class ConfigRoute(Route):
|
||||
|
||||
async def post_update_provider(self):
|
||||
update_provider_config = await request.json
|
||||
origin_provider_id = update_provider_config.get("id", None)
|
||||
provider_id = update_provider_config.get("id", None)
|
||||
new_config = update_provider_config.get("config", None)
|
||||
if not origin_provider_id or not new_config:
|
||||
if not provider_id or not new_config:
|
||||
return Response().error("参数错误").__dict__
|
||||
|
||||
for i, provider in enumerate(self.config["provider"]):
|
||||
if provider["id"] == provider_id:
|
||||
self.config["provider"][i] = new_config
|
||||
break
|
||||
else:
|
||||
return Response().error("未找到对应服务提供商").__dict__
|
||||
|
||||
try:
|
||||
await self.core_lifecycle.provider_manager.update_provider(
|
||||
origin_provider_id, new_config
|
||||
)
|
||||
save_config(self.config, self.config, is_core=True)
|
||||
await self.core_lifecycle.provider_manager.reload(new_config)
|
||||
except Exception as e:
|
||||
return Response().error(str(e)).__dict__
|
||||
return Response().ok(None, "更新成功,已经实时生效~").__dict__
|
||||
@@ -963,17 +646,19 @@ class ConfigRoute(Route):
|
||||
|
||||
async def post_delete_provider(self):
|
||||
provider_id = await request.json
|
||||
provider_id = provider_id.get("id", "")
|
||||
if not provider_id:
|
||||
return Response().error("缺少参数 id").__dict__
|
||||
|
||||
provider_id = provider_id.get("id")
|
||||
for i, provider in enumerate(self.config["provider"]):
|
||||
if provider["id"] == provider_id:
|
||||
del self.config["provider"][i]
|
||||
break
|
||||
else:
|
||||
return Response().error("未找到对应服务提供商").__dict__
|
||||
try:
|
||||
await self.core_lifecycle.provider_manager.delete_provider(
|
||||
provider_id=provider_id
|
||||
)
|
||||
save_config(self.config, self.config, is_core=True)
|
||||
await self.core_lifecycle.provider_manager.terminate_provider(provider_id)
|
||||
except Exception as e:
|
||||
return Response().error(str(e)).__dict__
|
||||
return Response().ok(None, "删除成功,已经实时生效。").__dict__
|
||||
return Response().ok(None, "删除成功,已经实时生效~").__dict__
|
||||
|
||||
async def get_llm_tools(self):
|
||||
"""获取函数调用工具。包含了本地加载的以及 MCP 服务的工具"""
|
||||
|
||||
@@ -1,9 +1,7 @@
|
||||
import json
|
||||
import traceback
|
||||
from datetime import datetime
|
||||
from io import BytesIO
|
||||
|
||||
from quart import request, send_file
|
||||
from quart import request
|
||||
|
||||
from astrbot.core import logger
|
||||
from astrbot.core.core_lifecycle import AstrBotCoreLifecycle
|
||||
@@ -32,7 +30,6 @@ class ConversationRoute(Route):
|
||||
"POST",
|
||||
self.update_history,
|
||||
),
|
||||
"/conversation/export": ("POST", self.export_conversations),
|
||||
}
|
||||
self.db_helper = db_helper
|
||||
self.conv_mgr = core_lifecycle.conversation_manager
|
||||
@@ -286,90 +283,3 @@ class ConversationRoute(Route):
|
||||
except Exception as e:
|
||||
logger.error(f"更新对话历史失败: {e!s}\n{traceback.format_exc()}")
|
||||
return Response().error(f"更新对话历史失败: {e!s}").__dict__
|
||||
|
||||
async def export_conversations(self):
|
||||
"""批量导出对话为 JSONL 格式"""
|
||||
try:
|
||||
data = await request.get_json()
|
||||
conversations_to_export = data.get("conversations", [])
|
||||
|
||||
if not conversations_to_export:
|
||||
return Response().error("导出列表不能为空").__dict__
|
||||
|
||||
# 收集所有对话的内容
|
||||
jsonl_lines = []
|
||||
exported_count = 0
|
||||
failed_items = []
|
||||
|
||||
for conv_info in conversations_to_export:
|
||||
user_id = conv_info.get("user_id")
|
||||
cid = conv_info.get("cid")
|
||||
|
||||
if not user_id or not cid:
|
||||
failed_items.append(
|
||||
f"user_id:{user_id}, cid:{cid} - 缺少必要参数",
|
||||
)
|
||||
continue
|
||||
|
||||
try:
|
||||
conversation = await self.conv_mgr.get_conversation(
|
||||
unified_msg_origin=user_id,
|
||||
conversation_id=cid,
|
||||
)
|
||||
|
||||
if not conversation:
|
||||
failed_items.append(
|
||||
f"user_id:{user_id}, cid:{cid} - 对话不存在"
|
||||
)
|
||||
continue
|
||||
|
||||
# 解析对话内容 (history is always a JSON string from _convert_conv_from_v2_to_v1)
|
||||
content = json.loads(conversation.history)
|
||||
|
||||
# 创建导出记录
|
||||
export_record = {
|
||||
"cid": cid,
|
||||
"user_id": user_id,
|
||||
"platform_id": conversation.platform_id,
|
||||
"title": conversation.title,
|
||||
"persona_id": conversation.persona_id,
|
||||
"created_at": conversation.created_at,
|
||||
"updated_at": conversation.updated_at,
|
||||
"content": content,
|
||||
}
|
||||
|
||||
# 将记录转换为 JSON 字符串并添加到 JSONL
|
||||
jsonl_lines.append(json.dumps(export_record, ensure_ascii=False))
|
||||
exported_count += 1
|
||||
|
||||
except Exception as e:
|
||||
failed_items.append(f"user_id:{user_id}, cid:{cid} - {e!s}")
|
||||
logger.error(
|
||||
f"导出对话失败: user_id={user_id}, cid={cid}, error={e!s}"
|
||||
)
|
||||
|
||||
if exported_count == 0:
|
||||
return Response().error("没有成功导出任何对话").__dict__
|
||||
|
||||
# 创建 JSONL 内容
|
||||
jsonl_content = "\n".join(jsonl_lines)
|
||||
|
||||
# 创建一个内存文件对象
|
||||
file_obj = BytesIO(jsonl_content.encode("utf-8"))
|
||||
file_obj.seek(0)
|
||||
|
||||
# 生成文件名
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
filename = f"astrbot_conversations_export_{timestamp}.jsonl"
|
||||
|
||||
# 返回文件流
|
||||
return await send_file(
|
||||
file_obj,
|
||||
mimetype="application/jsonl",
|
||||
as_attachment=True,
|
||||
attachment_filename=filename,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"批量导出对话失败: {e!s}\n{traceback.format_exc()}")
|
||||
return Response().error(f"批量导出对话失败: {e!s}").__dict__
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user