Yuerchu/disknext
1
0
Fork 0
Code Issues 10 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

feat: migrate ORM base to sqlmodel-ext, add file viewers and WOPI integration

Browse Source 
- Migrate SQLModel base classes, mixins, and database management to
  external sqlmodel-ext package; remove sqlmodels/base/, sqlmodels/mixin/,
  and sqlmodels/database.py
- Add file viewer/editor system with WOPI protocol support for
  collaborative editing (OnlyOffice, Collabora)
- Add enterprise edition license verification module (ee/)
- Add Dockerfile multi-stage build with Cython compilation support
- Add new dependencies: sqlmodel-ext, cryptography, whatthepatch

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
于小丘
2026-02-14 14:23:17 +08:00
parent 53b757de7a
commit eac0766e79
74 changed files with 4819 additions and 4837 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
sqlmodels/__init__.py
View File

@@ -115,6 +115,14 @@ from .storage_pack import StoragePack
from .tag import Tag, TagType
from .task import Task, TaskProps, TaskPropsBase, TaskStatus, TaskType, TaskSummary
from .webdav import WebDAV
from .file_app import (
FileApp, FileAppType, FileAppExtension, FileAppGroupLink, UserFileAppDefault,
# DTO
FileAppSummary, FileViewersResponse, SetDefaultViewerRequest, UserFileAppDefaultResponse,
FileAppCreateRequest, FileAppUpdateRequest, FileAppResponse, FileAppListResponse,
ExtensionUpdateRequest, GroupAccessUpdateRequest, WopiSessionResponse,
)
from .wopi import WopiFileInfo, WopiAccessTokenPayload
from .database_connection import DatabaseManager
@@ -131,5 +139,5 @@ from .model_base import (
AdminSummaryResponse,
)
# mixin 中的通用分页模型
from .mixin import ListResponse
# 通用分页模型
from sqlmodel_ext import ListResponse

3
sqlmodels/auth_identity.py
View File

@@ -10,8 +10,7 @@ from uuid import UUID
from sqlmodel import Field, Relationship, UniqueConstraint
from .base import SQLModelBase
from .mixin import UUIDTableBaseMixin
from sqlmodel_ext import SQLModelBase, UUIDTableBaseMixin
if TYPE_CHECKING:
from .user import User

657
sqlmodels/base/README.md
View File

@@ -1,657 +0,0 @@
# SQLModels Base Module
This module provides `SQLModelBase`, the root base class for all SQLModel models in this project. It includes a custom metaclass with automatic type injection and Python 3.14 compatibility.
**Note**: Table base classes (`TableBaseMixin`, `UUIDTableBaseMixin`) and polymorphic utilities have been migrated to the [`sqlmodels.mixin`](../mixin/README.md) module. See the mixin documentation for CRUD operations, polymorphic inheritance patterns, and pagination utilities.
## Table of Contents
- [Overview](#overview)
- [Migration Notice](#migration-notice)
- [Python 3.14 Compatibility](#python-314-compatibility)
- [Core Component](#core-component)
- [SQLModelBase](#sqlmodelbase)
- [Metaclass Features](#metaclass-features)
- [Automatic sa_type Injection](#automatic-sa_type-injection)
- [Table Configuration](#table-configuration)
- [Polymorphic Support](#polymorphic-support)
- [Custom Types Integration](#custom-types-integration)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)
## Overview
The `sqlmodels.base` module provides `SQLModelBase`, the foundational base class for all SQLModel models. It features:
- **Smart metaclass** that automatically extracts and injects SQLAlchemy types from type annotations
- **Python 3.14 compatibility** through comprehensive PEP 649/749 support
- **Flexible configuration** through class parameters and automatic docstring support
- **Type-safe annotations** with automatic validation
All models in this project should directly or indirectly inherit from `SQLModelBase`.
---
## Migration Notice
As of the recent refactoring, the following components have been moved:
| Component | Old Location | New Location |
|-----------|-------------|--------------|
| `TableBase` → `TableBaseMixin` | `sqlmodels.base` | `sqlmodels.mixin` |
| `UUIDTableBase` → `UUIDTableBaseMixin` | `sqlmodels.base` | `sqlmodels.mixin` |
| `PolymorphicBaseMixin` | `sqlmodels.base` | `sqlmodels.mixin` |
| `create_subclass_id_mixin()` | `sqlmodels.base` | `sqlmodels.mixin` |
| `AutoPolymorphicIdentityMixin` | `sqlmodels.base` | `sqlmodels.mixin` |
| `TableViewRequest` | `sqlmodels.base` | `sqlmodels.mixin` |
| `now()`, `now_date()` | `sqlmodels.base` | `sqlmodels.mixin` |
**Update your imports**:
```python
# ❌ Old (deprecated)
from sqlmodels.base import TableBase, UUIDTableBase
# ✅ New (correct)
from sqlmodels.mixin import TableBaseMixin, UUIDTableBaseMixin
```
For detailed documentation on table mixins, CRUD operations, and polymorphic patterns, see [`sqlmodels/mixin/README.md`](../mixin/README.md).
---
## Python 3.14 Compatibility
### Overview
This module provides full compatibility with **Python 3.14's PEP 649** (Deferred Evaluation of Annotations) and **PEP 749** (making it the default).
**Key Changes in Python 3.14**:
- Annotations are no longer evaluated at class definition time
- Type hints are stored as deferred code objects
- `__annotate__` function generates annotations on demand
- Forward references become `ForwardRef` objects
### Implementation Strategy
We use **`typing.get_type_hints()`** as the universal annotations resolver:
```python
def _resolve_annotations(attrs: dict[str, Any]) -> tuple[...]:
# Create temporary proxy class
temp_cls = type('AnnotationProxy', (object,), dict(attrs))
# Use get_type_hints with include_extras=True
evaluated = get_type_hints(
temp_cls,
globalns=module_globals,
localns=localns,
include_extras=True # Preserve Annotated metadata
)
return dict(evaluated), {}, module_globals, localns
```
**Why `get_type_hints()`?**
- ✅ Works across Python 3.10-3.14+
- ✅ Handles PEP 649 automatically
- ✅ Preserves `Annotated` metadata (with `include_extras=True`)
- ✅ Resolves forward references
- ✅ Recommended by Python documentation
### SQLModel Compatibility Patch
**Problem**: SQLModel's `get_sqlalchemy_type()` doesn't recognize custom types with `__sqlmodel_sa_type__` attribute.
**Solution**: Global monkey-patch that checks for SQLAlchemy type before falling back to original logic:
```python
if sys.version_info >= (3, 14):
def _patched_get_sqlalchemy_type(field):
annotation = getattr(field, 'annotation', None)
if annotation is not None:
# Priority 1: Check __sqlmodel_sa_type__ attribute
# Handles NumpyVector[dims, dtype] and similar custom types
if hasattr(annotation, '__sqlmodel_sa_type__'):
return annotation.__sqlmodel_sa_type__
# Priority 2: Check Annotated metadata
if get_origin(annotation) is Annotated:
for metadata in get_args(annotation)[1:]:
if hasattr(metadata, '__sqlmodel_sa_type__'):
return metadata.__sqlmodel_sa_type__
# ... handle ForwardRef, ClassVar, etc.
return _original_get_sqlalchemy_type(field)
```
### Supported Patterns
#### Pattern 1: Direct Custom Type Usage
```python
from sqlmodels.sqlmodel_types.dialects.postgresql import NumpyVector
from sqlmodels.mixin import UUIDTableBaseMixin
class SpeakerInfo(UUIDTableBaseMixin, table=True):
embedding: NumpyVector[256, np.float32]
"""Voice embedding - sa_type automatically extracted"""
```
#### Pattern 2: Annotated Wrapper
```python
from typing import Annotated
from sqlmodels.mixin import UUIDTableBaseMixin
EmbeddingVector = Annotated[np.ndarray, NumpyVector[256, np.float32]]
class SpeakerInfo(UUIDTableBaseMixin, table=True):
embedding: EmbeddingVector
```
#### Pattern 3: Array Type
```python
from sqlmodels.sqlmodel_types.dialects.postgresql import Array
from sqlmodels.mixin import TableBaseMixin
class ServerConfig(TableBaseMixin, table=True):
protocols: Array[ProtocolEnum]
"""Allowed protocols - sa_type from Array handler"""
```
### Migration from Python 3.13
**No code changes required!** The implementation is transparent:
- Uses `typing.get_type_hints()` which works in both Python 3.13 and 3.14
- Custom types already use `__sqlmodel_sa_type__` attribute
- Monkey-patch only activates for Python 3.14+
---
## Core Component
### SQLModelBase
`SQLModelBase` is the root base class for all SQLModel models. It uses a custom metaclass (`__DeclarativeMeta`) that provides advanced features beyond standard SQLModel capabilities.
**Key Features**:
- Automatic `use_attribute_docstrings` configuration (use docstrings instead of `Field(description=...)`)
- Automatic `validate_by_name` configuration
- Custom metaclass for sa_type injection and polymorphic setup
- Integration with Pydantic v2
- Python 3.14 PEP 649 compatibility
**Usage**:
```python
from sqlmodels.base import SQLModelBase
class UserBase(SQLModelBase):
name: str
"""User's display name"""
email: str
"""User's email address"""
```
**Important Notes**:
- Use **docstrings** for field descriptions, not `Field(description=...)`
- Do NOT override `model_config` in subclasses (it's already configured in SQLModelBase)
- This class should be used for non-table models (DTOs, request/response models)
**For table models**, use mixins from `sqlmodels.mixin`:
- `TableBaseMixin` - Integer primary key with timestamps
- `UUIDTableBaseMixin` - UUID primary key with timestamps
See [`sqlmodels/mixin/README.md`](../mixin/README.md) for complete table mixin documentation.
---
## Metaclass Features
### Automatic sa_type Injection
The metaclass automatically extracts SQLAlchemy types from custom type annotations, enabling clean syntax for complex database types.
**Before** (verbose):
```python
from sqlmodels.sqlmodel_types.dialects.postgresql.numpy_vector import _NumpyVectorSQLAlchemyType
from sqlmodels.mixin import UUIDTableBaseMixin
class SpeakerInfo(UUIDTableBaseMixin, table=True):
embedding: np.ndarray = Field(
sa_type=_NumpyVectorSQLAlchemyType(256, np.float32)
)
```
**After** (clean):
```python
from sqlmodels.sqlmodel_types.dialects.postgresql import NumpyVector
from sqlmodels.mixin import UUIDTableBaseMixin
class SpeakerInfo(UUIDTableBaseMixin, table=True):
embedding: NumpyVector[256, np.float32]
"""Speaker voice embedding"""
```
**How It Works**:
The metaclass uses a three-tier detection strategy:
1. **Direct `__sqlmodel_sa_type__` attribute** (Priority 1)
```python
if hasattr(annotation, '__sqlmodel_sa_type__'):
return annotation.__sqlmodel_sa_type__
```
2. **Annotated metadata** (Priority 2)
```python
# For Annotated[np.ndarray, NumpyVector[256, np.float32]]
if get_origin(annotation) is typing.Annotated:
for item in metadata_items:
if hasattr(item, '__sqlmodel_sa_type__'):
return item.__sqlmodel_sa_type__
```
3. **Pydantic Core Schema metadata** (Priority 3)
```python
schema = annotation.__get_pydantic_core_schema__(...)
if schema['metadata'].get('sa_type'):
return schema['metadata']['sa_type']
```
After extracting `sa_type`, the metaclass:
- Creates `Field(sa_type=sa_type)` if no Field is defined
- Injects `sa_type` into existing Field if not already set
- Respects explicit `Field(sa_type=...)` (no override)
**Supported Patterns**:
```python
from sqlmodels.mixin import UUIDTableBaseMixin
# Pattern 1: Direct usage (recommended)
class Model(UUIDTableBaseMixin, table=True):
embedding: NumpyVector[256, np.float32]
# Pattern 2: With Field constraints
class Model(UUIDTableBaseMixin, table=True):
embedding: NumpyVector[256, np.float32] = Field(nullable=False)
# Pattern 3: Annotated wrapper
EmbeddingVector = Annotated[np.ndarray, NumpyVector[256, np.float32]]
class Model(UUIDTableBaseMixin, table=True):
embedding: EmbeddingVector
# Pattern 4: Explicit sa_type (override)
class Model(UUIDTableBaseMixin, table=True):
embedding: NumpyVector[256, np.float32] = Field(
sa_type=_NumpyVectorSQLAlchemyType(128, np.float16)
)
```
### Table Configuration
The metaclass provides smart defaults and flexible configuration:
**Automatic `table=True`**:
```python
# Classes inheriting from TableBaseMixin automatically get table=True
from sqlmodels.mixin import UUIDTableBaseMixin
class MyModel(UUIDTableBaseMixin): # table=True is automatic
pass
```
**Convenient mapper arguments**:
```python
# Instead of verbose __mapper_args__
from sqlmodels.mixin import UUIDTableBaseMixin
class MyModel(
UUIDTableBaseMixin,
polymorphic_on='_polymorphic_name',
polymorphic_abstract=True
):
pass
# Equivalent to:
class MyModel(UUIDTableBaseMixin):
__mapper_args__ = {
'polymorphic_on': '_polymorphic_name',
'polymorphic_abstract': True
}
```
**Smart merging**:
```python
# Dictionary and keyword arguments are merged
from sqlmodels.mixin import UUIDTableBaseMixin
class MyModel(
UUIDTableBaseMixin,
mapper_args={'version_id_col': 'version'},
polymorphic_on='type' # Merged into __mapper_args__
):
pass
```
### Polymorphic Support
The metaclass supports SQLAlchemy's joined table inheritance through convenient parameters:
**Supported parameters**:
- `polymorphic_on`: Discriminator column name
- `polymorphic_identity`: Identity value for this class
- `polymorphic_abstract`: Whether this is an abstract base
- `table_args`: SQLAlchemy table arguments
- `table_name`: Override table name (becomes `__tablename__`)
**For complete polymorphic inheritance patterns**, including `PolymorphicBaseMixin`, `create_subclass_id_mixin()`, and `AutoPolymorphicIdentityMixin`, see [`sqlmodels/mixin/README.md`](../mixin/README.md).
---
## Custom Types Integration
### Using NumpyVector
The `NumpyVector` type demonstrates automatic sa_type injection:
```python
from sqlmodels.sqlmodel_types.dialects.postgresql import NumpyVector
from sqlmodels.mixin import UUIDTableBaseMixin
import numpy as np
class SpeakerInfo(UUIDTableBaseMixin, table=True):
embedding: NumpyVector[256, np.float32]
"""Speaker voice embedding - sa_type automatically injected"""
```
**How NumpyVector works**:
```python
# NumpyVector[dims, dtype] returns a class with:
class _NumpyVectorType:
__sqlmodel_sa_type__ = _NumpyVectorSQLAlchemyType(dimensions, dtype)
@classmethod
def __get_pydantic_core_schema__(cls, source_type, handler):
return handler.generate_schema(np.ndarray)
```
This dual approach ensures:
1. Metaclass can extract `sa_type` via `__sqlmodel_sa_type__`
2. Pydantic can validate as `np.ndarray`
### Creating Custom SQLAlchemy Types
To create types that work with automatic injection, provide one of:
**Option 1: `__sqlmodel_sa_type__` attribute** (preferred):
```python
from sqlalchemy import TypeDecorator, String
class UpperCaseString(TypeDecorator):
impl = String
def process_bind_param(self, value, dialect):
return value.upper() if value else value
class UpperCaseType:
__sqlmodel_sa_type__ = UpperCaseString()
@classmethod
def __get_pydantic_core_schema__(cls, source_type, handler):
return core_schema.str_schema()
# Usage
from sqlmodels.mixin import UUIDTableBaseMixin
class MyModel(UUIDTableBaseMixin, table=True):
code: UpperCaseType # Automatically uses UpperCaseString()
```
**Option 2: Pydantic metadata with sa_type**:
```python
def __get_pydantic_core_schema__(self, source_type, handler):
return core_schema.json_or_python_schema(
json_schema=core_schema.str_schema(),
python_schema=core_schema.str_schema(),
metadata={'sa_type': UpperCaseString()}
)
```
**Option 3: Using Annotated**:
```python
from typing import Annotated
from sqlmodels.mixin import UUIDTableBaseMixin
UpperCase = Annotated[str, UpperCaseType()]
class MyModel(UUIDTableBaseMixin, table=True):
code: UpperCase
```
---
## Best Practices
### 1. Inherit from correct base classes
```python
from sqlmodels.base import SQLModelBase
from sqlmodels.mixin import TableBaseMixin, UUIDTableBaseMixin
# ✅ For non-table models (DTOs, requests, responses)
class UserBase(SQLModelBase):
name: str
# ✅ For table models with UUID primary key
class User(UserBase, UUIDTableBaseMixin, table=True):
email: str
# ✅ For table models with custom primary key
class LegacyUser(TableBaseMixin, table=True):
id: int = Field(primary_key=True)
username: str
```
### 2. Use docstrings for field descriptions
```python
from sqlmodels.mixin import UUIDTableBaseMixin
# ✅ Recommended
class User(UUIDTableBaseMixin, table=True):
name: str
"""User's display name"""
# ❌ Avoid
class User(UUIDTableBaseMixin, table=True):
name: str = Field(description="User's display name")
```
**Why?** SQLModelBase has `use_attribute_docstrings=True`, so docstrings automatically become field descriptions in API docs.
### 3. Leverage automatic sa_type injection
```python
from sqlmodels.mixin import UUIDTableBaseMixin
# ✅ Clean and recommended
class SpeakerInfo(UUIDTableBaseMixin, table=True):
embedding: NumpyVector[256, np.float32]
"""Voice embedding"""
# ❌ Verbose and unnecessary
class SpeakerInfo(UUIDTableBaseMixin, table=True):
embedding: np.ndarray = Field(
sa_type=_NumpyVectorSQLAlchemyType(256, np.float32)
)
```
### 4. Follow polymorphic naming conventions
See [`sqlmodels/mixin/README.md`](../mixin/README.md) for complete polymorphic inheritance patterns using `PolymorphicBaseMixin`, `create_subclass_id_mixin()`, and `AutoPolymorphicIdentityMixin`.
### 5. Separate Base, Parent, and Implementation classes
```python
from abc import ABC, abstractmethod
from sqlmodels.base import SQLModelBase
from sqlmodels.mixin import UUIDTableBaseMixin, PolymorphicBaseMixin
# ✅ Recommended structure
class ASRBase(SQLModelBase):
"""Pure data fields, no table"""
name: str
base_url: str
class ASR(ASRBase, UUIDTableBaseMixin, PolymorphicBaseMixin, ABC):
"""Abstract parent with table"""
@abstractmethod
async def transcribe(self, audio: bytes) -> str:
pass
class WhisperASR(ASR, table=True):
"""Concrete implementation"""
model_size: str
async def transcribe(self, audio: bytes) -> str:
# Implementation
pass
```
**Why?**
- Base class can be reused for DTOs
- Parent class defines the polymorphic hierarchy
- Implementation classes are clean and focused
---
## Troubleshooting
### Issue: ValueError: X has no matching SQLAlchemy type
**Solution**: Ensure your custom type provides `__sqlmodel_sa_type__` attribute or proper Pydantic metadata with `sa_type`.
```python
# ✅ Provide __sqlmodel_sa_type__
class MyType:
__sqlmodel_sa_type__ = MyCustomSQLAlchemyType()
```
### Issue: Can't generate DDL for NullType()
**Symptoms**: Error during table creation saying a column has `NullType`.
**Root Cause**: Custom type's `sa_type` not detected by SQLModel.
**Solution**:
1. Ensure your type has `__sqlmodel_sa_type__` class attribute
2. Check that the monkey-patch is active (`sys.version_info >= (3, 14)`)
3. Verify type annotation is correct (not a string forward reference)
```python
from sqlmodels.mixin import UUIDTableBaseMixin
# ✅ Correct
class Model(UUIDTableBaseMixin, table=True):
data: NumpyVector[256, np.float32] # __sqlmodel_sa_type__ detected
# ❌ Wrong (string annotation)
class Model(UUIDTableBaseMixin, table=True):
data: 'NumpyVector[256, np.float32]' # sa_type lost
```
### Issue: Polymorphic identity conflicts
**Symptoms**: SQLAlchemy raises errors about duplicate polymorphic identities.
**Solution**:
1. Check that each concrete class has a unique identity
2. Use `AutoPolymorphicIdentityMixin` for automatic naming
3. Manually specify identity if needed:
```python
class MyClass(Parent, polymorphic_identity='unique.name', table=True):
pass
```
### Issue: Python 3.14 annotation errors
**Symptoms**: Errors related to `__annotations__` or type resolution.
**Solution**: The implementation uses `get_type_hints()` which handles PEP 649 automatically. If issues persist:
1. Check for manual `__annotations__` manipulation (avoid it)
2. Ensure all types are properly imported
3. Avoid `from __future__ import annotations` (can cause SQLModel issues)
### Issue: Polymorphic and CRUD-related errors
For issues related to polymorphic inheritance, CRUD operations, or table mixins, see the troubleshooting section in [`sqlmodels/mixin/README.md`](../mixin/README.md).
---
## Implementation Details
For developers modifying this module:
**Core files**:
- `sqlmodel_base.py` - Contains `__DeclarativeMeta` and `SQLModelBase`
- `../mixin/table.py` - Contains `TableBaseMixin` and `UUIDTableBaseMixin`
- `../mixin/polymorphic.py` - Contains `PolymorphicBaseMixin`, `create_subclass_id_mixin()`, and `AutoPolymorphicIdentityMixin`
**Key functions in this module**:
1. **`_resolve_annotations(attrs: dict[str, Any])`**
- Uses `typing.get_type_hints()` for Python 3.14 compatibility
- Returns tuple: `(annotations, annotation_strings, globalns, localns)`
- Preserves `Annotated` metadata with `include_extras=True`
2. **`_extract_sa_type_from_annotation(annotation: Any) -> Any | None`**
- Extracts SQLAlchemy type from type annotations
- Supports `__sqlmodel_sa_type__`, `Annotated`, and Pydantic core schema
- Called by metaclass during class creation
3. **`_patched_get_sqlalchemy_type(field)`** (Python 3.14+)
- Global monkey-patch for SQLModel
- Checks `__sqlmodel_sa_type__` before falling back to original logic
- Handles custom types like `NumpyVector` and `Array`
4. **`__DeclarativeMeta.__new__()`**
- Processes class definition parameters
- Injects `sa_type` into field definitions
- Sets up `__mapper_args__`, `__table_args__`, etc.
- Handles Python 3.14 annotations via `get_type_hints()`
**Metaclass processing order**:
1. Check if class should be a table (`_has_table_mixin`)
2. Collect `__mapper_args__` from kwargs and explicit dict
3. Process `table_args`, `table_name`, `abstract` parameters
4. Resolve annotations using `get_type_hints()`
5. For each field, try to extract `sa_type` and inject into Field
6. Call parent metaclass with cleaned kwargs
For table mixin implementation details, see [`sqlmodels/mixin/README.md`](../mixin/README.md).
---
## See Also
**Project Documentation**:
- [SQLModel Mixin Documentation](../mixin/README.md) - Table mixins, CRUD operations, polymorphic patterns
- [Project Coding Standards (CLAUDE.md)](/mnt/c/Users/Administrator/PycharmProjects/emoecho-backend-server/CLAUDE.md)
- [Custom SQLModel Types Guide](/mnt/c/Users/Administrator/PycharmProjects/emoecho-backend-server/sqlmodels/sqlmodel_types/README.md)
**External References**:
- [SQLAlchemy Joined Table Inheritance](https://docs.sqlalchemy.org/en/20/orm/inheritance.html#joined-table-inheritance)
- [Pydantic V2 Documentation](https://docs.pydantic.dev/latest/)
- [SQLModel Documentation](https://sqlmodel.tiangolo.com/)
- [PEP 649: Deferred Evaluation of Annotations](https://peps.python.org/pep-0649/)
- [PEP 749: Implementing PEP 649](https://peps.python.org/pep-0749/)
- [Python Annotations Best Practices](https://docs.python.org/3/howto/annotations.html)

12
sqlmodels/base/__init__.py
View File

@@ -1,12 +0,0 @@
"""
SQLModel 基础模块
包含：
- SQLModelBase: 所有 SQLModel 类的基类（真正的基类）
注意：
TableBase, UUIDTableBase, PolymorphicBaseMixin 已迁移到 sqlmodels.mixin
为了避免循环导入，此处不再重新导出它们
请直接从 sqlmodels.mixin 导入这些类
"""
from .sqlmodel_base import SQLModelBase

846
sqlmodels/base/sqlmodel_base.py
View File

@@ -1,846 +0,0 @@
import sys
import typing
from typing import Any, Mapping, get_args, get_origin, get_type_hints
from pydantic import ConfigDict
from pydantic.fields import FieldInfo
from pydantic_core import PydanticUndefined as Undefined
from sqlalchemy.orm import Mapped
from sqlmodel import Field, SQLModel
from sqlmodel.main import SQLModelMetaclass
# Python 3.14+ PEP 649支持
if sys.version_info >= (3, 14):
import annotationlib
# 全局Monkey-patch: 修复SQLModel在Python 3.14上的兼容性问题
import sqlmodel.main
_original_get_sqlalchemy_type = sqlmodel.main.get_sqlalchemy_type
def _patched_get_sqlalchemy_type(field):
"""
修复SQLModel的get_sqlalchemy_type函数，处理Python 3.14的类型问题。
问题：
1. ForwardRef对象（来自Relationship字段）会导致issubclass错误
2. typing._GenericAlias对象（如ClassVar[T]）也会导致同样问题
3. list/dict等泛型类型在没有Field/Relationship时可能导致错误
4. Mapped类型在Python 3.14下可能出现在annotation中
5. Annotated类型可能包含sa_type metadata（如Array[T]）
6. 自定义类型（如NumpyVector）有__sqlmodel_sa_type__属性
7. Pydantic已处理的Annotated类型会将metadata存储在field.metadata中
解决：
- 优先检查field.metadata中的__get_pydantic_core_schema__（Pydantic已处理的情况）
- 检测__sqlmodel_sa_type__属性（NumpyVector等）
- 检测Relationship/ClassVar等返回None
- 对于Annotated类型，尝试提取sa_type metadata
- 其他情况调用原始函数
"""
# 优先检查 field.metadata（Pydantic已处理Annotated类型的情况）
# 当使用 Array[T] 或 Annotated[T, metadata] 时，Pydantic会将metadata存储在这里
metadata = getattr(field, 'metadata', None)
if metadata:
# metadata是一个列表，包含所有Annotated的元数据项
for metadata_item in metadata:
# 检查metadata_item是否有__get_pydantic_core_schema__方法
if hasattr(metadata_item, '__get_pydantic_core_schema__'):
try:
# 调用获取schema
schema = metadata_item.__get_pydantic_core_schema__(None, None)
# 检查schema的metadata中是否有sa_type
if isinstance(schema, dict) and 'metadata' in schema:
sa_type = schema['metadata'].get('sa_type')
if sa_type is not None:
return sa_type
except (TypeError, AttributeError, KeyError):
# Pydantic schema获取可能失败（类型不匹配、缺少属性等）
# 这是正常情况，继续检查下一个metadata项
pass
annotation = getattr(field, 'annotation', None)
if annotation is not None:
# 优先检查 __sqlmodel_sa_type__ 属性
# 这处理 NumpyVector[dims, dtype] 等自定义类型
if hasattr(annotation, '__sqlmodel_sa_type__'):
return annotation.__sqlmodel_sa_type__
# 检查自定义类型（如JSON100K）的 __get_pydantic_core_schema__ 方法
# 这些类型在schema的metadata中定义sa_type
if hasattr(annotation, '__get_pydantic_core_schema__'):
try:
# 调用获取schema（传None作为handler，因为我们只需要metadata）
schema = annotation.__get_pydantic_core_schema__(annotation, lambda x: None)
# 检查schema的metadata中是否有sa_type
if isinstance(schema, dict) and 'metadata' in schema:
sa_type = schema['metadata'].get('sa_type')
if sa_type is not None:
return sa_type
except (TypeError, AttributeError, KeyError):
# Schema获取失败，继续其他检查
pass
anno_type_name = type(annotation).__name__
# ForwardRef: Relationship字段的annotation
if anno_type_name == 'ForwardRef':
return None
# AnnotatedAlias: 检查是否有sa_type metadata（如Array[T]）
if anno_type_name == 'AnnotatedAlias' or anno_type_name == '_AnnotatedAlias':
from typing import get_origin, get_args
import typing
# 尝试提取Annotated的metadata
if hasattr(typing, 'get_args'):
args = get_args(annotation)
# args[0]是实际类型，args[1:]是metadata
for metadata in args[1:]:
# 检查metadata是否有__get_pydantic_core_schema__方法
if hasattr(metadata, '__get_pydantic_core_schema__'):
try:
# 调用获取schema
schema = metadata.__get_pydantic_core_schema__(None, None)
# 检查schema中是否有sa_type
if isinstance(schema, dict) and 'metadata' in schema:
sa_type = schema['metadata'].get('sa_type')
if sa_type is not None:
return sa_type
except (TypeError, AttributeError, KeyError):
# Annotated metadata的schema获取可能失败
# 这是正常的类型检查过程，继续检查下一个metadata
pass
# _GenericAlias或GenericAlias: typing泛型类型
if anno_type_name in ('_GenericAlias', 'GenericAlias'):
from typing import get_origin
import typing
origin = get_origin(annotation)
# ClassVar必须跳过
if origin is typing.ClassVar:
return None
# list/dict/tuple/set等内置泛型，如果字段没有明确的Field或Relationship，也跳过
# 这通常意味着它是Relationship字段或类变量
if origin in (list, dict, tuple, set):
# 检查field_info是否存在且有意义
# Relationship字段会有特殊的field_info
field_info = getattr(field, 'field_info', None)
if field_info is None:
return None
# Mapped: SQLAlchemy 2.0的Mapped类型，SQLModel不应该处理
# 这可能是从父类继承的字段或Python 3.14注解处理的副作用
# 检查类型名称和annotation的字符串表示
if 'Mapped' in anno_type_name or 'Mapped' in str(annotation):
return None
# 检查annotation是否是Mapped类或其实例
try:
from sqlalchemy.orm import Mapped as SAMapped
# 检查origin（对于Mapped[T]这种泛型）
from typing import get_origin
if get_origin(annotation) is SAMapped:
return None
# 检查类型本身
if annotation is SAMapped or isinstance(annotation, type) and issubclass(annotation, SAMapped):
return None
except (ImportError, TypeError):
# 如果SQLAlchemy没有Mapped或检查失败，继续
pass
# 其他情况正常处理
return _original_get_sqlalchemy_type(field)
sqlmodel.main.get_sqlalchemy_type = _patched_get_sqlalchemy_type
# 第二个Monkey-patch: 修复继承表类中InstrumentedAttribute作为默认值的问题
# 在Python 3.14 + SQLModel组合下，当子类（如SMSBaoProvider）继承父类（如VerificationCodeProvider）时，
# 父类的关系字段（如server_config）会在子类的model_fields中出现，
# 但其default值错误地设置为InstrumentedAttribute对象，而不是None
# 这导致实例化时尝试设置InstrumentedAttribute为字段值，触发SQLAlchemy内部错误
import sqlmodel._compat as _compat
from sqlalchemy.orm import attributes as _sa_attributes
_original_sqlmodel_table_construct = _compat.sqlmodel_table_construct
def _patched_sqlmodel_table_construct(self_instance, values):
"""
修复sqlmodel_table_construct，跳过InstrumentedAttribute默认值
问题：
- 继承自polymorphic基类的表类（如FishAudioTTS, SMSBaoProvider）
- 其model_fields中的继承字段default值为InstrumentedAttribute
- 原函数尝试将InstrumentedAttribute设置为字段值
- SQLAlchemy无法处理，抛出 '_sa_instance_state' 错误
解决：
- 只设置用户提供的值和非InstrumentedAttribute默认值
- InstrumentedAttribute默认值跳过（让SQLAlchemy自己处理）
"""
cls = type(self_instance)
# 收集要设置的字段值
fields_to_set = {}
for name, field in cls.model_fields.items():
# 如果用户提供了值，直接使用
if name in values:
fields_to_set[name] = values[name]
continue
# 否则检查默认值
# 跳过InstrumentedAttribute默认值 - 这些是继承字段的错误默认值
if isinstance(field.default, _sa_attributes.InstrumentedAttribute):
continue
# 使用正常的默认值
if field.default is not Undefined:
fields_to_set[name] = field.default
elif field.default_factory is not None:
fields_to_set[name] = field.get_default(call_default_factory=True)
# 设置属性 - 只设置非InstrumentedAttribute值
for key, value in fields_to_set.items():
if not isinstance(value, _sa_attributes.InstrumentedAttribute):
setattr(self_instance, key, value)
# 设置Pydantic内部属性
object.__setattr__(self_instance, '__pydantic_fields_set__', set(values.keys()))
if not cls.__pydantic_root_model__:
_extra = None
if cls.model_config.get('extra') == 'allow':
_extra = {}
for k, v in values.items():
if k not in cls.model_fields:
_extra[k] = v
object.__setattr__(self_instance, '__pydantic_extra__', _extra)
if cls.__pydantic_post_init__:
self_instance.model_post_init(None)
elif not cls.__pydantic_root_model__:
object.__setattr__(self_instance, '__pydantic_private__', None)
# 设置关系
for key in self_instance.__sqlmodel_relationships__:
value = values.get(key, Undefined)
if value is not Undefined:
setattr(self_instance, key, value)
return self_instance
_compat.sqlmodel_table_construct = _patched_sqlmodel_table_construct
else:
annotationlib = None
def _extract_sa_type_from_annotation(annotation: Any) -> Any | None:
"""
从类型注解中提取SQLAlchemy类型。
支持以下形式：
1. NumpyVector[256, np.float32] - 直接使用类型（有__sqlmodel_sa_type__属性）
2. Annotated[np.ndarray, NumpyVector[256, np.float32]] - Annotated包装
3. 任何有__get_pydantic_core_schema__且返回metadata['sa_type']的类型
Args:
annotation: 字段的类型注解
Returns:
提取到的SQLAlchemy类型，如果没有则返回None
"""
# 方法1：直接检查类型本身是否有__sqlmodel_sa_type__属性
# 这涵盖了 NumpyVector[256, np.float32] 这种直接使用的情况
if hasattr(annotation, '__sqlmodel_sa_type__'):
return annotation.__sqlmodel_sa_type__
# 方法2：检查是否为Annotated类型
if get_origin(annotation) is typing.Annotated:
# 获取元数据项（跳过第一个实际类型参数）
args = get_args(annotation)
if len(args) >= 2:
metadata_items = args[1:] # 第一个是实际类型，后面都是元数据
# 遍历元数据，查找包含sa_type的项
for item in metadata_items:
# 检查元数据项是否有__sqlmodel_sa_type__属性
if hasattr(item, '__sqlmodel_sa_type__'):
return item.__sqlmodel_sa_type__
# 检查是否有__get_pydantic_core_schema__方法
if hasattr(item, '__get_pydantic_core_schema__'):
try:
# 调用该方法获取core schema
schema = item.__get_pydantic_core_schema__(
annotation,
lambda x: None # 虚拟handler
)
# 检查schema的metadata中是否有sa_type
if isinstance(schema, dict) and 'metadata' in schema:
sa_type = schema['metadata'].get('sa_type')
if sa_type is not None:
return sa_type
except (TypeError, AttributeError, KeyError, ValueError):
# Pydantic core schema获取可能失败：
# - TypeError: 参数不匹配
# - AttributeError: metadata不存在
# - KeyError: schema结构不符合预期
# - ValueError: 无效的类型定义
# 这是正常的类型探测过程，继续检查下一个metadata项
pass
# 方法3：检查类型本身是否有__get_pydantic_core_schema__
# （虽然NumpyVector已经在方法1处理，但这是通用的fallback）
if hasattr(annotation, '__get_pydantic_core_schema__'):
try:
schema = annotation.__get_pydantic_core_schema__(
annotation,
lambda x: None # 虚拟handler
)
if isinstance(schema, dict) and 'metadata' in schema:
sa_type = schema['metadata'].get('sa_type')
if sa_type is not None:
return sa_type
except (TypeError, AttributeError, KeyError, ValueError):
# 类型本身的schema获取失败
# 这是正常的fallback机制，annotation可能不支持此协议
pass
return None
def _resolve_annotations(attrs: dict[str, Any]) -> tuple[
dict[str, Any],
dict[str, str],
Mapping[str, Any],
Mapping[str, Any],
]:
"""
Resolve annotations from a class namespace with Python 3.14 (PEP 649) support.
This helper prefers evaluated annotations (Format.VALUE) so that `typing.Annotated`
metadata and custom types remain accessible. Forward references that cannot be
evaluated are replaced with typing.ForwardRef placeholders to avoid aborting the
whole resolution process.
"""
raw_annotations = attrs.get('__annotations__') or {}
try:
base_annotations = dict(raw_annotations)
except TypeError:
base_annotations = {}
module_name = attrs.get('__module__')
module_globals: dict[str, Any]
if module_name and module_name in sys.modules:
module_globals = dict(sys.modules[module_name].__dict__)
else:
module_globals = {}
module_globals.setdefault('__builtins__', __builtins__)
localns: dict[str, Any] = dict(attrs)
try:
temp_cls = type('AnnotationProxy', (object,), dict(attrs))
temp_cls.__module__ = module_name
extras_kw = {'include_extras': True} if sys.version_info >= (3, 10) else {}
evaluated = get_type_hints(
temp_cls,
globalns=module_globals,
localns=localns,
**extras_kw,
)
except (NameError, AttributeError, TypeError, RecursionError):
# get_type_hints可能失败的原因：
# - NameError: 前向引用无法解析（类型尚未定义）
# - AttributeError: 模块或类型不存在
# - TypeError: 无效的类型注解
# - RecursionError: 循环依赖的类型定义
# 这是正常情况，回退到原始注解字符串
evaluated = base_annotations
return dict(evaluated), {}, module_globals, localns
def _evaluate_annotation_from_string(
field_name: str,
annotation_strings: dict[str, str],
current_type: Any,
globalns: Mapping[str, Any],
localns: Mapping[str, Any],
) -> Any:
"""
Attempt to re-evaluate the original annotation string for a field.
This is used as a fallback when the resolved annotation lost its metadata
(e.g., Annotated wrappers) and we need to recover custom sa_type data.
"""
if not annotation_strings:
return current_type
expr = annotation_strings.get(field_name)
if not expr or not isinstance(expr, str):
return current_type
try:
return eval(expr, globalns, localns)
except (NameError, SyntaxError, AttributeError, TypeError):
# eval可能失败的原因：
# - NameError: 类型名称在namespace中不存在
# - SyntaxError: 注解字符串有语法错误
# - AttributeError: 访问不存在的模块属性
# - TypeError: 无效的类型表达式
# 这是正常的fallback机制，返回当前已解析的类型
return current_type
class __DeclarativeMeta(SQLModelMetaclass):
"""
一个智能的混合模式元类，它提供了灵活性和清晰度：
1. **自动设置 `table=True`**: 如果一个类继承了 `TableBaseMixin`，则自动应用 `table=True`。
2. **明确的字典参数**: 支持 `mapper_args={...}`, `table_args={...}`, `table_name='...'`。
3. **便捷的关键字参数**: 支持最常见的 mapper 参数作为顶级关键字（如 `polymorphic_on`）。
4. **智能合并**: 当字典和关键字同时提供时，会自动合并，且关键字参数有更高优先级。
"""
_KNOWN_MAPPER_KEYS = {
"polymorphic_on",
"polymorphic_identity",
"polymorphic_abstract",
"version_id_col",
"concrete",
}
def __new__(cls, name, bases, attrs, **kwargs):
# 1. 约定优于配置：自动设置 table=True
is_intended_as_table = any(getattr(b, '_has_table_mixin', False) for b in bases)
if is_intended_as_table and 'table' not in kwargs:
kwargs['table'] = True
# 2. 智能合并 __mapper_args__
collected_mapper_args = {}
# 首先，处理明确的 mapper_args 字典 (优先级较低)
if 'mapper_args' in kwargs:
collected_mapper_args.update(kwargs.pop('mapper_args'))
# 其次，处理便捷的关键字参数 (优先级更高)
for key in cls._KNOWN_MAPPER_KEYS:
if key in kwargs:
# .pop() 获取值并移除，避免传递给父类
collected_mapper_args[key] = kwargs.pop(key)
# 如果收集到了任何 mapper 参数，则更新到类的属性中
if collected_mapper_args:
existing = attrs.get('__mapper_args__', {}).copy()
existing.update(collected_mapper_args)
attrs['__mapper_args__'] = existing
# 3. 处理其他明确的参数
if 'table_args' in kwargs:
attrs['__table_args__'] = kwargs.pop('table_args')
if 'table_name' in kwargs:
attrs['__tablename__'] = kwargs.pop('table_name')
if 'abstract' in kwargs:
attrs['__abstract__'] = kwargs.pop('abstract')
# 4. 从Annotated元数据中提取sa_type并注入到Field
# 重要：必须在调用父类__new__之前处理，因为SQLModel会消费annotations
#
# Python 3.14兼容性问题：
# - SQLModel在Python 3.14上会因为ClassVar[T]类型而崩溃（issubclass错误）
# - 我们必须在SQLModel看到annotations之前过滤掉ClassVar字段
# - 虽然PEP 749建议不修改__annotations__，但这是修复SQLModel bug的必要措施
#
# 获取annotations的策略：
# - Python 3.14+: 优先从__annotate__获取（如果存在）
# - fallback: 从__annotations__读取（如果存在）
# - 最终fallback: 空字典
annotations, annotation_strings, eval_globals, eval_locals = _resolve_annotations(attrs)
if annotations:
attrs['__annotations__'] = annotations
if annotationlib is not None:
# 在Python 3.14中禁用descriptor，转为普通dict
attrs['__annotate__'] = None
for field_name, field_type in annotations.items():
field_type = _evaluate_annotation_from_string(
field_name,
annotation_strings,
field_type,
eval_globals,
eval_locals,
)
# 跳过字符串或ForwardRef类型注解，让SQLModel自己处理
if isinstance(field_type, str) or isinstance(field_type, typing.ForwardRef):
continue
# 跳过特殊类型的字段
origin = get_origin(field_type)
# 跳过 ClassVar 字段 - 它们不是数据库字段
if origin is typing.ClassVar:
continue
# 跳过 Mapped 字段 - SQLAlchemy 2.0+ 的声明式字段，已经有 mapped_column
if origin is Mapped:
continue
# 尝试从注解中提取sa_type
sa_type = _extract_sa_type_from_annotation(field_type)
if sa_type is not None:
# 检查字段是否已有Field定义
field_value = attrs.get(field_name, Undefined)
if field_value is Undefined:
# 没有Field定义，创建一个新的Field并注入sa_type
attrs[field_name] = Field(sa_type=sa_type)
elif isinstance(field_value, FieldInfo):
# 已有Field定义，检查是否已设置sa_type
# 注意：只有在未设置时才注入，尊重显式配置
# SQLModel使用Undefined作为"未设置"的标记
if not hasattr(field_value, 'sa_type') or field_value.sa_type is Undefined:
field_value.sa_type = sa_type
# 如果field_value是其他类型（如默认值），不处理
# SQLModel会在后续处理中将其转换为Field
# 5. 调用父类的 __new__ 方法，传入被清理过的 kwargs
result = super().__new__(cls, name, bases, attrs, **kwargs)
# 6. 修复：在联表继承场景下，继承父类的 __sqlmodel_relationships__
# SQLModel 为每个 table=True 的类创建新的空 __sqlmodel_relationships__
# 这导致子类丢失父类的关系定义，触发错误的 Column 创建
# 必须在 super().__new__() 之后修复，因为 SQLModel 会覆盖我们预设的值
if kwargs.get('table', False):
for base in bases:
if hasattr(base, '__sqlmodel_relationships__'):
for rel_name, rel_info in base.__sqlmodel_relationships__.items():
# 只继承子类没有重新定义的关系
if rel_name not in result.__sqlmodel_relationships__:
result.__sqlmodel_relationships__[rel_name] = rel_info
# 同时修复被错误创建的 Column - 恢复为父类的 relationship
if hasattr(base, rel_name):
base_attr = getattr(base, rel_name)
setattr(result, rel_name, base_attr)
# 7. 检测：禁止子类重定义父类的 Relationship 字段
# 子类重定义同名的 Relationship 字段会导致 SQLAlchemy 关系映射混乱，
# 应该在类定义时立即报错，而不是在运行时出现难以调试的问题。
for base in bases:
parent_relationships = getattr(base, '__sqlmodel_relationships__', {})
for rel_name in parent_relationships:
# 检查当前类是否在 attrs 中重新定义了这个关系字段
if rel_name in attrs:
raise TypeError(
f"类 {name} 不允许重定义父类 {base.__name__} 的 Relationship 字段 '{rel_name}'。"
f"如需修改关系配置，请在父类中修改。"
)
# 8. 修复：从 model_fields/__pydantic_fields__ 中移除 Relationship 字段
# SQLModel 0.0.27 bug：子类会错误地继承父类的 Relationship 字段到 model_fields
# 这导致 Pydantic 尝试为 Relationship 字段生成 schema，因为类型是
# Mapped[list['Character']] 这种前向引用，Pydantic 无法解析，
# 导致 __pydantic_complete__ = False
#
# 修复策略：
# - 检查类的 __sqlmodel_relationships__ 属性
# - 从 model_fields 和 __pydantic_fields__ 中移除这些字段
# - Relationship 字段由 SQLAlchemy 管理，不需要 Pydantic 参与
relationships = getattr(result, '__sqlmodel_relationships__', {})
if relationships:
model_fields = getattr(result, 'model_fields', {})
pydantic_fields = getattr(result, '__pydantic_fields__', {})
fields_removed = False
for rel_name in relationships:
if rel_name in model_fields:
del model_fields[rel_name]
fields_removed = True
if rel_name in pydantic_fields:
del pydantic_fields[rel_name]
fields_removed = True
# 如果移除了字段，重新构建 Pydantic 模式
# 注意：只在有字段被移除时才 rebuild，避免不必要的开销
if fields_removed and hasattr(result, 'model_rebuild'):
result.model_rebuild(force=True)
return result
def __init__(
cls,
classname: str,
bases: tuple[type, ...],
dict_: dict[str, typing.Any],
**kw: typing.Any,
) -> None:
"""
重写 SQLModel 的 __init__ 以支持联表继承（Joined Table Inheritance）
SQLModel 原始行为：
- 如果任何基类是表模型，则不调用 DeclarativeMeta.__init__
- 这阻止了子类创建自己的表
修复逻辑：
- 检测联表继承场景（子类有自己的 __tablename__ 且有外键指向父表）
- 强制调用 DeclarativeMeta.__init__ 来创建子表
"""
from sqlmodel.main import is_table_model_class, DeclarativeMeta, ModelMetaclass
# 检查是否是表模型
if not is_table_model_class(cls):
ModelMetaclass.__init__(cls, classname, bases, dict_, **kw)
return
# 检查是否有基类是表模型
base_is_table = any(is_table_model_class(base) for base in bases)
if not base_is_table:
# 没有基类是表模型，走正常的 SQLModel 流程
# 处理关系字段
cls._setup_relationships()
DeclarativeMeta.__init__(cls, classname, bases, dict_, **kw)
return
# 关键：检测联表继承场景
# 条件：
# 1. 当前类的 __tablename__ 与父类不同（表示需要新表）
# 2. 当前类有字段带有 foreign_key 指向父表
current_tablename = getattr(cls, '__tablename__', None)
# 查找父表信息
parent_table = None
parent_tablename = None
for base in bases:
if is_table_model_class(base) and hasattr(base, '__tablename__'):
parent_tablename = base.__tablename__
break
# 检查是否有不同的 tablename
has_different_tablename = (
current_tablename is not None
and parent_tablename is not None
and current_tablename != parent_tablename
)
# 检查是否有外键字段指向父表的主键
# 注意：由于字段合并，我们需要检查直接基类的 model_fields
# 而不是当前类的合并后的 model_fields
has_fk_to_parent = False
def _normalize_tablename(name: str) -> str:
"""标准化表名以进行比较（移除下划线，转小写）"""
return name.replace('_', '').lower()
def _fk_matches_parent(fk_str: str, parent_table: str) -> bool:
"""检查 FK 字符串是否指向父表"""
if not fk_str or not parent_table:
return False
# FK 格式: "tablename.column" 或 "schema.tablename.column"
parts = fk_str.split('.')
if len(parts) >= 2:
fk_table = parts[-2] # 取倒数第二个作为表名
# 标准化比较（处理下划线差异）
return _normalize_tablename(fk_table) == _normalize_tablename(parent_table)
return False
if has_different_tablename and parent_tablename:
# 首先检查当前类的 model_fields
for field_name, field_info in cls.model_fields.items():
fk = getattr(field_info, 'foreign_key', None)
if fk is not None and isinstance(fk, str) and _fk_matches_parent(fk, parent_tablename):
has_fk_to_parent = True
break
# 如果没找到，检查直接基类的 model_fields（解决 mixin 字段被覆盖的问题）
if not has_fk_to_parent:
for base in bases:
if hasattr(base, 'model_fields'):
for field_name, field_info in base.model_fields.items():
fk = getattr(field_info, 'foreign_key', None)
if fk is not None and isinstance(fk, str) and _fk_matches_parent(fk, parent_tablename):
has_fk_to_parent = True
break
if has_fk_to_parent:
break
is_joined_inheritance = has_different_tablename and has_fk_to_parent
if is_joined_inheritance:
# 联表继承：需要创建子表
# 修复外键字段：由于字段合并，外键信息可能丢失
# 需要从基类的 mixin 中找回外键信息，并重建列
from sqlalchemy import Column, ForeignKey, inspect as sa_inspect
from sqlalchemy.dialects.postgresql import UUID as SA_UUID
from sqlalchemy.exc import NoInspectionAvailable
from sqlalchemy.orm.attributes import InstrumentedAttribute
# 联表继承：子表只应该有 id（FK 到父表）+ 子类特有的字段
# 所有继承自祖先表的列都不应该在子表中重复创建
# 收集整个继承链中所有祖先表的列名（这些列不应该在子表中重复）
# 需要遍历整个 MRO，因为可能是多级继承（如 Tool -> Function -> GetWeatherFunction）
ancestor_column_names: set[str] = set()
for ancestor in cls.__mro__:
if ancestor is cls:
continue # 跳过当前类
if is_table_model_class(ancestor):
try:
# 使用 inspect() 获取 mapper 的公开属性
# 源码确认: mapper.local_table 是公开属性 (mapper.py:979-998)
mapper = sa_inspect(ancestor)
for col in mapper.local_table.columns:
# 跳过 _polymorphic_name 列（鉴别器，由根父表管理）
if col.name.startswith('_polymorphic'):
continue
ancestor_column_names.add(col.name)
except NoInspectionAvailable:
continue
# 找到子类自己定义的字段（不在父类中的）
child_own_fields: set[str] = set()
for field_name in cls.model_fields:
# 检查这个字段是否是在当前类直接定义的（不是继承的）
# 通过检查父类是否有这个字段来判断
is_inherited = False
for base in bases:
if hasattr(base, 'model_fields') and field_name in base.model_fields:
is_inherited = True
break
if not is_inherited:
child_own_fields.add(field_name)
# 从子类类属性中移除父表已有的列定义
# 这样 SQLAlchemy 就不会在子表中创建这些列
fk_field_name = None
for base in bases:
if hasattr(base, 'model_fields'):
for field_name, field_info in base.model_fields.items():
fk = getattr(field_info, 'foreign_key', None)
pk = getattr(field_info, 'primary_key', False)
if fk is not None and isinstance(fk, str) and _fk_matches_parent(fk, parent_tablename):
fk_field_name = field_name
# 找到了外键字段，重建它
# 创建一个新的 Column 对象包含外键约束
new_col = Column(
field_name,
SA_UUID(as_uuid=True),
ForeignKey(fk),
primary_key=pk if pk else False
)
setattr(cls, field_name, new_col)
break
else:
continue
break
# 移除继承自祖先表的列属性（除了 FK/PK 和子类自己的字段）
# 这防止 SQLAlchemy 在子表中创建重复列
# 注意：在 __init__ 阶段，列是 Column 对象，不是 InstrumentedAttribute
for col_name in ancestor_column_names:
if col_name == fk_field_name:
continue # 保留 FK/PK 列（子表的主键，同时是父表的外键）
if col_name == 'id':
continue # id 会被 FK 字段覆盖
if col_name in child_own_fields:
continue # 保留子类自己定义的字段
# 检查类属性是否是 Column 或 InstrumentedAttribute
if col_name in cls.__dict__:
attr = cls.__dict__[col_name]
# Column 对象或 InstrumentedAttribute 都需要删除
if isinstance(attr, (Column, InstrumentedAttribute)):
try:
delattr(cls, col_name)
except AttributeError:
pass
# 找到子类自己定义的关系（不在父类中的）
# 继承的关系会从父类自动获取，只需要设置子类新增的关系
child_own_relationships: set[str] = set()
for rel_name in cls.__sqlmodel_relationships__:
is_inherited = False
for base in bases:
if hasattr(base, '__sqlmodel_relationships__') and rel_name in base.__sqlmodel_relationships__:
is_inherited = True
break
if not is_inherited:
child_own_relationships.add(rel_name)
# 只为子类自己定义的新关系调用关系设置
if child_own_relationships:
cls._setup_relationships(only_these=child_own_relationships)
# 强制调用 DeclarativeMeta.__init__
DeclarativeMeta.__init__(cls, classname, bases, dict_, **kw)
else:
# 非联表继承：单表继承或正常 Pydantic 模型
ModelMetaclass.__init__(cls, classname, bases, dict_, **kw)
def _setup_relationships(cls, only_these: set[str] | None = None) -> None:
"""
设置 SQLAlchemy 关系字段（从 SQLModel 源码复制）
Args:
only_these: 如果提供，只设置这些关系（用于 joined table inheritance 子类）
如果为 None，设置所有关系（默认行为）
"""
from sqlalchemy.orm import relationship, Mapped
from sqlalchemy import inspect
from sqlmodel.main import get_relationship_to
from typing import get_origin
for rel_name, rel_info in cls.__sqlmodel_relationships__.items():
# 如果指定了 only_these，只设置这些关系
if only_these is not None and rel_name not in only_these:
continue
if rel_info.sa_relationship:
setattr(cls, rel_name, rel_info.sa_relationship)
continue
raw_ann = cls.__annotations__[rel_name]
origin: typing.Any = get_origin(raw_ann)
if origin is Mapped:
ann = raw_ann.__args__[0]
else:
ann = raw_ann
cls.__annotations__[rel_name] = Mapped[ann]
relationship_to = get_relationship_to(
name=rel_name, rel_info=rel_info, annotation=ann
)
rel_kwargs: dict[str, typing.Any] = {}
if rel_info.back_populates:
rel_kwargs["back_populates"] = rel_info.back_populates
if rel_info.cascade_delete:
rel_kwargs["cascade"] = "all, delete-orphan"
if rel_info.passive_deletes:
rel_kwargs["passive_deletes"] = rel_info.passive_deletes
if rel_info.link_model:
ins = inspect(rel_info.link_model)
local_table = getattr(ins, "local_table")
if local_table is None:
raise RuntimeError(
f"Couldn't find secondary table for {rel_info.link_model}"
)
rel_kwargs["secondary"] = local_table
rel_args: list[typing.Any] = []
if rel_info.sa_relationship_args:
rel_args.extend(rel_info.sa_relationship_args)
if rel_info.sa_relationship_kwargs:
rel_kwargs.update(rel_info.sa_relationship_kwargs)
rel_value = relationship(relationship_to, *rel_args, **rel_kwargs)
setattr(cls, rel_name, rel_value)
class SQLModelBase(SQLModel, metaclass=__DeclarativeMeta):
"""此类必须和TableBase系列类搭配使用"""
model_config = ConfigDict(use_attribute_docstrings=True, validate_by_name=True)

2
sqlmodels/color.py
View File

@@ -1,6 +1,6 @@
from enum import StrEnum
from .base import SQLModelBase
from sqlmodel_ext import SQLModelBase
class ChromaticColor(StrEnum):

33
sqlmodels/database.py
View File

@@ -1,33 +0,0 @@
from sqlmodel import SQLModel
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
from sqlmodel.ext.asyncio.session import AsyncSession
from utils.conf import appmeta
from sqlalchemy.orm import sessionmaker
from typing import AsyncGenerator
ASYNC_DATABASE_URL = appmeta.database_url
engine: AsyncEngine = create_async_engine(
ASYNC_DATABASE_URL,
echo=appmeta.debug,
connect_args={
"check_same_thread": False
} if ASYNC_DATABASE_URL.startswith("sqlite") else {},
future=True,
# pool_size=POOL_SIZE,
# max_overflow=64,
)
_async_session_factory = sessionmaker(engine, class_=AsyncSession)
async def get_session() -> AsyncGenerator[AsyncSession, None]:
async with _async_session_factory() as session:
yield session
async def init_db(
url: str = ASYNC_DATABASE_URL
):
"""创建数据库结构"""
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)

3
sqlmodels/download.py
View File

@@ -4,8 +4,7 @@ from uuid import UUID
from sqlmodel import Field, Relationship, UniqueConstraint, Index
from .base import SQLModelBase
from .mixin import UUIDTableBaseMixin, TableBaseMixin
from sqlmodel_ext import SQLModelBase, UUIDTableBaseMixin, TableBaseMixin
if TYPE_CHECKING:
from .user import User

409
sqlmodels/file_app.py Normal file
View File

@@ -0,0 +1,409 @@
"""
文件查看器应用模块
提供文件预览应用选择器系统的数据模型和 DTO。
类似 Android 的"使用什么应用打开"机制：
- 管理员注册应用（内置/iframe/WOPI）
- 用户按扩展名查询可用查看器
- 用户可设置"始终使用"偏好
- 支持用户组级别的访问控制
架构：
FileApp (应用注册表)
├── FileAppExtension (扩展名关联)
├── FileAppGroupLink (用户组访问控制)
└── UserFileAppDefault (用户默认偏好)
"""
from enum import StrEnum
from typing import TYPE_CHECKING
from uuid import UUID
from sqlmodel import Field, Relationship, UniqueConstraint
from sqlmodel_ext import SQLModelBase, TableBaseMixin, UUIDTableBaseMixin
if TYPE_CHECKING:
from .group import Group
# ==================== 枚举 ====================
class FileAppType(StrEnum):
"""文件应用类型"""
BUILTIN = "builtin"
"""前端内置查看器（如 pdf.js, Monaco）"""
IFRAME = "iframe"
"""iframe 内嵌第三方服务"""
WOPI = "wopi"
"""WOPI 协议（OnlyOffice / Collabora）"""
# ==================== Link 表 ====================
class FileAppGroupLink(SQLModelBase, table=True):
"""应用-用户组访问控制关联表"""
app_id: UUID = Field(foreign_key="fileapp.id", primary_key=True, ondelete="CASCADE")
"""关联的应用UUID"""
group_id: UUID = Field(foreign_key="group.id", primary_key=True, ondelete="CASCADE")
"""关联的用户组UUID"""
# ==================== DTO 模型 ====================
class FileAppSummary(SQLModelBase):
"""查看器列表项 DTO，用于选择器弹窗"""
id: UUID
"""应用UUID"""
name: str
"""应用名称"""
app_key: str
"""应用唯一标识"""
type: FileAppType
"""应用类型"""
icon: str | None = None
"""图标名称/URL"""
description: str | None = None
"""应用描述"""
iframe_url_template: str | None = None
"""iframe URL 模板"""
wopi_editor_url_template: str | None = None
"""WOPI 编辑器 URL 模板"""
class FileViewersResponse(SQLModelBase):
"""查看器查询响应 DTO"""
viewers: list[FileAppSummary] = []
"""可用查看器列表（已按 priority 排序）"""
default_viewer_id: UUID | None = None
"""用户默认查看器UUID（如果已设置"始终使用"）"""
class SetDefaultViewerRequest(SQLModelBase):
"""设置默认查看器请求 DTO"""
extension: str = Field(max_length=20)
"""文件扩展名（小写，无点号）"""
app_id: UUID
"""应用UUID"""
class UserFileAppDefaultResponse(SQLModelBase):
"""用户默认查看器响应 DTO"""
id: UUID
"""记录UUID"""
extension: str
"""扩展名"""
app: FileAppSummary
"""关联的应用摘要"""
class FileAppCreateRequest(SQLModelBase):
"""管理员创建应用请求 DTO"""
name: str = Field(max_length=100)
"""应用名称"""
app_key: str = Field(max_length=50)
"""应用唯一标识"""
type: FileAppType
"""应用类型"""
icon: str | None = Field(default=None, max_length=255)
"""图标名称/URL"""
description: str | None = Field(default=None, max_length=500)
"""应用描述"""
is_enabled: bool = True
"""是否启用"""
is_restricted: bool = False
"""是否限制用户组访问"""
iframe_url_template: str | None = Field(default=None, max_length=1024)
"""iframe URL 模板"""
wopi_discovery_url: str | None = Field(default=None, max_length=512)
"""WOPI 发现端点 URL"""
wopi_editor_url_template: str | None = Field(default=None, max_length=1024)
"""WOPI 编辑器 URL 模板"""
extensions: list[str] = []
"""关联的扩展名列表"""
allowed_group_ids: list[UUID] = []
"""允许访问的用户组UUID列表"""
class FileAppUpdateRequest(SQLModelBase):
"""管理员更新应用请求 DTO（所有字段可选）"""
name: str | None = Field(default=None, max_length=100)
"""应用名称"""
app_key: str | None = Field(default=None, max_length=50)
"""应用唯一标识"""
type: FileAppType | None = None
"""应用类型"""
icon: str | None = Field(default=None, max_length=255)
"""图标名称/URL"""
description: str | None = Field(default=None, max_length=500)
"""应用描述"""
is_enabled: bool | None = None
"""是否启用"""
is_restricted: bool | None = None
"""是否限制用户组访问"""
iframe_url_template: str | None = Field(default=None, max_length=1024)
"""iframe URL 模板"""
wopi_discovery_url: str | None = Field(default=None, max_length=512)
"""WOPI 发现端点 URL"""
wopi_editor_url_template: str | None = Field(default=None, max_length=1024)
"""WOPI 编辑器 URL 模板"""
class FileAppResponse(SQLModelBase):
"""管理员应用详情响应 DTO"""
id: UUID
"""应用UUID"""
name: str
"""应用名称"""
app_key: str
"""应用唯一标识"""
type: FileAppType
"""应用类型"""
icon: str | None = None
"""图标名称/URL"""
description: str | None = None
"""应用描述"""
is_enabled: bool = True
"""是否启用"""
is_restricted: bool = False
"""是否限制用户组访问"""
iframe_url_template: str | None = None
"""iframe URL 模板"""
wopi_discovery_url: str | None = None
"""WOPI 发现端点 URL"""
wopi_editor_url_template: str | None = None
"""WOPI 编辑器 URL 模板"""
extensions: list[str] = []
"""关联的扩展名列表"""
allowed_group_ids: list[UUID] = []
"""允许访问的用户组UUID列表"""
@classmethod
def from_app(
cls,
app: "FileApp",
extensions: list["FileAppExtension"],
group_links: list[FileAppGroupLink],
) -> "FileAppResponse":
"""从 ORM 对象构建 DTO"""
return cls(
id=app.id,
name=app.name,
app_key=app.app_key,
type=app.type,
icon=app.icon,
description=app.description,
is_enabled=app.is_enabled,
is_restricted=app.is_restricted,
iframe_url_template=app.iframe_url_template,
wopi_discovery_url=app.wopi_discovery_url,
wopi_editor_url_template=app.wopi_editor_url_template,
extensions=[ext.extension for ext in extensions],
allowed_group_ids=[link.group_id for link in group_links],
)
class FileAppListResponse(SQLModelBase):
"""管理员应用列表响应 DTO"""
apps: list[FileAppResponse] = []
"""应用列表"""
total: int = 0
"""总数"""
class ExtensionUpdateRequest(SQLModelBase):
"""扩展名全量替换请求 DTO"""
extensions: list[str]
"""扩展名列表（小写，无点号）"""
class GroupAccessUpdateRequest(SQLModelBase):
"""用户组权限全量替换请求 DTO"""
group_ids: list[UUID]
"""允许访问的用户组UUID列表"""
class WopiSessionResponse(SQLModelBase):
"""WOPI 会话响应 DTO"""
wopi_src: str
"""WOPI 源 URL"""
access_token: str
"""WOPI 访问令牌"""
access_token_ttl: int
"""令牌过期时间戳（毫秒，WOPI 规范要求）"""
editor_url: str
"""完整的编辑器 URL"""
# ==================== 数据库模型 ====================
class FileApp(SQLModelBase, UUIDTableBaseMixin):
"""文件查看器应用注册表"""
name: str = Field(max_length=100)
"""应用名称"""
app_key: str = Field(max_length=50, unique=True, index=True)
"""应用唯一标识，前端路由用"""
type: FileAppType
"""应用类型"""
icon: str | None = Field(default=None, max_length=255)
"""图标名称/URL"""
description: str | None = Field(default=None, max_length=500)
"""应用描述"""
is_enabled: bool = True
"""是否启用"""
is_restricted: bool = False
"""是否限制用户组访问"""
iframe_url_template: str | None = Field(default=None, max_length=1024)
"""iframe URL 模板，支持 {file_url} 占位符"""
wopi_discovery_url: str | None = Field(default=None, max_length=512)
"""WOPI 客户端发现端点 URL"""
wopi_editor_url_template: str | None = Field(default=None, max_length=1024)
"""WOPI 编辑器 URL 模板，支持 {wopi_src} {access_token} {access_token_ttl}"""
# 关系
extensions: list["FileAppExtension"] = Relationship(
back_populates="app",
sa_relationship_kwargs={"cascade": "all, delete-orphan"}
)
user_defaults: list["UserFileAppDefault"] = Relationship(
back_populates="app",
sa_relationship_kwargs={"cascade": "all, delete-orphan"}
)
allowed_groups: list["Group"] = Relationship(
link_model=FileAppGroupLink,
)
def to_summary(self) -> FileAppSummary:
"""转换为摘要 DTO"""
return FileAppSummary(
id=self.id,
name=self.name,
app_key=self.app_key,
type=self.type,
icon=self.icon,
description=self.description,
iframe_url_template=self.iframe_url_template,
wopi_editor_url_template=self.wopi_editor_url_template,
)
class FileAppExtension(SQLModelBase, TableBaseMixin):
"""扩展名关联表"""
__table_args__ = (
UniqueConstraint("app_id", "extension", name="uq_fileappextension_app_extension"),
)
app_id: UUID = Field(foreign_key="fileapp.id", index=True, ondelete="CASCADE")
"""关联的应用UUID"""
extension: str = Field(max_length=20, index=True)
"""扩展名（小写，无点号）"""
priority: int = Field(default=0, ge=0)
"""排序优先级（越小越优先）"""
# 关系
app: FileApp = Relationship(back_populates="extensions")
class UserFileAppDefault(SQLModelBase, UUIDTableBaseMixin):
"""用户"始终使用"偏好"""
__table_args__ = (
UniqueConstraint("user_id", "extension", name="uq_userfileappdefault_user_extension"),
)
user_id: UUID = Field(foreign_key="user.id", index=True, ondelete="CASCADE")
"""用户UUID"""
extension: str = Field(max_length=20)
"""扩展名（小写，无点号）"""
app_id: UUID = Field(foreign_key="fileapp.id", index=True, ondelete="CASCADE")
"""关联的应用UUID"""
# 关系
app: FileApp = Relationship(back_populates="user_defaults")
def to_response(self) -> UserFileAppDefaultResponse:
"""转换为响应 DTO（需预加载 app 关系）"""
return UserFileAppDefaultResponse(
id=self.id,
extension=self.extension,
app=self.app.to_summary(),
)

3
sqlmodels/group.py
View File

@@ -4,8 +4,7 @@ from uuid import UUID
from sqlmodel import Field, Relationship, text
from .base import SQLModelBase
from .mixin import TableBaseMixin, UUIDTableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin, UUIDTableBaseMixin
if TYPE_CHECKING:
from .user import User

144
sqlmodels/migration.py
View File

@@ -17,6 +17,7 @@ async def migration() -> None:
await init_default_group()
await init_default_user()
await init_default_theme_presets()
await init_default_file_apps()
log.info('数据库初始化结束')
@@ -372,3 +373,146 @@ async def init_default_theme_presets() -> None:
)
await default_preset.save(session)
log.info('已创建默认主题预设')
# ==================== 默认文件查看器应用种子数据 ====================
_DEFAULT_FILE_APPS: list[dict] = [
# 内置应用（type=builtin，默认启用）
{
"name": "PDF 阅读器",
"app_key": "pdfjs",
"type": "builtin",
"icon": "file-pdf",
"description": "基于 pdf.js 的 PDF 在线阅读器",
"is_enabled": True,
"extensions": ["pdf"],
},
{
"name": "代码编辑器",
"app_key": "monaco",
"type": "builtin",
"icon": "code",
"description": "基于 Monaco Editor 的代码编辑器",
"is_enabled": True,
"extensions": [
"txt", "md", "json", "xml", "yaml", "yml",
"py", "js", "ts", "jsx", "tsx",
"html", "css", "scss", "less",
"sh", "bash", "zsh",
"c", "cpp", "h", "hpp",
"java", "kt", "go", "rs", "rb",
"sql", "graphql",
"toml", "ini", "cfg", "conf",
"env", "gitignore", "dockerfile",
"vue", "svelte",
],
},
{
"name": "Markdown 预览",
"app_key": "markdown",
"type": "builtin",
"icon": "markdown",
"description": "Markdown 实时预览",
"is_enabled": True,
"extensions": ["md", "markdown", "mdx"],
},
{
"name": "图片查看器",
"app_key": "image_viewer",
"type": "builtin",
"icon": "image",
"description": "图片在线查看器",
"is_enabled": True,
"extensions": ["jpg", "jpeg", "png", "gif", "bmp", "webp", "svg", "ico", "avif"],
},
{
"name": "视频播放器",
"app_key": "video_player",
"type": "builtin",
"icon": "video",
"description": "HTML5 视频播放器",
"is_enabled": True,
"extensions": ["mp4", "webm", "ogg", "mov", "mkv", "m3u8"],
},
{
"name": "音频播放器",
"app_key": "audio_player",
"type": "builtin",
"icon": "audio",
"description": "HTML5 音频播放器",
"is_enabled": True,
"extensions": ["mp3", "wav", "ogg", "flac", "aac", "m4a", "opus"],
},
# iframe 应用（默认禁用）
{
"name": "Office 在线预览",
"app_key": "office_viewer",
"type": "iframe",
"icon": "file-word",
"description": "使用 Microsoft Office Online 预览文档",
"is_enabled": False,
"iframe_url_template": "https://view.officeapps.live.com/op/embed.aspx?src={file_url}",
"extensions": ["doc", "docx", "xls", "xlsx", "ppt", "pptx"],
},
# WOPI 应用（默认禁用）
{
"name": "Collabora Online",
"app_key": "collabora",
"type": "wopi",
"icon": "file-text",
"description": "Collabora Online 文档编辑器（需自行部署）",
"is_enabled": False,
"extensions": ["doc", "docx", "xls", "xlsx", "ppt", "pptx", "odt", "ods", "odp"],
},
{
"name": "OnlyOffice",
"app_key": "onlyoffice",
"type": "wopi",
"icon": "file-text",
"description": "OnlyOffice 文档编辑器（需自行部署）",
"is_enabled": False,
"extensions": ["doc", "docx", "xls", "xlsx", "ppt", "pptx"],
},
]
async def init_default_file_apps() -> None:
"""初始化默认文件查看器应用"""
from .file_app import FileApp, FileAppExtension, FileAppType
from .database_connection import DatabaseManager
log.info('初始化文件查看器应用...')
async for session in DatabaseManager.get_session():
# 已存在应用则跳过
existing_count = await FileApp.count(session)
if existing_count > 0:
return
for app_data in _DEFAULT_FILE_APPS:
extensions = app_data.pop("extensions")
app = FileApp(
name=app_data["name"],
app_key=app_data["app_key"],
type=FileAppType(app_data["type"]),
icon=app_data.get("icon"),
description=app_data.get("description"),
is_enabled=app_data.get("is_enabled", True),
iframe_url_template=app_data.get("iframe_url_template"),
wopi_discovery_url=app_data.get("wopi_discovery_url"),
wopi_editor_url_template=app_data.get("wopi_editor_url_template"),
)
app = await app.save(session)
app_id = app.id
for i, ext in enumerate(extensions):
ext_record = FileAppExtension(
app_id=app_id,
extension=ext.lower(),
priority=i,
)
await ext_record.save(session)
log.info(f'已创建 {len(_DEFAULT_FILE_APPS)} 个默认文件查看器应用')

543
sqlmodels/mixin/README.md
View File

@@ -1,543 +0,0 @@
# SQLModel Mixin Module
This module provides composable Mixin classes for SQLModel entities, enabling reusable functionality such as CRUD operations, polymorphic inheritance, JWT authentication, and standardized response DTOs.
## Module Overview
The `sqlmodels.mixin` module contains various Mixin classes that follow the "Composition over Inheritance" design philosophy. These mixins provide:
- **CRUD Operations**: Async database operations (add, save, update, delete, get, count)
- **Polymorphic Inheritance**: Tools for joined table inheritance patterns
- **JWT Authentication**: Token generation and validation
- **Pagination & Sorting**: Standardized table view parameters
- **Response DTOs**: Consistent id/timestamp fields for API responses
## Module Structure
```
sqlmodels/mixin/
├── __init__.py # Module exports
├── polymorphic.py # PolymorphicBaseMixin, create_subclass_id_mixin, AutoPolymorphicIdentityMixin
├── table.py # TableBaseMixin, UUIDTableBaseMixin, TableViewRequest
├── info_response.py # Response DTO Mixins (IntIdInfoMixin, UUIDIdInfoMixin, etc.)
└── jwt/ # JWT authentication
├── __init__.py
├── key.py # JWTKey database model
├── payload.py # JWTPayloadBase
├── manager.py # JWTManager singleton
├── auth.py # JWTAuthMixin
├── exceptions.py # JWT-related exceptions
└── responses.py # TokenResponse DTO
```
## Dependency Hierarchy
The module has a strict import order to avoid circular dependencies:
1. **polymorphic.py** - Only depends on `SQLModelBase`
2. **table.py** - Depends on `polymorphic.py`
3. **jwt/** - May depend on both `polymorphic.py` and `table.py`
4. **info_response.py** - Only depends on `SQLModelBase`
## Core Components
### 1. TableBaseMixin
Base mixin for database table models with integer primary keys.
**Features:**
- Provides CRUD methods: `add()`, `save()`, `update()`, `delete()`, `get()`, `count()`, `get_exist_one()`
- Automatic timestamp management (`created_at`, `updated_at`)
- Async relationship loading support (via `AsyncAttrs`)
- Pagination and sorting via `TableViewRequest`
- Polymorphic subclass loading support
**Fields:**
- `id: int | None` - Integer primary key (auto-increment)
- `created_at: datetime` - Record creation timestamp
- `updated_at: datetime` - Record update timestamp (auto-updated)
**Usage:**
```python
from sqlmodels.mixin import TableBaseMixin
from sqlmodels.base import SQLModelBase
class User(SQLModelBase, TableBaseMixin, table=True):
name: str
email: str
"""User email"""
# CRUD operations
async def example(session: AsyncSession):
# Add
user = User(name="Alice", email="alice@example.com")
user = await user.save(session)
# Get
user = await User.get(session, User.id == 1)
# Update
update_data = UserUpdateRequest(name="Alice Smith")
user = await user.update(session, update_data)
# Delete
await User.delete(session, user)
# Count
count = await User.count(session, User.is_active == True)
```
**Important Notes:**
- `save()` and `update()` return refreshed instances - **always use the return value**:
```python
# ✅ Correct
device = await device.save(session)
return device
# ❌ Wrong - device is expired after commit
await device.save(session)
return device
```
### 2. UUIDTableBaseMixin
Extends `TableBaseMixin` with UUID primary keys instead of integers.
**Differences from TableBaseMixin:**
- `id: UUID` - UUID primary key (auto-generated via `uuid.uuid4()`)
- `get_exist_one()` accepts `UUID` instead of `int`
**Usage:**
```python
from sqlmodels.mixin import UUIDTableBaseMixin
class Character(SQLModelBase, UUIDTableBaseMixin, table=True):
name: str
description: str | None = None
"""Character description"""
```
**Recommendation:** Use `UUIDTableBaseMixin` for most new models, as UUIDs provide better scalability and avoid ID collisions.
### 3. TableViewRequest
Standardized pagination and sorting parameters for LIST endpoints.
**Fields:**
- `offset: int | None` - Skip first N records (default: 0)
- `limit: int | None` - Return max N records (default: 50, max: 100)
- `desc: bool | None` - Sort descending (default: True)
- `order: Literal["created_at", "updated_at"] | None` - Sort field (default: "created_at")
**Usage with TableBaseMixin.get():**
```python
from dependencies import TableViewRequestDep
@router.get("/list")
async def list_characters(
session: SessionDep,
table_view: TableViewRequestDep
) -> list[Character]:
"""List characters with pagination and sorting"""
return await Character.get(
session,
fetch_mode="all",
table_view=table_view # Automatically handles pagination and sorting
)
```
**Manual usage:**
```python
table_view = TableViewRequest(offset=0, limit=20, desc=True, order="created_at")
characters = await Character.get(session, fetch_mode="all", table_view=table_view)
```
**Backward Compatibility:**
The traditional `offset`, `limit`, `order_by` parameters still work, but `table_view` is recommended for new code.
### 4. PolymorphicBaseMixin
Base mixin for joined table inheritance, automatically configuring polymorphic settings.
**Automatic Configuration:**
- Defines `_polymorphic_name: str` field (indexed)
- Sets `polymorphic_on='_polymorphic_name'`
- Detects abstract classes (via ABC and abstract methods) and sets `polymorphic_abstract=True`
**Methods:**
- `get_concrete_subclasses()` - Get all non-abstract subclasses (for `selectin_polymorphic`)
- `get_polymorphic_discriminator()` - Get the polymorphic discriminator field name
- `get_identity_to_class_map()` - Map `polymorphic_identity` to subclass types
**Usage:**
```python
from abc import ABC, abstractmethod
from sqlmodels.mixin import PolymorphicBaseMixin, UUIDTableBaseMixin
class Tool(PolymorphicBaseMixin, UUIDTableBaseMixin, ABC):
"""Abstract base class for all tools"""
name: str
description: str
"""Tool description"""
@abstractmethod
async def execute(self, params: dict) -> dict:
"""Execute the tool"""
pass
```
**Why Single Underscore Prefix?**
- SQLAlchemy maps single-underscore fields to database columns
- Pydantic treats them as private (excluded from serialization)
- Double-underscore fields would be excluded by SQLAlchemy (not mapped to database)
### 5. create_subclass_id_mixin()
Factory function to create ID mixins for subclasses in joined table inheritance.
**Purpose:** In joined table inheritance, subclasses need a foreign key pointing to the parent table's primary key. This function generates a mixin class providing that foreign key field.
**Signature:**
```python
def create_subclass_id_mixin(parent_table_name: str) -> type[SQLModelBase]:
"""
Args:
parent_table_name: Parent table name (e.g., 'asr', 'tts', 'tool', 'function')
Returns:
A mixin class containing id field (foreign key + primary key)
"""
```
**Usage:**
```python
from sqlmodels.mixin import create_subclass_id_mixin
# Create mixin for ASR subclasses
ASRSubclassIdMixin = create_subclass_id_mixin('asr')
class FunASR(ASRSubclassIdMixin, ASR, AutoPolymorphicIdentityMixin, table=True):
"""FunASR implementation"""
pass
```
**Important:** The ID mixin **must be first in the inheritance list** to ensure MRO (Method Resolution Order) correctly overrides the parent's `id` field.
### 6. AutoPolymorphicIdentityMixin
Automatically generates `polymorphic_identity` based on class name.
**Naming Convention:**
- Format: `{parent_identity}.{classname_lowercase}`
- If no parent identity exists, uses `{classname_lowercase}`
**Usage:**
```python
from sqlmodels.mixin import AutoPolymorphicIdentityMixin
class Function(Tool, AutoPolymorphicIdentityMixin, polymorphic_abstract=True):
"""Base class for function-type tools"""
pass
# polymorphic_identity = 'function'
class GetWeatherFunction(Function, table=True):
"""Weather query function"""
pass
# polymorphic_identity = 'function.getweatherfunction'
```
**Manual Override:**
```python
class CustomTool(
Tool,
AutoPolymorphicIdentityMixin,
polymorphic_identity='custom_name', # Override auto-generated name
table=True
):
pass
```
### 7. JWTAuthMixin
Provides JWT token generation and validation for entity classes (User, Client).
**Methods:**
- `async issue_jwt(session: AsyncSession) -> str` - Generate JWT token for current instance
- `@classmethod async from_jwt(session: AsyncSession, token: str) -> Self` - Validate token and retrieve entity
**Requirements:**
Subclasses must define:
- `JWTPayload` - Payload model (inherits from `JWTPayloadBase`)
- `jwt_key_purpose` - ClassVar specifying the JWT key purpose enum value
**Usage:**
```python
from sqlmodels.mixin import JWTAuthMixin, UUIDTableBaseMixin
class User(SQLModelBase, UUIDTableBaseMixin, JWTAuthMixin, table=True):
JWTPayload = UserJWTPayload # Define payload model
jwt_key_purpose: ClassVar[JWTKeyPurposeEnum] = JWTKeyPurposeEnum.user
email: str
is_admin: bool = False
is_active: bool = True
"""User active status"""
# Generate token
async def login(session: AsyncSession, user: User) -> str:
token = await user.issue_jwt(session)
return token
# Validate token
async def verify(session: AsyncSession, token: str) -> User:
user = await User.from_jwt(session, token)
return user
```
### 8. Response DTO Mixins
Mixins for standardized InfoResponse DTOs, defining id and timestamp fields.
**Available Mixins:**
- `IntIdInfoMixin` - Integer ID field
- `UUIDIdInfoMixin` - UUID ID field
- `DatetimeInfoMixin` - `created_at` and `updated_at` fields
- `IntIdDatetimeInfoMixin` - Integer ID + timestamps
- `UUIDIdDatetimeInfoMixin` - UUID ID + timestamps
**Design Note:** These fields are non-nullable in DTOs because database records always have these values when returned.
**Usage:**
```python
from sqlmodels.mixin import UUIDIdDatetimeInfoMixin
class CharacterInfoResponse(CharacterBase, UUIDIdDatetimeInfoMixin):
"""Character response DTO with id and timestamps"""
pass # Inherits id, created_at, updated_at from mixin
```
## Complete Joined Table Inheritance Example
Here's a complete example demonstrating polymorphic inheritance:
```python
from abc import ABC, abstractmethod
from sqlmodels.base import SQLModelBase
from sqlmodels.mixin import (
UUIDTableBaseMixin,
PolymorphicBaseMixin,
create_subclass_id_mixin,
AutoPolymorphicIdentityMixin,
)
# 1. Define Base class (fields only, no table)
class ASRBase(SQLModelBase):
name: str
"""Configuration name"""
base_url: str
"""Service URL"""
# 2. Define abstract parent class (with table)
class ASR(ASRBase, UUIDTableBaseMixin, PolymorphicBaseMixin, ABC):
"""Abstract base class for ASR configurations"""
# PolymorphicBaseMixin automatically provides:
# - _polymorphic_name field
# - polymorphic_on='_polymorphic_name'
# - polymorphic_abstract=True (when ABC with abstract methods)
@abstractmethod
async def transcribe(self, pcm_data: bytes) -> str:
"""Transcribe audio to text"""
pass
# 3. Create ID Mixin for second-level subclasses
ASRSubclassIdMixin = create_subclass_id_mixin('asr')
# 4. Create second-level abstract class (if needed)
class FunASR(
ASRSubclassIdMixin,
ASR,
AutoPolymorphicIdentityMixin,
polymorphic_abstract=True
):
"""FunASR abstract base (may have multiple implementations)"""
pass
# polymorphic_identity = 'funasr'
# 5. Create concrete implementation classes
class FunASRLocal(FunASR, table=True):
"""FunASR local deployment"""
# polymorphic_identity = 'funasr.funasrlocal'
async def transcribe(self, pcm_data: bytes) -> str:
# Implementation...
return "transcribed text"
# 6. Get all concrete subclasses (for selectin_polymorphic)
concrete_asrs = ASR.get_concrete_subclasses()
# Returns: [FunASRLocal, ...]
```
## Import Guidelines
**Standard Import:**
```python
from sqlmodels.mixin import (
TableBaseMixin,
UUIDTableBaseMixin,
PolymorphicBaseMixin,
TableViewRequest,
create_subclass_id_mixin,
AutoPolymorphicIdentityMixin,
JWTAuthMixin,
UUIDIdDatetimeInfoMixin,
now,
now_date,
)
```
**Backward Compatibility:**
Some exports are also available from `sqlmodels.base` for backward compatibility:
```python
# Legacy import path (still works)
from sqlmodels.base import UUIDTableBase, TableViewRequest
# Recommended new import path
from sqlmodels.mixin import UUIDTableBaseMixin, TableViewRequest
```
## Best Practices
### 1. Mixin Order Matters
**Correct Order:**
```python
# ✅ ID Mixin first, then parent, then AutoPolymorphicIdentityMixin
class SubTool(ToolSubclassIdMixin, Tool, AutoPolymorphicIdentityMixin, table=True):
pass
```
**Wrong Order:**
```python
# ❌ ID Mixin not first - won't override parent's id field
class SubTool(Tool, ToolSubclassIdMixin, AutoPolymorphicIdentityMixin, table=True):
pass
```
### 2. Always Use Return Values from save() and update()
```python
# ✅ Correct - use returned instance
device = await device.save(session)
return device
# ❌ Wrong - device is expired after commit
await device.save(session)
return device # AttributeError when accessing fields
```
### 3. Prefer table_view Over Manual Pagination
```python
# ✅ Recommended - consistent across all endpoints
characters = await Character.get(
session,
fetch_mode="all",
table_view=table_view
)
# ⚠️ Works but not recommended - manual parameter management
characters = await Character.get(
session,
fetch_mode="all",
offset=0,
limit=20,
order_by=[desc(Character.created_at)]
)
```
### 4. Polymorphic Loading for Many Subclasses
```python
# When loading relationships with > 10 polymorphic subclasses, use load_polymorphic='all'
tool_set = await ToolSet.get(
session,
ToolSet.id == tool_set_id,
load=ToolSet.tools,
load_polymorphic='all' # Two-phase query - only loads actual related subclasses
)
# For fewer subclasses, specify the list explicitly
tool_set = await ToolSet.get(
session,
ToolSet.id == tool_set_id,
load=ToolSet.tools,
load_polymorphic=[GetWeatherFunction, CodeInterpreterFunction]
)
```
### 5. Response DTOs Should Inherit Base Classes
```python
# ✅ Correct - inherits from CharacterBase
class CharacterInfoResponse(CharacterBase, UUIDIdDatetimeInfoMixin):
pass
# ❌ Wrong - doesn't inherit from CharacterBase
class CharacterInfoResponse(SQLModelBase, UUIDIdDatetimeInfoMixin):
name: str # Duplicated field definition
description: str | None = None
```
**Reason:** Inheriting from Base classes ensures:
- Type checking via `isinstance(obj, XxxBase)`
- Consistency across related DTOs
- Future field additions automatically propagate
### 6. Use Specific Types, Not Containers
```python
# ✅ Correct - specific DTO for config updates
class GetWeatherFunctionUpdateRequest(GetWeatherFunctionConfigBase):
weather_api_key: str | None = None
default_location: str | None = None
"""Default location"""
# ❌ Wrong - lose type safety
class ToolUpdateRequest(SQLModelBase):
config: dict[str, Any] # No field validation
```
## Type Variables
```python
from sqlmodels.mixin import T, M
T = TypeVar("T", bound="TableBaseMixin") # For CRUD methods
M = TypeVar("M", bound="SQLModel") # For update() method
```
## Utility Functions
```python
from sqlmodels.mixin import now, now_date
# Lambda functions for default factories
now = lambda: datetime.now()
now_date = lambda: datetime.now().date()
```
## Related Modules
- **sqlmodels.base** - Base classes (`SQLModelBase`, backward-compatible exports)
- **dependencies** - FastAPI dependencies (`SessionDep`, `TableViewRequestDep`)
- **sqlmodels.user** - User model with JWT authentication
- **sqlmodels.client** - Client model with JWT authentication
- **sqlmodels.character.llm.openai_compatibles.tools** - Polymorphic tool hierarchy
## Additional Resources
- `POLYMORPHIC_NAME_DESIGN.md` - Design rationale for `_polymorphic_name` field
- `CLAUDE.md` - Project coding standards and design philosophy
- SQLAlchemy Documentation - [Joined Table Inheritance](https://docs.sqlalchemy.org/en/20/orm/inheritance.html#joined-table-inheritance)

62
sqlmodels/mixin/__init__.py
View File

@@ -1,62 +0,0 @@
"""
SQLModel Mixin模块
提供各种Mixin类供SQLModel实体使用。
包含：
- polymorphic: 联表继承工具（create_subclass_id_mixin, AutoPolymorphicIdentityMixin, PolymorphicBaseMixin）
- optimistic_lock: 乐观锁（OptimisticLockMixin, OptimisticLockError）
- table: 表基类（TableBaseMixin, UUIDTableBaseMixin）
- table: 查询参数类（TimeFilterRequest, PaginationRequest, TableViewRequest）
- relation_preload: 关系预加载（RelationPreloadMixin, requires_relations）
- jwt/: JWT认证相关（JWTAuthMixin, JWTManager, JWTKey等）- 需要时直接从 .jwt 导入
- info_response: InfoResponse DTO的id/时间戳Mixin
导入顺序很重要，避免循环导入：
1. polymorphic（只依赖 SQLModelBase）
2. optimistic_lock（只依赖 SQLAlchemy）
3. table（依赖 polymorphic 和 optimistic_lock）
4. relation_preload（只依赖 SQLModelBase）
注意：jwt 模块不在此处导入，因为 jwt/manager.py 导入 ServerConfig，
而 ServerConfig 导入本模块，会形成循环。需要 jwt 功能时请直接从 .jwt 导入。
"""
# polymorphic 必须先导入
from .polymorphic import (
AutoPolymorphicIdentityMixin,
PolymorphicBaseMixin,
create_subclass_id_mixin,
register_sti_column_properties_for_all_subclasses,
register_sti_columns_for_all_subclasses,
)
# optimistic_lock 只依赖 SQLAlchemy，必须在 table 之前
from .optimistic_lock import (
OptimisticLockError,
OptimisticLockMixin,
)
# table 依赖 polymorphic 和 optimistic_lock
from .table import (
ListResponse,
PaginationRequest,
T,
TableBaseMixin,
TableViewRequest,
TimeFilterRequest,
UUIDTableBaseMixin,
now,
now_date,
)
# relation_preload 只依赖 SQLModelBase
from .relation_preload import (
RelationPreloadMixin,
requires_relations,
)
# jwt 不在此处导入（避免循环：jwt/manager.py → ServerConfig → mixin → jwt）
# 需要时直接从 sqlmodels.mixin.jwt 导入
from .info_response import (
DatetimeInfoMixin,
IntIdDatetimeInfoMixin,
IntIdInfoMixin,
UUIDIdDatetimeInfoMixin,
UUIDIdInfoMixin,
)

46
sqlmodels/mixin/info_response.py
View File

@@ -1,46 +0,0 @@
"""
InfoResponse DTO Mixin模块
提供用于InfoResponse类型DTO的Mixin，统一定义id/created_at/updated_at字段。
设计说明：
- 这些Mixin用于**响应DTO**，不是数据库表
- 从数据库返回时这些字段永远不为空，所以定义为必填字段
- TableBase中的id=None和default_factory=now是正确的（入库前为None，数据库生成）
- 这些Mixin让DTO明确表示"返回给客户端时这些字段必定有值"
"""
from datetime import datetime
from uuid import UUID
from sqlmodels.base import SQLModelBase
class IntIdInfoMixin(SQLModelBase):
"""整数ID响应mixin - 用于InfoResponse DTO"""
id: int
"""记录ID"""
class UUIDIdInfoMixin(SQLModelBase):
"""UUID ID响应mixin - 用于InfoResponse DTO"""
id: UUID
"""记录ID"""
class DatetimeInfoMixin(SQLModelBase):
"""时间戳响应mixin - 用于InfoResponse DTO"""
created_at: datetime
"""创建时间"""
updated_at: datetime
"""更新时间"""
class IntIdDatetimeInfoMixin(IntIdInfoMixin, DatetimeInfoMixin):
"""整数ID + 时间戳响应mixin"""
pass
class UUIDIdDatetimeInfoMixin(UUIDIdInfoMixin, DatetimeInfoMixin):
"""UUID ID + 时间戳响应mixin"""
pass

90
sqlmodels/mixin/optimistic_lock.py
View File

@@ -1,90 +0,0 @@
"""
乐观锁 Mixin
提供基于 SQLAlchemy version_id_col 机制的乐观锁支持。
乐观锁适用场景：
- 涉及"状态转换"的表（如：待支付 -> 已支付）
- 涉及"数值变动"的表（如：余额、库存）
不适用场景：
- 日志表、纯插入表、低价值统计表
- 能用 UPDATE table SET col = col + 1 解决的简单计数问题
使用示例：
class Order(OptimisticLockMixin, UUIDTableBaseMixin, table=True):
status: OrderStatusEnum
amount: Decimal
# save/update 时自动检查版本号
# 如果版本号不匹配（其他事务已修改），会抛出 OptimisticLockError
try:
order = await order.save(session)
except OptimisticLockError as e:
# 处理冲突：重新查询并重试，或报错给用户
l.warning(f"乐观锁冲突: {e}")
"""
from typing import ClassVar
from sqlalchemy.orm.exc import StaleDataError
class OptimisticLockError(Exception):
"""
乐观锁冲突异常
当 save/update 操作检测到版本号不匹配时抛出。
这意味着在读取和写入之间，其他事务已经修改了该记录。
Attributes:
model_class: 发生冲突的模型类名
record_id: 记录 ID（如果可用）
expected_version: 期望的版本号（如果可用）
original_error: 原始的 StaleDataError
"""
def __init__(
self,
message: str,
model_class: str | None = None,
record_id: str | None = None,
expected_version: int | None = None,
original_error: StaleDataError | None = None,
):
super().__init__(message)
self.model_class = model_class
self.record_id = record_id
self.expected_version = expected_version
self.original_error = original_error
class OptimisticLockMixin:
"""
乐观锁 Mixin
使用 SQLAlchemy 的 version_id_col 机制实现乐观锁。
每次 UPDATE 时自动检查并增加版本号，如果版本号不匹配（即其他事务已修改），
session.commit() 会抛出 StaleDataError，被 save/update 方法捕获并转换为 OptimisticLockError。
原理：
1. 每条记录有一个 version 字段，初始值为 0
2. 每次 UPDATE 时，SQLAlchemy 生成的 SQL 类似：
UPDATE table SET ..., version = version + 1 WHERE id = ? AND version = ?
3. 如果 WHERE 条件不匹配（version 已被其他事务修改），
UPDATE 影响 0 行，SQLAlchemy 抛出 StaleDataError
继承顺序：
OptimisticLockMixin 必须放在 TableBaseMixin/UUIDTableBaseMixin 之前：
class Order(OptimisticLockMixin, UUIDTableBaseMixin, table=True):
...
配套重试：
如果加了乐观锁，业务层需要处理 OptimisticLockError：
- 报错给用户："数据已被修改，请刷新后重试"
- 自动重试：重新查询最新数据并再次尝试
"""
_has_optimistic_lock: ClassVar[bool] = True
"""标记此类启用了乐观锁"""
version: int = 0
"""乐观锁版本号，每次更新自动递增"""

710
sqlmodels/mixin/polymorphic.py
View File

@@ -1,710 +0,0 @@
"""
联表继承（Joined Table Inheritance）的通用工具
提供用于简化SQLModel多态表设计的辅助函数和Mixin。
Usage Example:
from sqlmodels.base import SQLModelBase
from sqlmodels.mixin import UUIDTableBaseMixin
from sqlmodels.mixin.polymorphic import (
PolymorphicBaseMixin,
create_subclass_id_mixin,
AutoPolymorphicIdentityMixin
)
# 1. 定义Base类（只有字段，无表）
class ASRBase(SQLModelBase):
name: str
\"\"\"配置名称\"\"\"
base_url: str
\"\"\"服务地址\"\"\"
# 2. 定义抽象父类（有表），使用 PolymorphicBaseMixin
class ASR(
ASRBase,
UUIDTableBaseMixin,
PolymorphicBaseMixin,
ABC
):
\"\"\"ASR配置的抽象基类\"\"\"
# PolymorphicBaseMixin 自动提供:
# - _polymorphic_name 字段
# - polymorphic_on='_polymorphic_name'
# - polymorphic_abstract=True（当有抽象方法时）
# 3. 为第二层子类创建ID Mixin
ASRSubclassIdMixin = create_subclass_id_mixin('asr')
# 4. 创建第二层抽象类（如果需要）
class FunASR(
ASRSubclassIdMixin,
ASR,
AutoPolymorphicIdentityMixin,
polymorphic_abstract=True
):
\"\"\"FunASR的抽象基类，可能有多个实现\"\"\"
pass
# 5. 创建具体实现类
class FunASRLocal(FunASR, table=True):
\"\"\"FunASR本地部署版本\"\"\"
# polymorphic_identity 会自动设置为 'asr.funasrlocal'
pass
# 6. 获取所有具体子类（用于 selectin_polymorphic）
concrete_asrs = ASR.get_concrete_subclasses()
# 返回 [FunASRLocal, ...]
"""
import uuid
from abc import ABC
from uuid import UUID
from loguru import logger as l
from pydantic.fields import FieldInfo
from pydantic_core import PydanticUndefined
from sqlalchemy import Column, String, inspect
from sqlalchemy.orm import ColumnProperty, Mapped, mapped_column
from sqlalchemy.orm.attributes import InstrumentedAttribute
from sqlmodel import Field
from sqlmodel.main import get_column_from_field
from sqlmodels.base.sqlmodel_base import SQLModelBase
# 用于延迟注册 STI 子类列的队列
# 在所有模型加载完成后，调用 register_sti_columns_for_all_subclasses() 处理
_sti_subclasses_to_register: list[type] = []
def register_sti_columns_for_all_subclasses() -> None:
"""
为所有已注册的 STI 子类执行列注册（第一阶段：添加列到表）
此函数应在 configure_mappers() 之前调用。
将 STI 子类的字段添加到父表的 metadata 中。
同时修复被 Column 对象污染的 model_fields。
"""
for cls in _sti_subclasses_to_register:
try:
cls._register_sti_columns()
except Exception as e:
l.warning(f"注册 STI 子类 {cls.__name__} 的列时出错: {e}")
# 修复被 Column 对象污染的 model_fields
# 必须在列注册后立即修复，因为 Column 污染在类定义时就已发生
try:
_fix_polluted_model_fields(cls)
except Exception as e:
l.warning(f"修复 STI 子类 {cls.__name__} 的 model_fields 时出错: {e}")
def register_sti_column_properties_for_all_subclasses() -> None:
"""
为所有已注册的 STI 子类添加列属性到 mapper（第二阶段）
此函数应在 configure_mappers() 之后调用。
将 STI 子类的字段作为 ColumnProperty 添加到 mapper 中。
"""
for cls in _sti_subclasses_to_register:
try:
cls._register_sti_column_properties()
except Exception as e:
l.warning(f"注册 STI 子类 {cls.__name__} 的列属性时出错: {e}")
# 清空队列
_sti_subclasses_to_register.clear()
def _fix_polluted_model_fields(cls: type) -> None:
"""
修复被 SQLAlchemy InstrumentedAttribute 或 Column 污染的 model_fields
当 SQLModel 类继承有表的父类时，SQLAlchemy 会在类上创建 InstrumentedAttribute
或 Column 对象替换原始的字段默认值。这会导致 Pydantic 在构建子类 model_fields
时错误地使用这些 SQLAlchemy 对象作为默认值。
此函数从 MRO 中查找原始的字段定义，并修复被污染的 model_fields。
:param cls: 要修复的类
"""
if not hasattr(cls, 'model_fields'):
return
def find_original_field_info(field_name: str) -> FieldInfo | None:
"""从 MRO 中查找字段的原始定义（未被污染的）"""
for base in cls.__mro__[1:]: # 跳过自己
if hasattr(base, 'model_fields') and field_name in base.model_fields:
field_info = base.model_fields[field_name]
# 跳过被 InstrumentedAttribute 或 Column 污染的
if not isinstance(field_info.default, (InstrumentedAttribute, Column)):
return field_info
return None
for field_name, current_field in cls.model_fields.items():
# 检查是否被污染（default 是 InstrumentedAttribute 或 Column）
# Column 污染发生在 STI 继承链中：当 FunctionBase.show_arguments = True
# 被继承到有表的子类时，SQLModel 会创建一个 Column 对象替换原始默认值
if not isinstance(current_field.default, (InstrumentedAttribute, Column)):
continue # 未被污染，跳过
# 从父类查找原始定义
original = find_original_field_info(field_name)
if original is None:
continue # 找不到原始定义，跳过
# 根据原始定义的 default/default_factory 来修复
if original.default_factory:
# 有 default_factory（如 uuid.uuid4, now）
new_field = FieldInfo(
default_factory=original.default_factory,
annotation=current_field.annotation,
json_schema_extra=current_field.json_schema_extra,
)
elif original.default is not PydanticUndefined:
# 有明确的 default 值（如 None, 0, True），且不是 PydanticUndefined
# PydanticUndefined 表示字段没有默认值（必填）
new_field = FieldInfo(
default=original.default,
annotation=current_field.annotation,
json_schema_extra=current_field.json_schema_extra,
)
else:
continue # 既没有 default_factory 也没有有效的 default，跳过
# 复制 SQLModel 特有的属性
if hasattr(current_field, 'foreign_key'):
new_field.foreign_key = current_field.foreign_key
if hasattr(current_field, 'primary_key'):
new_field.primary_key = current_field.primary_key
cls.model_fields[field_name] = new_field
def create_subclass_id_mixin(parent_table_name: str) -> type['SQLModelBase']:
"""
动态创建SubclassIdMixin类
在联表继承中，子类需要一个外键指向父表的主键。
此函数生成一个Mixin类，提供这个外键字段，并自动生成UUID。
Args:
parent_table_name: 父表名称（如'asr', 'tts', 'tool', 'function'）
Returns:
一个Mixin类，包含id字段（外键 + 主键 + default_factory=uuid.uuid4）
Example:
>>> ASRSubclassIdMixin = create_subclass_id_mixin('asr')
>>> class FunASR(ASRSubclassIdMixin, ASR, table=True):
... pass
Note:
- 生成的Mixin应该放在继承列表的第一位，确保通过MRO覆盖UUIDTableBaseMixin的id
- 生成的类名为 {ParentTableName}SubclassIdMixin（PascalCase）
- 本项目所有联表继承均使用UUID主键（UUIDTableBaseMixin）
"""
if not parent_table_name:
raise ValueError("parent_table_name 不能为空")
# 转换为PascalCase作为类名
class_name_parts = parent_table_name.split('_')
class_name = ''.join(part.capitalize() for part in class_name_parts) + 'SubclassIdMixin'
# 使用闭包捕获parent_table_name
_parent_table_name = parent_table_name
# 创建带有__init_subclass__的mixin类，用于在子类定义后修复model_fields
class SubclassIdMixin(SQLModelBase):
# 定义id字段
id: UUID = Field(
default_factory=uuid.uuid4,
foreign_key=f'{_parent_table_name}.id',
primary_key=True,
)
@classmethod
def __pydantic_init_subclass__(cls, **kwargs):
"""
Pydantic v2 的子类初始化钩子，在模型完全构建后调用
修复联表继承中子类字段的 default_factory 丢失问题。
SQLAlchemy 的 InstrumentedAttribute 或 Column 会污染从父类继承的字段，
导致 INSERT 语句中出现 `table.column` 引用而非实际值。
"""
super().__pydantic_init_subclass__(**kwargs)
_fix_polluted_model_fields(cls)
# 设置类名和文档
SubclassIdMixin.__name__ = class_name
SubclassIdMixin.__qualname__ = class_name
SubclassIdMixin.__doc__ = f"""
{parent_table_name}子类的ID Mixin
用于{parent_table_name}的子类，提供外键指向父表。
通过MRO确保此id字段覆盖继承的id字段。
"""
return SubclassIdMixin
class AutoPolymorphicIdentityMixin:
"""
自动生成polymorphic_identity的Mixin，并支持STI子类列注册
使用此Mixin的类会自动根据类名生成polymorphic_identity。
格式：{parent_polymorphic_identity}.{classname_lowercase}
如果没有父类的polymorphic_identity，则直接使用类名小写。
**重要：数据库迁移注意事项**
编写数据迁移脚本时，必须使用完整的 polymorphic_identity 格式，包括父类前缀！
例如，对于以下继承链::
LLM (polymorphic_on='_polymorphic_name')
└── AnthropicCompatibleLLM (polymorphic_identity='anthropiccompatiblellm')
└── TuziAnthropicLLM (polymorphic_identity='anthropiccompatiblellm.tuzianthropicllm')
迁移脚本中设置 _polymorphic_name 时::
# ❌ 错误：缺少父类前缀
UPDATE llm SET _polymorphic_name = 'tuzianthropicllm' WHERE id = :id
# ✅ 正确：包含完整的继承链前缀
UPDATE llm SET _polymorphic_name = 'anthropiccompatiblellm.tuzianthropicllm' WHERE id = :id
**STI（单表继承）支持**：
当子类与父类共用同一张表（STI模式）时，此Mixin会自动将子类的新字段
添加到父表的列定义中。这解决了SQLModel在STI模式下子类字段不被
注册到父表的问题。
Example (JTI):
>>> class Tool(UUIDTableBaseMixin, polymorphic_on='__polymorphic_name', polymorphic_abstract=True):
... __polymorphic_name: str
...
>>> class Function(Tool, AutoPolymorphicIdentityMixin, polymorphic_abstract=True):
... pass
... # polymorphic_identity 会自动设置为 'function'
...
>>> class CodeInterpreterFunction(Function, table=True):
... pass
... # polymorphic_identity 会自动设置为 'function.codeinterpreterfunction'
Example (STI):
>>> class UserFile(UUIDTableBaseMixin, PolymorphicBaseMixin, table=True, polymorphic_abstract=True):
... user_id: UUID
...
>>> class PendingFile(UserFile, AutoPolymorphicIdentityMixin, table=True):
... upload_deadline: datetime | None = None # 自动添加到 userfile 表
... # polymorphic_identity 会自动设置为 'pendingfile'
Note:
- 如果手动在__mapper_args__中指定了polymorphic_identity，会被保留
- 此Mixin应该在继承列表中靠后的位置（在表基类之前）
- STI模式下，新字段会在类定义时自动添加到父表的metadata中
"""
def __init_subclass__(cls, polymorphic_identity: str | None = None, **kwargs):
"""
子类化钩子，自动生成polymorphic_identity并处理STI列注册
Args:
polymorphic_identity: 如果手动指定，则使用指定的值
**kwargs: 其他SQLModel参数（如table=True, polymorphic_abstract=True）
"""
super().__init_subclass__(**kwargs)
# 如果手动指定了polymorphic_identity，使用指定的值
if polymorphic_identity is not None:
identity = polymorphic_identity
else:
# 自动生成polymorphic_identity
class_name = cls.__name__.lower()
# 尝试从父类获取polymorphic_identity作为前缀
parent_identity = None
for base in cls.__mro__[1:]: # 跳过自己
if hasattr(base, '__mapper_args__') and isinstance(base.__mapper_args__, dict):
parent_identity = base.__mapper_args__.get('polymorphic_identity')
if parent_identity:
break
# 构建identity
if parent_identity:
identity = f'{parent_identity}.{class_name}'
else:
identity = class_name
# 设置到__mapper_args__
if '__mapper_args__' not in cls.__dict__:
cls.__mapper_args__ = {}
# 只在尚未设置polymorphic_identity时设置
if 'polymorphic_identity' not in cls.__mapper_args__:
cls.__mapper_args__['polymorphic_identity'] = identity
# 注册 STI 子类列的延迟执行
# 由于 __init_subclass__ 在类定义过程中被调用，此时 model_fields 还不完整
# 需要在模块加载完成后调用 register_sti_columns_for_all_subclasses()
_sti_subclasses_to_register.append(cls)
@classmethod
def __pydantic_init_subclass__(cls, **kwargs):
"""
Pydantic v2 的子类初始化钩子，在模型完全构建后调用
修复 STI 继承中子类字段被 Column 对象污染的问题。
当 FunctionBase.show_arguments = True 等字段被继承到有表的子类时，
SQLModel 会创建一个 Column 对象替换原始默认值，导致实例化时字段值不正确。
"""
super().__pydantic_init_subclass__(**kwargs)
_fix_polluted_model_fields(cls)
@classmethod
def _register_sti_columns(cls) -> None:
"""
将STI子类的新字段注册到父表的列定义中
检测当前类是否是STI子类（与父类共用同一张表），
如果是，则将子类定义的新字段添加到父表的metadata中。
JTI（联表继承）类会被自动跳过，因为它们有自己独立的表。
"""
# 查找父表（在 MRO 中找到第一个有 __table__ 的父类）
parent_table = None
parent_fields: set[str] = set()
for base in cls.__mro__[1:]:
if hasattr(base, '__table__') and base.__table__ is not None:
parent_table = base.__table__
# 收集父类的所有字段名
if hasattr(base, 'model_fields'):
parent_fields.update(base.model_fields.keys())
break
if parent_table is None:
return # 没有找到父表，可能是根类
# JTI 检测：如果当前类有自己的表且与父表不同，则是 JTI
# JTI 类有自己独立的表，不需要将列注册到父表
if hasattr(cls, '__table__') and cls.__table__ is not None:
if cls.__table__.name != parent_table.name:
return # JTI，跳过 STI 列注册
# 获取当前类的新字段（不在父类中的字段）
if not hasattr(cls, 'model_fields'):
return
existing_columns = {col.name for col in parent_table.columns}
for field_name, field_info in cls.model_fields.items():
# 跳过从父类继承的字段
if field_name in parent_fields:
continue
# 跳过私有字段和ClassVar
if field_name.startswith('_'):
continue
# 跳过已存在的列
if field_name in existing_columns:
continue
# 使用 SQLModel 的内置 API 创建列
try:
column = get_column_from_field(field_info)
column.name = field_name
column.key = field_name
# STI子类字段在数据库层面必须可空，因为其他子类的行不会有这些字段的值
# Pydantic层面的约束仍然有效（创建特定子类时会验证必填字段）
column.nullable = True
# 将列添加到父表
parent_table.append_column(column)
except Exception as e:
l.warning(f"为 {cls.__name__} 创建列 {field_name} 失败: {e}")
@classmethod
def _register_sti_column_properties(cls) -> None:
"""
将 STI 子类的列作为 ColumnProperty 添加到 mapper
此方法在 configure_mappers() 之后调用，将已添加到表中的列
注册为 mapper 的属性，使 ORM 查询能正确识别这些列。
**重要**：子类的列属性会同时注册到子类和父类的 mapper 上。
这确保了查询父类时，SELECT 语句包含所有 STI 子类的列，
避免在响应序列化时触发懒加载（MissingGreenlet 错误）。
JTI（联表继承）类会被自动跳过，因为它们有自己独立的表。
"""
# 查找父表和父类（在 MRO 中找到第一个有 __table__ 的父类）
parent_table = None
parent_class = None
for base in cls.__mro__[1:]:
if hasattr(base, '__table__') and base.__table__ is not None:
parent_table = base.__table__
parent_class = base
break
if parent_table is None:
return # 没有找到父表，可能是根类
# JTI 检测：如果当前类有自己的表且与父表不同，则是 JTI
# JTI 类有自己独立的表，不需要将列属性注册到 mapper
if hasattr(cls, '__table__') and cls.__table__ is not None:
if cls.__table__.name != parent_table.name:
return # JTI，跳过 STI 列属性注册
# 获取子类和父类的 mapper
child_mapper = inspect(cls).mapper
parent_mapper = inspect(parent_class).mapper
local_table = child_mapper.local_table
# 查找父类的所有字段名
parent_fields: set[str] = set()
if hasattr(parent_class, 'model_fields'):
parent_fields.update(parent_class.model_fields.keys())
if not hasattr(cls, 'model_fields'):
return
# 获取两个 mapper 已有的列属性
child_existing_props = {p.key for p in child_mapper.column_attrs}
parent_existing_props = {p.key for p in parent_mapper.column_attrs}
for field_name in cls.model_fields:
# 跳过从父类继承的字段
if field_name in parent_fields:
continue
# 跳过私有字段
if field_name.startswith('_'):
continue
# 检查表中是否有这个列
if field_name not in local_table.columns:
continue
column = local_table.columns[field_name]
# 添加到子类的 mapper（如果尚不存在）
if field_name not in child_existing_props:
try:
prop = ColumnProperty(column)
child_mapper.add_property(field_name, prop)
except Exception as e:
l.warning(f"为 {cls.__name__} 添加列属性 {field_name} 失败: {e}")
# 同时添加到父类的 mapper（确保查询父类时 SELECT 包含所有 STI 子类的列）
if field_name not in parent_existing_props:
try:
prop = ColumnProperty(column)
parent_mapper.add_property(field_name, prop)
except Exception as e:
l.warning(f"为父类 {parent_class.__name__} 添加子类 {cls.__name__} 的列属性 {field_name} 失败: {e}")
class PolymorphicBaseMixin:
"""
为联表继承链中的基类自动配置 polymorphic 设置的 Mixin
此 Mixin 自动设置以下内容：
- `polymorphic_on='_polymorphic_name'`: 使用 _polymorphic_name 字段作为多态鉴别器
- `_polymorphic_name: str`: 定义多态鉴别器字段（带索引）
- `polymorphic_abstract=True`: 当类继承自 ABC 且有抽象方法时，自动标记为抽象类
使用场景：
适用于需要 joined table inheritance 的基类，例如 Tool、ASR、TTS 等。
用法示例：
```python
from abc import ABC
from sqlmodels.mixin import UUIDTableBaseMixin
from sqlmodels.mixin.polymorphic import PolymorphicBaseMixin
# 定义基类
class MyTool(UUIDTableBaseMixin, PolymorphicBaseMixin, ABC):
__tablename__ = 'mytool'
# 不需要手动定义 _polymorphic_name
# 不需要手动设置 polymorphic_on
# 不需要手动设置 polymorphic_abstract
# 定义子类
class SpecificTool(MyTool):
__tablename__ = 'specifictool'
# 会自动继承 polymorphic 配置
```
自动行为：
1. 定义 `_polymorphic_name: str` 字段（带索引）
2. 设置 `__mapper_args__['polymorphic_on'] = '_polymorphic_name'`
3. 自动检测抽象类：
- 如果类继承了 ABC 且有未实现的抽象方法，设置 polymorphic_abstract=True
- 否则设置为 False
手动覆盖：
可以在类定义时手动指定参数来覆盖自动行为：
```python
class MyTool(
UUIDTableBaseMixin,
PolymorphicBaseMixin,
ABC,
polymorphic_on='custom_field', # 覆盖默认的 _polymorphic_name
polymorphic_abstract=False # 强制不设为抽象类
):
pass
```
注意事项：
- 此 Mixin 应该与 UUIDTableBaseMixin 或 TableBaseMixin 配合使用
- 适用于联表继承（joined table inheritance）场景
- 子类会自动继承 _polymorphic_name 字段定义
- 使用单下划线前缀是因为：
* SQLAlchemy 会映射单下划线字段为数据库列
* Pydantic 将其视为私有属性，不参与序列化
* 双下划线字段会被 SQLAlchemy 排除，不映射为数据库列
"""
# 定义 _polymorphic_name 字段，所有使用此 mixin 的类都会有这个字段
#
# 设计选择：使用单下划线前缀 + Mapped[str] + mapped_column
#
# 为什么这样做：
# 1. 单下划线前缀表示"内部实现细节"，防止外部通过 API 直接修改
# 2. Mapped + mapped_column 绕过 Pydantic v2 的字段名限制（不允许下划线前缀）
# 3. 字段仍然被 SQLAlchemy 映射到数据库，供多态查询使用
# 4. 字段不出现在 Pydantic 序列化中（model_dump() 和 JSON schema）
# 5. 内部代码仍然可以正常访问和修改此字段
#
# 详细说明请参考：sqlmodels/base/POLYMORPHIC_NAME_DESIGN.md
_polymorphic_name: Mapped[str] = mapped_column(String, index=True)
"""
多态鉴别器字段，用于标识具体的子类类型
注意：此字段使用单下划线前缀，表示内部使用。
- ✅ 存储到数据库
- ✅ 不出现在 API 序列化中
- ✅ 防止外部直接修改
"""
def __init_subclass__(
cls,
polymorphic_on: str | None = None,
polymorphic_abstract: bool | None = None,
**kwargs
):
"""
在子类定义时自动配置 polymorphic 设置
Args:
polymorphic_on: polymorphic_on 字段名，默认为 '_polymorphic_name'。
设置为其他值可以使用不同的字段作为多态鉴别器。
polymorphic_abstract: 是否为抽象类。
- None: 自动检测（默认）
- True: 强制设为抽象类
- False: 强制设为非抽象类
**kwargs: 传递给父类的其他参数
"""
super().__init_subclass__(**kwargs)
# 初始化 __mapper_args__（如果还没有）
if '__mapper_args__' not in cls.__dict__:
cls.__mapper_args__ = {}
# 设置 polymorphic_on（默认为 _polymorphic_name）
if 'polymorphic_on' not in cls.__mapper_args__:
cls.__mapper_args__['polymorphic_on'] = polymorphic_on or '_polymorphic_name'
# 自动检测或设置 polymorphic_abstract
if 'polymorphic_abstract' not in cls.__mapper_args__:
if polymorphic_abstract is None:
# 自动检测：如果继承了 ABC 且有抽象方法，则为抽象类
has_abc = ABC in cls.__mro__
has_abstract_methods = bool(getattr(cls, '__abstractmethods__', set()))
polymorphic_abstract = has_abc and has_abstract_methods
cls.__mapper_args__['polymorphic_abstract'] = polymorphic_abstract
@classmethod
def _is_joined_table_inheritance(cls) -> bool:
"""
检测当前类是否使用联表继承（Joined Table Inheritance）
通过检查子类是否有独立的表来判断：
- JTI: 子类有独立的 local_table（与父类不同）
- STI: 子类与父类共用同一个 local_table
:return: True 表示 JTI，False 表示 STI 或无子类
"""
mapper = inspect(cls)
base_table_name = mapper.local_table.name
# 检查所有直接子类
for subclass in cls.__subclasses__():
sub_mapper = inspect(subclass)
# 如果任何子类有不同的表名，说明是 JTI
if sub_mapper.local_table.name != base_table_name:
return True
return False
@classmethod
def get_concrete_subclasses(cls) -> list[type['PolymorphicBaseMixin']]:
"""
递归获取当前类的所有具体（非抽象）子类
用于 selectin_polymorphic 加载策略，自动检测联表继承的所有具体子类。
可在任意多态基类上调用，返回该类的所有非抽象子类。
:return: 所有具体子类的列表（不包含 polymorphic_abstract=True 的抽象类）
"""
result: list[type[PolymorphicBaseMixin]] = []
for subclass in cls.__subclasses__():
# 使用 inspect() 获取 mapper 的公开属性
# 源码确认: mapper.polymorphic_abstract 是公开属性 (mapper.py:811)
mapper = inspect(subclass)
if not mapper.polymorphic_abstract:
result.append(subclass)
# 无论是否抽象，都需要递归（抽象类可能有具体子类）
if hasattr(subclass, 'get_concrete_subclasses'):
result.extend(subclass.get_concrete_subclasses())
return result
@classmethod
def get_polymorphic_discriminator(cls) -> str:
"""
获取多态鉴别字段名
使用 SQLAlchemy inspect 从 mapper 获取，支持从子类调用。
:return: 多态鉴别字段名（如 '_polymorphic_name'）
:raises ValueError: 如果类未配置 polymorphic_on
"""
polymorphic_on = inspect(cls).polymorphic_on
if polymorphic_on is None:
raise ValueError(
f"{cls.__name__} 未配置 polymorphic_on，"
f"请确保正确继承 PolymorphicBaseMixin"
)
return polymorphic_on.key
@classmethod
def get_identity_to_class_map(cls) -> dict[str, type['PolymorphicBaseMixin']]:
"""
获取 polymorphic_identity 到具体子类的映射
包含所有层级的具体子类（如 Function 和 ModelSwitchFunction 都会被包含）。
:return: identity 到子类的映射字典
"""
result: dict[str, type[PolymorphicBaseMixin]] = {}
for subclass in cls.get_concrete_subclasses():
identity = inspect(subclass).polymorphic_identity
if identity:
result[identity] = subclass
return result

470
sqlmodels/mixin/relation_preload.py
View File

@@ -1,470 +0,0 @@
"""
关系预加载 Mixin
提供方法级别的关系声明和按需增量加载，避免 MissingGreenlet 错误，同时保证 SQL 查询数理论最优。
设计原则：
- 按需加载：只加载被调用方法需要的关系
- 增量加载：已加载的关系不重复加载
- 查询最优：相同关系只查询一次，不同关系增量查询
- 零侵入：调用方无需任何改动
- Commit 安全：基于 SQLAlchemy inspect 检测真实加载状态，自动处理 expire
使用方式：
from sqlmodels.mixin import RelationPreloadMixin, requires_relations
class KlingO1VideoFunction(RelationPreloadMixin, Function, table=True):
kling_video_generator: KlingO1Generator = Relationship(...)
@requires_relations('kling_video_generator', KlingO1Generator.kling_o1)
async def cost(self, params, context, session) -> ToolCost:
# 自动加载，可以安全访问
price = self.kling_video_generator.kling_o1.pro_price_per_second
...
# 调用方 - 无需任何改动
await tool.cost(params, context, session) # 自动加载 cost 需要的关系
await tool._call(...) # 关系相同则跳过，否则增量加载
支持 AsyncGenerator：
@requires_relations('twitter_api')
async def _call(self, ...) -> AsyncGenerator[ToolResponse, None]:
yield ToolResponse(...) # 装饰器正确处理 async generator
"""
import inspect as python_inspect
from functools import wraps
from typing import Callable, TypeVar, ParamSpec, Any
from loguru import logger as l
from sqlalchemy import inspect as sa_inspect
from sqlmodel.ext.asyncio.session import AsyncSession
from sqlmodel.main import RelationshipInfo
P = ParamSpec('P')
R = TypeVar('R')
def _extract_session(
func: Callable,
args: tuple[Any, ...],
kwargs: dict[str, Any],
) -> AsyncSession | None:
"""
从方法参数中提取 AsyncSession
按以下顺序查找：
1. kwargs 中名为 'session' 的参数
2. 根据函数签名定位 'session' 参数的位置，从 args 提取
3. kwargs 中类型为 AsyncSession 的参数
"""
# 1. 优先从 kwargs 查找
if 'session' in kwargs:
return kwargs['session']
# 2. 从函数签名定位位置参数
try:
sig = python_inspect.signature(func)
param_names = list(sig.parameters.keys())
if 'session' in param_names:
# 计算位置（减去 self）
idx = param_names.index('session') - 1
if 0 <= idx < len(args):
return args[idx]
except (ValueError, TypeError):
pass
# 3. 遍历 kwargs 找 AsyncSession 类型
for value in kwargs.values():
if isinstance(value, AsyncSession):
return value
return None
def _is_obj_relation_loaded(obj: Any, rel_name: str) -> bool:
"""
检查对象的关系是否已加载（独立函数版本）
Args:
obj: 要检查的对象
rel_name: 关系属性名
Returns:
True 如果关系已加载，False 如果未加载或已过期
"""
try:
state = sa_inspect(obj)
return rel_name not in state.unloaded
except Exception:
return False
def _find_relation_to_class(from_class: type, to_class: type) -> str | None:
"""
在类中查找指向目标类的关系属性名
Args:
from_class: 源类
to_class: 目标类
Returns:
关系属性名，如果找不到则返回 None
Example:
_find_relation_to_class(KlingO1VideoFunction, KlingO1Generator)
# 返回 'kling_video_generator'
"""
for attr_name in dir(from_class):
try:
attr = getattr(from_class, attr_name, None)
if attr is None:
continue
# 检查是否是 SQLAlchemy InstrumentedAttribute（关系属性）
# parent.class_ 是关系所在的类，property.mapper.class_ 是关系指向的目标类
if hasattr(attr, 'property') and hasattr(attr.property, 'mapper'):
target_class = attr.property.mapper.class_
if target_class == to_class:
return attr_name
except AttributeError:
continue
return None
def requires_relations(*relations: str | RelationshipInfo) -> Callable[[Callable[P, R]], Callable[P, R]]:
"""
装饰器：声明方法需要的关系，自动按需增量加载
参数格式：
- 字符串：本类属性名，如 'kling_video_generator'
- RelationshipInfo：外部类属性，如 KlingO1Generator.kling_o1
行为：
- 方法调用时自动检查关系是否已加载
- 未加载的关系会被增量加载（单次查询）
- 已加载的关系直接跳过
支持：
- 普通 async 方法：`async def cost(...) -> ToolCost`
- AsyncGenerator 方法：`async def _call(...) -> AsyncGenerator[ToolResponse, None]`
Example:
@requires_relations('kling_video_generator', KlingO1Generator.kling_o1)
async def cost(self, params, context, session) -> ToolCost:
# self.kling_video_generator.kling_o1 已自动加载
...
@requires_relations('twitter_api')
async def _call(self, ...) -> AsyncGenerator[ToolResponse, None]:
yield ToolResponse(...) # AsyncGenerator 正确处理
验证：
- 字符串格式的关系名在类创建时（__init_subclass__）验证
- 拼写错误会在导入时抛出 AttributeError
"""
def decorator(func: Callable[P, R]) -> Callable[P, R]:
# 检测是否是 async generator 函数
is_async_gen = python_inspect.isasyncgenfunction(func)
if is_async_gen:
# AsyncGenerator 需要特殊处理：wrapper 也必须是 async generator
@wraps(func)
async def wrapper(self, *args: P.args, **kwargs: P.kwargs) -> R:
session = _extract_session(func, args, kwargs)
if session is not None:
await self._ensure_relations_loaded(session, relations)
# 委托给原始 async generator，逐个 yield 值
async for item in func(self, *args, **kwargs):
yield item # type: ignore
else:
# 普通 async 函数：await 并返回结果
@wraps(func)
async def wrapper(self, *args: P.args, **kwargs: P.kwargs) -> R:
session = _extract_session(func, args, kwargs)
if session is not None:
await self._ensure_relations_loaded(session, relations)
return await func(self, *args, **kwargs)
# 保存关系声明供验证和内省使用
wrapper._required_relations = relations # type: ignore
return wrapper
return decorator
class RelationPreloadMixin:
"""
关系预加载 Mixin
提供按需增量加载能力，确保 SQL 查询数理论最优。
特性：
- 按需加载：只加载被调用方法需要的关系
- 增量加载：已加载的关系不重复加载
- 原地更新：直接修改 self，无需替换实例
- 导入时验证：字符串关系名在类创建时验证
- Commit 安全：基于 SQLAlchemy inspect 检测真实状态，自动处理 expire
"""
def __init_subclass__(cls, **kwargs) -> None:
"""类创建时验证所有 @requires_relations 声明"""
super().__init_subclass__(**kwargs)
# 收集类及其父类的所有注解（包含普通字段）
all_annotations: set[str] = set()
for klass in cls.__mro__:
if hasattr(klass, '__annotations__'):
all_annotations.update(klass.__annotations__.keys())
# 收集 SQLModel 的 Relationship 字段（存储在 __sqlmodel_relationships__）
sqlmodel_relationships: set[str] = set()
for klass in cls.__mro__:
if hasattr(klass, '__sqlmodel_relationships__'):
sqlmodel_relationships.update(klass.__sqlmodel_relationships__.keys())
# 合并所有可用的属性名
all_available_names = all_annotations | sqlmodel_relationships
for method_name in dir(cls):
if method_name.startswith('__'):
continue
try:
method = getattr(cls, method_name, None)
except AttributeError:
continue
if method is None or not hasattr(method, '_required_relations'):
continue
# 验证字符串格式的关系名
for spec in method._required_relations:
if isinstance(spec, str):
# 检查注解、Relationship 或已有属性
if spec not in all_available_names and not hasattr(cls, spec):
raise AttributeError(
f"{cls.__name__}.{method_name} 声明了关系 '{spec}'，"
f"但 {cls.__name__} 没有此属性"
)
def _is_relation_loaded(self, rel_name: str) -> bool:
"""
检查关系是否真正已加载（基于 SQLAlchemy inspect）
使用 SQLAlchemy 的 inspect 检测真实加载状态，
自动处理 commit 导致的 expire 问题。
Args:
rel_name: 关系属性名
Returns:
True 如果关系已加载，False 如果未加载或已过期
"""
try:
state = sa_inspect(self)
# unloaded 包含未加载的关系属性名
return rel_name not in state.unloaded
except Exception:
# 对象可能未被 SQLAlchemy 管理
return False
async def _ensure_relations_loaded(
self,
session: AsyncSession,
relations: tuple[str | RelationshipInfo, ...],
) -> None:
"""
确保指定关系已加载，只加载未加载的部分
基于 SQLAlchemy inspect 检测真实状态，自动处理：
- 首次访问的关系
- commit 后 expire 的关系
- 嵌套关系（如 KlingO1Generator.kling_o1）
Args:
session: 数据库会话
relations: 需要的关系规格
"""
# 找出真正未加载的关系（基于 SQLAlchemy inspect）
to_load: list[str | RelationshipInfo] = []
# 区分直接关系和嵌套关系的 key
direct_keys: set[str] = set() # 本类的直接关系属性名
nested_parent_keys: set[str] = set() # 嵌套关系所需的父关系属性名
for rel in relations:
if isinstance(rel, str):
# 直接关系：检查本类的关系是否已加载
if not self._is_relation_loaded(rel):
to_load.append(rel)
direct_keys.add(rel)
else:
# 嵌套关系（InstrumentedAttribute）：如 KlingO1Generator.kling_o1
# 1. 查找指向父类的关系属性
parent_class = rel.parent.class_
parent_attr = _find_relation_to_class(self.__class__, parent_class)
if parent_attr is None:
# 找不到路径，可能是配置错误，但仍尝试加载
l.warning(
f"无法找到从 {self.__class__.__name__} 到 {parent_class.__name__} 的关系路径，"
f"无法检查 {rel.key} 是否已加载"
)
to_load.append(rel)
continue
# 2. 检查父对象是否已加载
if not self._is_relation_loaded(parent_attr):
# 父对象未加载，需要同时加载父对象和嵌套关系
if parent_attr not in direct_keys and parent_attr not in nested_parent_keys:
to_load.append(parent_attr)
nested_parent_keys.add(parent_attr)
to_load.append(rel)
else:
# 3. 父对象已加载，检查嵌套关系是否已加载
parent_obj = getattr(self, parent_attr)
if not _is_obj_relation_loaded(parent_obj, rel.key):
# 嵌套关系未加载：需要同时传递父关系和嵌套关系
# 因为 _build_load_chains 需要完整的链来构建 selectinload
if parent_attr not in direct_keys and parent_attr not in nested_parent_keys:
to_load.append(parent_attr)
nested_parent_keys.add(parent_attr)
to_load.append(rel)
if not to_load:
return # 全部已加载，跳过
# 构建 load 参数
load_options = self._specs_to_load_options(to_load)
if not load_options:
return
# 安全地获取主键值（避免触发懒加载）
state = sa_inspect(self)
pk_tuple = state.key[1] if state.key else None
if pk_tuple is None:
l.warning(f"无法获取 {self.__class__.__name__} 的主键值")
return
# 主键是元组，取第一个值（假设单列主键）
pk_value = pk_tuple[0]
# 单次查询加载缺失的关系
fresh = await self.__class__.get(
session,
self.__class__.id == pk_value,
load=load_options,
)
if fresh is None:
l.warning(f"无法加载关系：{self.__class__.__name__} id={self.id} 不存在")
return
# 原地复制到 self（只复制直接关系，嵌套关系通过父关系自动可访问）
all_direct_keys = direct_keys | nested_parent_keys
for key in all_direct_keys:
value = getattr(fresh, key, None)
object.__setattr__(self, key, value)
def _specs_to_load_options(
self,
specs: list[str | RelationshipInfo],
) -> list[RelationshipInfo]:
"""
将关系规格转换为 load 参数
- 字符串 → cls.{name}
- RelationshipInfo → 直接使用
"""
result: list[RelationshipInfo] = []
for spec in specs:
if isinstance(spec, str):
rel = getattr(self.__class__, spec, None)
if rel is not None:
result.append(rel)
else:
l.warning(f"关系 '{spec}' 在类 {self.__class__.__name__} 中不存在")
else:
result.append(spec)
return result
# ==================== 可选的手动预加载 API ====================
@classmethod
def get_relations_for_method(cls, method_name: str) -> list[RelationshipInfo]:
"""
获取指定方法声明的关系（用于外部预加载场景）
Args:
method_name: 方法名
Returns:
RelationshipInfo 列表
"""
method = getattr(cls, method_name, None)
if method is None or not hasattr(method, '_required_relations'):
return []
result: list[RelationshipInfo] = []
for spec in method._required_relations:
if isinstance(spec, str):
rel = getattr(cls, spec, None)
if rel:
result.append(rel)
else:
result.append(spec)
return result
@classmethod
def get_relations_for_methods(cls, *method_names: str) -> list[RelationshipInfo]:
"""
获取多个方法的关系并去重（用于批量预加载场景）
Args:
method_names: 方法名列表
Returns:
去重后的 RelationshipInfo 列表
"""
seen: set[str] = set()
result: list[RelationshipInfo] = []
for method_name in method_names:
for rel in cls.get_relations_for_method(method_name):
key = rel.key
if key not in seen:
seen.add(key)
result.append(rel)
return result
async def preload_for(self, session: AsyncSession, *method_names: str) -> 'RelationPreloadMixin':
"""
手动预加载指定方法的关系（可选优化 API）
当需要确保在调用方法前完成所有加载时使用。
通常情况下不需要调用此方法，装饰器会自动处理。
Args:
session: 数据库会话
method_names: 方法名列表
Returns:
self（支持链式调用）
Example:
# 可选：显式预加载（通常不需要）
tool = await tool.preload_for(session, 'cost', '_call')
"""
all_relations: list[str | RelationshipInfo] = []
for method_name in method_names:
method = getattr(self.__class__, method_name, None)
if method and hasattr(method, '_required_relations'):
all_relations.extend(method._required_relations)
if all_relations:
await self._ensure_relations_loaded(session, tuple(all_relations))
return self

1247
sqlmodels/mixin/table.py
View File

File diff suppressed because it is too large Load Diff

2
sqlmodels/model_base.py
View File

@@ -4,7 +4,7 @@ from enum import StrEnum
from sqlmodel import Field
from .base import SQLModelBase
from sqlmodel_ext import SQLModelBase
class ResponseBase(SQLModelBase):

3
sqlmodels/node.py
View File

@@ -3,8 +3,7 @@ from typing import TYPE_CHECKING
from sqlmodel import Field, Relationship, text, Index
from .base import SQLModelBase
from .mixin import TableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin
if TYPE_CHECKING:
from .download import Download

3
sqlmodels/object.py
View File

@@ -7,8 +7,7 @@ from enum import StrEnum
from sqlalchemy import BigInteger
from sqlmodel import Field, Relationship, CheckConstraint, Index, text
from .base import SQLModelBase
from .mixin import UUIDTableBaseMixin
from sqlmodel_ext import SQLModelBase, UUIDTableBaseMixin
if TYPE_CHECKING:
from .user import User

3
sqlmodels/order.py
View File

@@ -4,8 +4,7 @@ from uuid import UUID
from sqlmodel import Field, Relationship
from .base import SQLModelBase
from .mixin import TableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin
if TYPE_CHECKING:
from .user import User

3
sqlmodels/physical_file.py
View File

@@ -15,8 +15,7 @@ from uuid import UUID
from sqlalchemy import BigInteger
from sqlmodel import Field, Relationship, Index
from .base import SQLModelBase
from .mixin import UUIDTableBaseMixin
from sqlmodel_ext import SQLModelBase, UUIDTableBaseMixin
if TYPE_CHECKING:
from .object import Object

3
sqlmodels/policy.py
View File

@@ -4,8 +4,7 @@ from uuid import UUID
from enum import StrEnum
from sqlmodel import Field, Relationship, text
from .base import SQLModelBase
from .mixin import UUIDTableBaseMixin
from sqlmodel_ext import SQLModelBase, UUIDTableBaseMixin
if TYPE_CHECKING:
from .object import Object

3
sqlmodels/redeem.py
View File

@@ -2,8 +2,7 @@ from enum import StrEnum
from sqlmodel import Field, text
from .base import SQLModelBase
from .mixin import TableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin
class RedeemType(StrEnum):

3
sqlmodels/report.py
View File

@@ -4,8 +4,7 @@ from uuid import UUID
from sqlmodel import Field, Relationship
from .base import SQLModelBase
from .mixin import TableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin
if TYPE_CHECKING:
from .share import Share

4
sqlmodels/setting.py
View File

@@ -2,9 +2,9 @@ from enum import StrEnum
from sqlmodel import UniqueConstraint
from sqlmodel_ext import SQLModelBase, TableBaseMixin
from .auth_identity import AuthProviderType
from .base import SQLModelBase
from .mixin import TableBaseMixin
from .user import UserResponse
class CaptchaType(StrEnum):

4
sqlmodels/share.py
View File

@@ -5,9 +5,9 @@ from uuid import UUID
from sqlmodel import Field, Relationship, text, UniqueConstraint, Index
from .base import SQLModelBase
from sqlmodel_ext import SQLModelBase, UUIDTableBaseMixin
from .model_base import ResponseBase
from .mixin import UUIDTableBaseMixin
from .object import ObjectType
if TYPE_CHECKING:

3
sqlmodels/source_link.py
View File

@@ -4,8 +4,7 @@ from uuid import UUID
from sqlmodel import Field, Relationship, Index
from .base import SQLModelBase
from .mixin import TableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin
if TYPE_CHECKING:
from .object import Object

3
sqlmodels/storage_pack.py
View File

@@ -5,8 +5,7 @@ from uuid import UUID
from sqlmodel import Field, Relationship, Column, func, DateTime
from .base import SQLModelBase
from .mixin import TableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin
if TYPE_CHECKING:
from .user import User

3
sqlmodels/tag.py
View File

@@ -5,8 +5,7 @@ from datetime import datetime
from sqlmodel import Field, Relationship, UniqueConstraint, Column, func, DateTime
from .base import SQLModelBase
from .mixin import TableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin
if TYPE_CHECKING:
from .user import User

3
sqlmodels/task.py
View File

@@ -5,8 +5,7 @@ from datetime import datetime
from sqlmodel import Field, Relationship, CheckConstraint, Index
from .base import SQLModelBase
from .mixin import TableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin
if TYPE_CHECKING:
from .download import Download

4
sqlmodels/theme_preset.py
View File

@@ -3,9 +3,9 @@ from uuid import UUID
from sqlmodel import Field
from .base import SQLModelBase
from sqlmodel_ext import SQLModelBase, UUIDTableBaseMixin
from .color import ChromaticColor, NeutralColor, ThemeColorsBase
from .mixin import UUIDTableBaseMixin
class ThemePresetBase(SQLModelBase):

2
sqlmodels/uri.py
View File

@@ -2,7 +2,7 @@
from enum import StrEnum
from urllib.parse import urlparse, parse_qs, urlencode, quote, unquote
from .base import SQLModelBase
from sqlmodel_ext import SQLModelBase
class FileSystemNamespace(StrEnum):

4
sqlmodels/user.py
View File

@@ -9,11 +9,11 @@ from sqlmodel import Field, Relationship
from sqlmodel.ext.asyncio.session import AsyncSession
from sqlmodel.main import RelationshipInfo
from sqlmodel_ext import SQLModelBase, UUIDTableBaseMixin, TableViewRequest, ListResponse
from .auth_identity import AuthProviderType
from .base import SQLModelBase
from .color import ChromaticColor, NeutralColor, ThemeColorsBase
from .model_base import ResponseBase
from .mixin import UUIDTableBaseMixin, TableViewRequest, ListResponse
T = TypeVar("T", bound="User")

3
sqlmodels/user_authn.py
View File

@@ -5,8 +5,7 @@ from uuid import UUID
from sqlalchemy import Column, Text
from sqlmodel import Field, Relationship
from .base import SQLModelBase
from .mixin import TableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin
if TYPE_CHECKING:
from .user import User

3
sqlmodels/webdav.py
View File

@@ -4,8 +4,7 @@ from uuid import UUID
from sqlmodel import Field, Relationship, UniqueConstraint
from .base import SQLModelBase
from .mixin import TableBaseMixin
from sqlmodel_ext import SQLModelBase, TableBaseMixin
if TYPE_CHECKING:
from .user import User

83
sqlmodels/wopi.py Normal file
View File

@@ -0,0 +1,83 @@
"""
WOPI（Web Application Open Platform Interface）协议模型
提供 WOPI CheckFileInfo 响应模型和 WOPI 访问令牌 Payload 定义。
"""
from uuid import UUID
from sqlmodel_ext import SQLModelBase
class WopiFileInfo(SQLModelBase):
"""
WOPI CheckFileInfo 响应模型。
字段命名遵循 WOPI 规范（PascalCase），通过 alias 映射。
参考: https://learn.microsoft.com/en-us/microsoft-365/cloud-storage-partner-program/rest/files/checkfileinfo
"""
base_file_name: str
"""文件名（含扩展名）"""
size: int
"""文件大小（字节）"""
owner_id: str
"""文件所有者标识"""
user_id: str
"""当前用户标识"""
user_friendly_name: str
"""用户显示名"""
version: str
"""文件版本标识（使用 updated_at）"""
sha256: str = ""
"""文件 SHA256 哈希（如果可用）"""
user_can_write: bool = False
"""用户是否可写"""
user_can_not_write_relative: bool = True
"""是否禁止创建关联文件"""
read_only: bool = True
"""文件是否只读"""
supports_locks: bool = False
"""是否支持锁（v1 不实现）"""
supports_update: bool = True
"""是否支持更新"""
def to_wopi_dict(self) -> dict[str, str | int | bool]:
"""转换为 WOPI 规范的 PascalCase 字典"""
return {
"BaseFileName": self.base_file_name,
"Size": self.size,
"OwnerId": self.owner_id,
"UserId": self.user_id,
"UserFriendlyName": self.user_friendly_name,
"Version": self.version,
"SHA256": self.sha256,
"UserCanWrite": self.user_can_write,
"UserCanNotWriteRelative": self.user_can_not_write_relative,
"ReadOnly": self.read_only,
"SupportsLocks": self.supports_locks,
"SupportsUpdate": self.supports_update,
}
class WopiAccessTokenPayload(SQLModelBase):
"""WOPI 访问令牌内部 Payload"""
file_id: UUID
"""文件UUID"""
user_id: UUID
"""用户UUID"""
can_write: bool = False
"""是否可写"""