Skip to content

Dev ych 20260305 #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ChihuaYing merged 3 commits into from
May 27, 2026
Merged

Dev ych 20260305 #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9ca940a
修理一个访问服务中路径解析的bug
ChihuaYing May 25, 2026
c68c87e
调整文档数据知识提取prompt
ChihuaYing May 25, 2026
8a34984
1.支持显示PB级数据 2.增加脚本添加数据源的请求发送延迟
ChihuaYing May 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion backend/src/main/java/com/storage/engine/service/adapter/StorageUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,6 @@ public static String toFileLeafPath(String iginxPath, String originalFileName) {
if (slash >= 0 && slash < base.length() - 1) {
base = base.substring(slash + 1);
}
base = base.replaceAll("[^a-zA-Z0-9._-]", "_");
if (base.isEmpty()) {
base = "content.bin";
}
Expand Down
3 changes: 2 additions & 1 deletion backend/src/main/resources/udf/metadata/extractors/document_extractor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ class DocumentMetadataExtractor(BaseMetadataExtractor):

USER_PROMPT_TEMPLATE = (
"请从输入的文档文本中抽取语义三元组。 "
"尽可能提取代表元数据的语义，而非具体的数值或事实。 "
"请严格只返回 JSON，不要输出 Markdown 或解释。 "
"返回格式要求："
"{{\"entities\":[\"entity\"],\"triples\":[{{\"subject\":\"entityA\",\"predicate\":\"relation\",\"object\":\"entityB\"}}]}}. "
Expand All @@ -15,7 +16,7 @@ class DocumentMetadataExtractor(BaseMetadataExtractor):
)

USER_RETRY_PROMPT_TEMPLATE = (
"你上一轮可能没有返回可用三元组。请再次检查内容并尽量抽取核心语义关系；若确实无关系，triples 返回空数组。 "
"你上一轮可能没有返回可用三元组。请再次检查内容并尽量抽取核心语义关系；尽可能提取代表元数据的语义，而非具体的数值或事实；若确实无关系，triples 返回空数组。 "
"请严格只返回 JSON，不要输出 Markdown 或解释。 "
"返回格式要求："
"{{\"entities\":[\"entity\"],\"triples\":[{{\"subject\":\"entityA\",\"predicate\":\"relation\",\"object\":\"entityB\"}}]}}. "
Expand Down
4 changes: 2 additions & 2 deletions frontend/app.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ function renderPagination(containerId, stateKey, totalPages, total, onPageChange
function formatBytes(bytes) {
const value = Number(bytes || 0);
if (!Number.isFinite(value) || value <= 0) return '0 B';
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
const units = ['B', 'KB', 'MB', 'GB', 'TB', 'PB'];
let size = value;
let idx = 0;
while (size >= 1024 && idx < units.length - 1) {
Expand Down Expand Up @@ -2982,7 +2982,7 @@ function escapeHtml(str) {

function formatFileSize(bytes) {
if (!bytes || bytes === 0) return '0 B';
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
const units = ['B', 'KB', 'MB', 'GB', 'TB', 'PB'];
const i = Math.floor(Math.log(bytes) / Math.log(1024));
return (bytes / Math.pow(1024, i)).toFixed(1) + ' ' + units[i];
}
Expand Down
7 changes: 5 additions & 2 deletions test/batch_add_datasource.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
SSH_PORT = 22

REQUEST_DELAY = 0.1 # 请求间隔（秒）
BATCH_SIZE = 10 # 每批请求数量
BATCH_REQUEST_DELAY = 0 # 每批请求间隔（秒）
TIMEOUT = 60 # 请求超时（秒）

# ==================== 颜色输出 ====================
Expand Down Expand Up @@ -124,8 +126,8 @@ def main():
Colors.RED
)

# 每10个打印进度
if i % 10 == 0:
# 每 BATCH_SIZE 个打印进度
if i % BATCH_SIZE == 0:
progress = (i * 100.0) / TOTAL_COUNT
print()
print_colored(
Expand All @@ -134,6 +136,7 @@ def main():
Colors.YELLOW
)
print()
time.sleep(BATCH_REQUEST_DELAY)

# 延迟
time.sleep(REQUEST_DELAY)
Expand Down
Loading