Merge branch 'OpenDCAI:main' into main

Author: HeRunming

.dockerignore

@@ -0,0 +1,26 @@
+# Git
+.git
+.gitignore
+
+# Docker
+.dockerignore
+
+# IDE settings
+.vscode/
+.idea/
+
+# Python artifacts
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+*.egg-info/
+dist/
+build/
+*.egg
+*.whl
+
+# Virtual environments
+.venv
+venv/
+env/

.gitignore

@@ -6,6 +6,7 @@
 /dataflow/example/ReasoningPipeline/pipeline_math_step*.jsonl
 !/dataflow/example/ReasoningPipeline/pipeline_math.json
 
+.DS_Store
 
 dataflow/example/KBCleaningPipeline/raw
 requirement_added.txt

Dockerfile

@@ -0,0 +1,39 @@
+# Base Image
+FROM --platform=linux/amd64 nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04
+
+# Environment variables
+ENV DEBIAN_FRONTEND=noninteractive \
+    LANG=C.UTF-8 LC_ALL=C.UTF-8 \
+    PYTHONUNBUFFERED=1 \
+    PIP_NO_CACHE_DIR=1
+
+# System dependencies
+RUN apt-get update && apt-get install -y --no-install-recommends \
+      python3 python3-venv python3-pip python3-dev \
+      git build-essential pkg-config \
+      ffmpeg libgl1 libglib2.0-0 \
+      ca-certificates curl \
+    && rm -rf /var/lib/apt/lists/*
+
+# Python environment
+RUN python3 -m venv /opt/venv
+ENV PATH="/opt/venv/bin:${PATH}"
+RUN python -m pip install --upgrade pip wheel
+
+# Setup pip mirror
+RUN mkdir -p /etc && \
+    printf "[global]\nindex-url = https://pypi.tuna.tsinghua.edu.cn/simple\ntimeout = 120\ntrusted-host = pypi.tuna.tsinghua.edu.cn\n" > /etc/pip.conf
+
+# Set up the application directory and copy source code into a subdirectory
+# DataFlow Commit b27d6bc24cf86835fda7bc6fe1a289cb9eb63bd2
+WORKDIR /app
+COPY . ./DataFlow/
+
+# Set the working directory to the project source
+WORKDIR /app/DataFlow
+
+# Install the project in editable mode with its dependencies
+RUN pip install -e ".[vllm]"
+
+# Set the container's default command to an interactive shell
+CMD ["/bin/bash"]

README-zh.md

@@ -120,45 +120,50 @@ open-dataflow codebase version: 1.0.0
 You are using the latest version: 1.0.0.
 ```
 
-### 🚀 5.2 使用Gradio Web界面
+#### 🐳 5.1.1 Docker安装（可选方式）
 
-DataFlow提供了两个交互式Web界面，帮助你使用算子、流水线和智能体：
+我们还提供了 **Dockerfile** 以便于部署，同时也提供了**预构建的 Docker 镜像**供您直接使用。
 
-#### 5.2.1 DataFlow算子界面
+##### 方式一：使用预构建的 Docker 镜像
 
-启动DataFlow算子界面来测试和可视化所有算子和流水线：
+您可以直接拉取并使用我们预构建的 Docker 镜像：
 
-```bash
-dataflow webui
-```
+```shell
+# 拉取预构建镜像
+docker pull molyheci/dataflow:cu124
 
-该命令将启动一个交互式 Web 界面，使你能够可视化并灵活使用所有算子和流水线。
+# 使用 GPU 支持运行容器
+docker run --gpus all -it molyheci/dataflow:cu124
 
-#### 5.2.2 DataFlow智能体界面
+# 在容器内验证安装
+dataflow -v
+```
 
-启动DataFlow智能体界面进行算子编写和流水线设计：
+##### 方式二：从 Dockerfile 构建
 
-```bash
-dataflow webui agent
-```
+或者，您也可以从项目提供的 Dockerfile 构建镜像：
 
-该命令将启动 DataFlow-Agent 界面，提供自动化算子编写功能和流水线推荐服务。
+```shell
+# 克隆代码仓库（HTTPS 方式）
+git clone https://github.com/OpenDCAI/DataFlow.git
+# 或使用 SSH 方式
+# git clone git@github.com:OpenDCAI/DataFlow.git
 
-https://github.com/user-attachments/assets/5c6aa003-9504-4e2a-9f4e-97bae739894a
+cd DataFlow
 
-### 🌐 5.3 ADP智能数据平台
+# 构建 Docker 镜像
+docker build -t dataflow:custom .
 
-除了本地Gradio界面，DataFlow还提供了基于Web的ADP智能数据平台：[https://adp.originhub.tech/login](https://adp.originhub.tech/login)
+# 运行容器
+docker run --gpus all -it dataflow:custom
 
-ADP是OriginHub推出的智能数据平台，具备四大核心能力：DataFlow数据准备全流程自动化、融合大规模多模态知识库的知识系统、多Agent协同的智能协作，以及支撑数据全链路管理的AI数据库，旨在加速企业通过AI能力充分发挥独有数据的价值。
+# 在容器内验证安装
+dataflow -v
+```
 
-<p align="center">
-  <a href="https://adp.originhub.tech/login">
-    <img src="https://github.com/user-attachments/assets/c63ac954-f0c8-4a1a-bfc8-5752c25a22cf" alt="ADP Platform Interface" width="75%">
-  </a>
-</p>
+> **注意**：Docker 镜像包含 CUDA 12.4.1 支持，并预装了 vLLM 用于 GPU 加速。请确保您已安装 [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) 以使用 GPU 功能。
 
-### 📖 5.4 参考DataFlow项目文档
+### 📖 5.2 参考DataFlow项目文档
 
 详细**使用说明**和**入门指南**，请参考我们的 [项目文档](https://OpenDCAI.github.io/DataFlow-Doc/)。
 

README.md

@@ -129,57 +129,50 @@ open-dataflow codebase version: 1.0.0
 You are using the latest version: 1.0.0.
 ```
 
-### 🚀 5.2 Using the Gradio Web Interface
+#### 🐳 5.1.1 Docker Installation (Alternative)
 
-DataFlow provides two interactive web interfaces to help you use operators, pipelines, and agents:
+We also provide a **Dockerfile** for easy deployment and a **pre-built Docker image** for immediate use.
 
-#### 5.2.1 DataFlow Operators Interface
+##### Option 1: Use Pre-built Docker Image
 
-Launch the DataFlow operator interface to test and visualize all operators and pipelines:
+You can directly pull and use our pre-built Docker image:
 
-```bash
-dataflow webui
-```
-
-This command will start an interactive web interface, allowing you to visualize and flexibly use all operators and pipelines.
-
-#### 5.2.2 DataFlow Agent Interface
+```shell
+# Pull the pre-built image
+docker pull molyheci/dataflow:cu124
 
-Launch the DataFlow agent interface for operator authoring and pipeline design:
+# Run the container with GPU support
+docker run --gpus all -it molyheci/dataflow:cu124
 
-```bash
-dataflow webui agent
+# Inside the container, verify installation
+dataflow -v
 ```
 
-This command will start the DataFlow-Agent interface, providing automated operator authoring and pipeline recommendation services.
-
-https://github.com/user-attachments/assets/fda1ad47-a9f3-447a-b5c0-cf4c9ad64763
+##### Option 2: Build from Dockerfile
 
-### 🌐 5.3 ADP Intelligent Data Platform
+Alternatively, you can build the Docker image from the provided Dockerfile:
 
-Beyond the local Gradio interface, **DataFlow** is also available as a fully-managed SaaS solution on the **ADP Intelligent Data Platform**.
-
-[**ADP**](https://adp.originhub.tech) is an end-to-end system by OriginHub, designed to help enterprises accelerate the development of custom Agents and Models by integrating Large Language Models (LLMs) with private data.
-
-#### Core Capabilities:
+```shell
+# Clone the repository (HTTPS)
+git clone https://github.com/OpenDCAI/DataFlow.git
+# Or use SSH
+# git clone git@github.com:OpenDCAI/DataFlow.git
 
-* 🤖 **Automated Data Preparation**: Leverage DataFlow for full-process automation of your data workflows.
-* 📚 **Unified Knowledge System**: Integrate and manage large-scale, multimodal knowledge bases.
-* 🤝 **Intelligent Collaboration**: Build and orchestrate powerful multi-agent systems.
-* 🗄️ **AI-Native Database**: Manage the full lifecycle of your multimodal data with a purpose-built AI database.
+cd DataFlow
 
-<p align="center">
-  <a href="https://adp.originhub.tech/login">
-    <img src="https://github.com/user-attachments/assets/c63ac954-f0c8-4a1a-bfc8-5752c25a22cf" alt="ADP Platform Interface" width="75%">
-  </a>
-</p>
+# Build the Docker image
+docker build -t dataflow:custom .
 
-#### Get Started for Free
+# Run the container
+docker run --gpus all -it dataflow:custom
 
+# Inside the container, verify installation
+dataflow -v
+```
 
-👉 **[Sign up now to claim your free compute credits!](https://adp.originhub.tech)**
+> **Note**: The Docker image includes CUDA 12.4.1 support and comes with vLLM pre-installed for GPU acceleration. Make sure you have [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) installed to use GPU features.
 
-### 📖 5.4 Reference Project Documentation
+### 📖 5.2 Reference Project Documentation
 
 For detailed **usage instructions** and **getting started guide**, please visit our [Documentation](https://OpenDCAI.github.io/DataFlow-Doc/).