gavin/duix.ai
1
0
Fork 0
mirror of https://github.com/GuijiAI/duix.ai.git synced 2026-03-12 17:51:43 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main'

Browse Source
This commit is contained in:
jingtonghuai
2025-12-16 13:50:17 +08:00
parent b5165b1468 6a33f4e406
commit 23aa96e2e8
49 changed files with 1836 additions and 1724 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.gitattributes vendored Normal file
View File

@@ -0,0 +1 @@
*.mp4 filter=lfs diff=lfs merge=lfs -text

BIN
Guiji-duix.ai-sdk-1.0.3 Model Community License_EN.pdf
View File

Binary file not shown.

8
GuijiAI:duix.ai.webloc
View File

@@ -1,8 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>URL</key>
<string>https://github.com/GuijiAI/duix.ai</string>
</dict>
</plist>

BIN
Interface Reference Handbook_EN.pdf
View File

Binary file not shown.

78
LICENSE
View File

@@ -1,32 +1,70 @@
Silicon Intelligence COMMUNITY LICENSE AGREEMENT
# DUIX.COM COMMUNITY LICENSE AGREEMENT
“Agreement” means the terms and conditions for use, reproduction, distribution and modification of this product forth herein.
**"Agreement"** means the terms and conditions for use, reproduction, distribution and modification of this product forth herein.
“Documentation” means the specifications, manuals and documentation by Silicon Intelligence.
**"Documentation"** means the specifications, manuals and documentation by duix.com.
“Licensee” or “you” means you, or your employer or any other person or entity (if you are entering into this Agreement on such person or entity’s behalf), of the age required under applicable laws, rules or regulations to provide legal consent and that has legal authority to bind your employer or such other person or entity if you are entering in this Agreement on their behalf.
**"Licensee" or "you"** means you, or your employer or any other person or entity (if you are entering into this Agreement on such person or entity's behalf), of the age required under applicable laws, rules or regulations to provide legal consent and that has legal authority to bind your employer or such other person or entity if you are entering in this Agreement on their behalf.
“Silicon Intelligence Materials” means, collectively, Silicon Intelligence’s proprietary code and Documentation (and any portion thereof) made available under this Agreement.
**"duix.com Materials"** means, collectively, duix.com's proprietary code and Documentation (and any portion thereof) made available under this Agreement.
By clicking “I Accept” below or by using or distributing any portion or element of the Silicon Intelligence Materials, you agree to be bound by this Agreement.
By clicking **"I Accept"** below or by using or distributing any portion or element of the duix.com Materials, you agree to be bound by this Agreement.
1. License Rights and Redistribution.
---
a. Grant of Rights. You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under ’s intellectual property or other rights owned by Silicon Intelligence embodied in the SILICON INTELLIGENCE Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the Silicon Intelligence Materials.
b. Redistribution and Use.
i. If you distribute or make available the Silicon Intelligence Materials (or any derivative works thereof), or a product or service that uses any of them, you shall (A) provide a copy of this Agreement with any such Silicon Intelligence Materials; and (B) prominently display “Built with Silicon Intelligence” on a related website, user interface, blogpost, about page, or product documentation. If you use the Silicon Intelligence Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Silicon Intelligence” at the beginning of any such AI model name.
ii. If you receive Silicon Intelligence Materials, or any derivative works thereof, from a Licensee as part of an integrated end user product, then Section 2 of this Agreement will not apply to you.
iii. You must retain in all copies of the Silicon Intelligence Materials that you distribute the following attribution notice within a “Notice” text file distributed as a part of such copies: “Silicon Intelligence is licensed under the Silicon Intelligence Community License, Copyright © Silicon Intelligence Platforms, Inc. All Rights Reserved.”
iv. Your use of the Silicon Intelligence Materials must comply with applicable laws and regulations (including trade compliance laws and regulations) .
## 1. License Rights and Redistribution
2. Additional Commercial Terms. If, on the Silicon Intelligence duix.ai version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 1 thousand monthly active users in the preceding calendar month, or your product based Silicon Intelligence material your active users greater 1 thousand, you must request a license from Silicon Intelligence, which Silicon Intelligence may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Silicon Intelligence otherwise expressly grants you such rights.
### a. Grant of Rights
You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under duix.com's intellectual property or other rights owned by duix.com embodied in the duix.com Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the duix.com Materials.
3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE SILICON INTELLIGENCE MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND SILICON INTELLIGENCE DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE SILICON INTELLIGENCE MATERIALS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE SILICON INTELLIGENCE MATERIALS AND ANY OUTPUT AND RESULTS.
### b. Redistribution and Use
i. If you distribute or make available the duix.com Materials (or any derivative works thereof), or a product or service that uses any of them, you shall:
- (A) provide a copy of this Agreement with any such duix.com Materials; and
- (B) prominently display **"Powered by Duix.com"** on a related website, user interface, blog post, about page, or product documentation.
4. Limitation of Liability. IN NO EVENT WILL SILICON INTELLIGENCE OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF SILICON INTELLIGENCE OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
If you use the duix/duix.com Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include **"duix.com"** at the beginning of any such AI model name.
5. Intellectual Property.
a. No trademark licenses are granted under this Agreement, and in connection with the Silicon Intelligence Materials, neither Silicon Intelligence nor Licensee may use any name or mark owned by or associated with the other or any of its affiliates, except as required for reasonable and customary use in describing and redistributing the Silicon Intelligence Materials or as set forth in this Section 5(a). Silicon Intelligence hereby grants you a license to use “Silicon Intelligence” solely as required to comply with the last sentence of Section 1.b.i. You will comply with Silicon Intelligence’s brand guidelines . All goodwill arising out of your use of the Mark will inure to the benefit of Silicon Intelligence.
b. If you institute litigation or other proceedings against Silicon Intelligenceor any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Silicon Intelligence Materials or outputs or results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or claim is filed or instituted. You will indemnify and hold harmless Silicon Intelligence from and against any claim by any third party arising out of or related to your use or distribution of the Silicon Intelligence Materials.
ii. If you receive duix.com Materials, or any derivative works thereof, from a Licensee as part of an integrated end user product, then Section 2 of this Agreement will not apply to you.
6. Term and Termination. The term of this Agreement will commence upon your acceptance of this Agreement or access to the Silicon Intelligence Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Silicon Intelligence may terminate this Agreement if you are in breach of any term or condition of this Agreement. Upon termination of this Agreement, you shall delete and cease use of the Silicon Intelligence Materials. Sections 3, 4 shall survive the termination of this Agreement.
iii. You must retain in all copies of the duix.com Materials that you distribute the following attribution notice within a "Notice" text file distributed as a part of such copies:
> "duix.com is licensed under the duix.com Community License, Copyright © duix.com Platforms, Inc. All Rights Reserved."
iv. Your use of the duix.com Materials must comply with applicable laws and regulations (including trade compliance laws and regulations).
---
## 2. Additional Commercial Terms
If, on the duix.com version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee's affiliates, is greater than 1 thousand monthly active users in the preceding calendar month, or your product based duix.com material has active users greater than 1 thousand, you must request a license from duix.com, which duix.com may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until duix.com otherwise expressly grants you such rights.
---
## 3. Disclaimer of Warranty
UNLESS REQUIRED BY APPLICABLE LAW, THE DUIX.COM MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, AND DUIX.COM DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE DUIX.COM MATERIALS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE DUIX.COM MATERIALS AND ANY OUTPUT AND RESULTS.
---
## 4. Limitation of Liability
IN NO EVENT WILL DUIX.COM OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF DUIX.COM OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
---
## 5. Intellectual Property
### a. Trademark and Branding
No trademark licenses are granted under this Agreement, and in connection with the duix.com Materials, neither duix.com nor Licensee may use any name or mark owned by or associated with the other or any of its affiliates, except as required for reasonable and customary use in describing and redistributing the duix.com Materials or as set forth in this Section 5(a).
duix.com hereby grants you a license to use "duix.com" solely as required to comply with the last sentence of Section 1.b.i.
You will comply with duix.com's brand guidelines.
All goodwill arising out of your use of the Mark will insure to the benefit of duix.com.
### b. Litigation and Indemnification
If you institute litigation or other proceedings against duix.com or any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the duix.com Materials or outputs or results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or claim is filed or instituted.
You will indemnify and hold harmless duix.com from and against any claim by any third party arising out of or related to your use or distribution of the duix.com Materials.
---
## 6. Term and Termination
The term of this Agreement will commence upon your acceptance of this Agreement or access to the duix.com Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein.
duix.com may terminate this Agreement if you are in breach of any term or condition of this Agreement.
Upon termination of this Agreement, you shall delete and cease use of the duix.com Materials.
Sections 3 and 4 shall survive the termination of this Agreement.

192
privacystatement.md → PRIVACYSTATEMENT.md
View File

@@ -1,96 +1,96 @@
# Duix SDK PrivaDcy Statement
**Last Updated:** 2025/06/09
**SDK Name:** Duix Real-time Interaction SDK
**Provider:** Duix.com Platform
## 1. Introduction
This Privacy Statement explains how [Duix.com] ("we," "us," or "our") processes user data through our open-source SDK when integrated into third-party applications ("Customer Apps"). The SDK enables real-time digital human interaction capabilities (voice/text).
**Key Notes for Developers:**
- **Compliance Responsibility:** Customers integrating this SDK must comply with applicable privacy laws and disclose our SDK in their privacy policy.
- **Consent Requirement:** Customers must obtain valid user consent before initializing the SDK or collecting data.
- **Minimal Data Collection:** We collect only essential data for core functionality and operational stability.
---
## 2. Data Collection & Purpose
### A. Core Functionality Data
| **Data Type** | **Purpose** |
|-------------------------|-----------------------------------------------------------------------------|
| Audio/Text Input | Process user queries to enable real-time digital human interactions. |
| Interaction Metadata | Maintain session stability (e.g., timestamps, session IDs). |
### B. Operational Data
| **Data Type** | **Purpose** | **Retention** |
|-----------------------------|-----------------------------------------------------------------------------|---------------------|
| Device Information<br>(e.g., OS) | Monitor SDK performance, diagnose errors, and improve reliability. | 6 months |
| Anonymized Usage Metrics<br>(e.g., feature adoption rates) | Optimize resource allocation and service quality. | Aggregated indefinitely |
**Note:**
- Audio/text data is processed in real-time and **not stored** after session completion.
---
## 3. Permissions & Controls
### Required Permissions
| **Permission** | **Platforms** | **Purpose** |
|------------------------|---------------------|---------------------------------------------------------------------------|
| Microphone Access | Android, iOS, Web | Capture voice input for real-time interactions. |
| Network State | Android, iOS, Web | Ensure stable connectivity during sessions. |
### Optional Permissions
| **Permission** | **Purpose** |
|------------------------|-----------------------------------------------------------------------------|
| Camera Access | Enable video simulation features (disabled by default). |
| Wake Lock | Prevent device sleep during active sessions. |
**Developer Obligations:**
- **Consent Layers:** Implement granular consent for permissions (e.g., separate toggles for microphone/camera).
- **Delayed Initialization:** Initialize the SDK only after obtaining user consent.
- **Configuration:** Disable unused features (e.g., camera) via SDK parameters.
---
## 4. Data Sharing
- **Third-Party Processors:** Audio/text data may be sent to AI models designated by the Customer App. We do not control these processors.
- **No Cross-Context Advertising:** We never sell user data.
---
## 5. User Rights & Compliance
### A. User Rights
Users may:
- Access, correct, or delete their data.
- Withdraw consent or object to processing.
- Request data portability.
**Fulfillment Process:**
1. Users must contact the **Customer App developer** to exercise rights.
2. We provide APIs to support data deletion/access requests upon developer request.
### B. Children’s Privacy
- The SDK is **not intended for users under 13** (or 16 in some regions).
- Customers must implement age-gating and obtain parental consent for child users.
---
## 6. Security Measures
- **Encryption:** Data in transit (TLS 1.3+) .
---
## 7. Policy Updates
- Changes will be communicated via GitHub repository releases and version tags.
- Customers must update integrated SDK versions to reflect changes.
---
## 8. Contact Us
For privacy inquiries:
- **Developers:** Open issues in our GitHub Repository：[https://github.com/duixcom].
- **End-Users:** Contact the Customer App’s support team.
- **Legal Requests:** james@duix.com
**DPO Contact:** james@duix.com
# Duix SDK Privacy Statement
**Last Updated:** 2025/06/09
**SDK Name:** Duix Real-time Interaction SDK
**Provider:** Duix.com Platform
## 1. Introduction
This Privacy Statement explains how [Duix.com] ("we," "us," or "our") processes user data through our open-source SDK when integrated into third-party applications ("Customer Apps"). The SDK enables real-time digital human interaction capabilities (voice/text).
**Key Notes for Developers:**
- **Compliance Responsibility:** Customers integrating this SDK must comply with applicable privacy laws and disclose our SDK in their privacy policy.
- **Consent Requirement:** Customers must obtain valid user consent before initializing the SDK or collecting data.
- **Minimal Data Collection:** We collect only essential data for core functionality and operational stability.
---
## 2. Data Collection & Purpose
### A. Core Functionality Data
| **Data Type** | **Purpose** |
|-------------------------|-----------------------------------------------------------------------------|
| Audio/Text Input | Process user queries to enable real-time digital human interactions. |
| Interaction Metadata | Maintain session stability (e.g., timestamps, session IDs). |
### B. Operational Data
| **Data Type** | **Purpose** | **Retention** |
|-----------------------------|-----------------------------------------------------------------------------|---------------------|
| Device Information<br>(e.g., OS) | Monitor SDK performance, diagnose errors, and improve reliability. | 6 months |
| Anonymized Usage Metrics<br>(e.g., feature adoption rates) | Optimize resource allocation and service quality. | Aggregated indefinitely |
**Note:**
- Audio/text data is processed in real-time and **not stored** after session completion.
---
## 3. Permissions & Controls
### Required Permissions
| **Permission** | **Platforms** | **Purpose** |
|------------------------|---------------------|---------------------------------------------------------------------------|
| Microphone Access | Android, iOS, Web | Capture voice input for real-time interactions. |
| Network State | Android, iOS, Web | Ensure stable connectivity during sessions. |
### Optional Permissions
| **Permission** | **Purpose** |
|------------------------|-----------------------------------------------------------------------------|
| Camera Access | Enable video simulation features (disabled by default). |
| Wake Lock | Prevent device sleep during active sessions. |
**Developer Obligations:**
- **Consent Layers:** Implement granular consent for permissions (e.g., separate toggles for microphone/camera).
- **Delayed Initialization:** Initialize the SDK only after obtaining user consent.
- **Configuration:** Disable unused features (e.g., camera) via SDK parameters.
---
## 4. Data Sharing
- **Third-Party Processors:** Audio/text data may be sent to AI models designated by the Customer App. We do not control these processors.
- **No Cross-Context Advertising:** We never sell user data.
---
## 5. User Rights & Compliance
### A. User Rights
Users may:
- Access, correct, or delete their data.
- Withdraw consent or object to processing.
- Request data portability.
**Fulfillment Process:**
1. Users must contact the **Customer App developer** to exercise rights.
2. We provide APIs to support data deletion/access requests upon developer request.
### B. Children’s Privacy
- The SDK is **not intended for users under 13** (or 16 in some regions).
- Customers must implement age-gating and obtain parental consent for child users.
---
## 6. Security Measures
- **Encryption:** Data in transit (TLS 1.3+) .
---
## 7. Policy Updates
- Changes will be communicated via GitHub repository releases and version tags.
- Customers must update integrated SDK versions to reflect changes.
---
## 8. Contact Us
For privacy inquiries:
- **Developers:** Open issues in our GitHub Repository：[https://github.com/duixcom].
- **End-Users:** Contact the Customer App’s support team.
- **Legal Requests:** support@duix.com
**DPO Contact:** support@duix.com

209
README.md
View File

@@ -1,153 +1,118 @@
简体中文 | [English](/README_en.md)
English | [中文](/README_zh.md)
[![Duix Mobile thumbnail](./res/main_video_thumbnail.jpg)](https://www.bilibili.com/video/BV1t2g7z3ERK/)
# 🚀🚀🚀 Duix Mobile — The Best Real-time Interactive AI Avatar Solution for Mobile Devices
# 🚀🚀🚀 Duix Mobile —— 全网效果最好的移动端【实时对话数字人】
🔗 **Official website**：[www.duix.com](http://www.duix.com)
**📱 跨平台支持：iOS / Android / 平板 / 车载系统 / VR设备 / IoT终端 / 大屏交互等**
**📱 Cross-platform support: iOS / Android / Tablet / Automotive / VR / IoT / Large Screen Interaction, etc.**
## 😎 Duix Mobile 是什么？
https://github.com/user-attachments/assets/6cfb59fc-d4bb-4c9f-8a2b-54009ce594a1
本次由硅基智能开源的 Duix Mobile，是一个**可部署在手机或嵌入式屏幕的实时对话数字人 SDK**。
## 😎 What is Duix Mobile?
开发者可以轻松集成自有或第三方的大语言模型（LLM）、语音识别（ASR）和语音合成（TTS）服务，快速构建能与用户自然对话的数字人界面。
Duix Mobile is an open-source SDK developed by [www.duix.com](http://www.duix.com) that enables developers to create real-time interactive AI avatars directly on **mobile devices** or **embedded screens**. It is designed for on-device deployment, with no dependency on cloud servers, making it lightweight, private, and highly responsive.
Duix Mobile 支持一键跨平台部署（Android/iOS），上手门槛低，适用于智能客服、虚拟医生、虚拟律师、虚拟陪伴、虚拟教学等多种应用场景。
Developers can easily integrate their own or third-party Large Language Models (LLM), Automatic Speech Recognition (ASR), and Text-to-Speech (TTS) services to quickly build AI avatar interfaces that can naturally converse with users.
现在就开始构建你自己的交互数字人，大幅提升你的产品业绩吧！
Duix Mobile supports one-click cross-platform deployment (Android/iOS), has a low learning curve, and is suitable for various application scenarios such as intelligent customer service, virtual doctors, virtual lawyers, virtual companions, and virtual tutors.
## 🤩 有哪些应用场景？
Start building your own interactive AI avatar now and significantly boost your product performance!
- Duix Mobile 在 Andorid/iOS/Pad/大屏等设备下可以支持到多种实际应用场景；
- 大幅度提升你的产品表现力，从而提升你的营收水平。
## 🤩 Application Scenarios
![](./res/example.png)
- Duix Mobile supports various practical application scenarios across Android/iOS/Pad/large screen devices;
- Significantly enhance your product performance and boost your revenue levels.
## 🥳 有什么优势？
<!-- ![example.png](./res/example.png) -->
- **仿真数字人体验**：自然呈现面部表情、语调和情绪共鸣，打造「像人一样」的 AI 对话。
- **支持流式音频**：边合成、边说话，支持中途打断、抢话，让数字人不仅会说话，而且更像「人」。
- **极致响应速度**：数字人响应延迟低于 120ms（测试设备为骁龙® 8 Gen 2 SoC），带来毫秒级流畅互动体验。
- **成本友好，随处部署**：轻量化运行，资源占用极低，轻松适配手机、平板、智能屏等终端。
- **无惧弱网环境**：核心处理本地完成，对网络依赖极低，尤其适合金融、政务、法律等高稳定性场景。
- **全行业适配**：模块化设计，支持快速定制，轻松打造各行业专属数字人解决方案。
## 🥳 Advantages
## 📑 开发文档
- **Realistic AI avatar Experience**: Natural facial expressions, tone, and emotional cues enable truly human-like conversations.
- **Streaming Audio Support**: Synthesize and speak simultaneously, supports interruption and barge-in, making AI avatars not only talk but also behave more "human-like".
- **Ultra-Low Latency**: AI avatar response latency under 120ms (tested on Snapdragon® 8 Gen 2 SoC), delivering millisecond-level smooth interaction experience.
- **Cost-Friendly, Deploy Anywhere**: Lightweight operation, extremely low resource consumption, easily adaptable to phones, tablets, smart screens, and other terminals.
- **Stable in Poor Networks**: Core functions run locally with low network dependence, ideal for finance, government, and legal use cases.
- **Modular & Customizable**: Designed with modularity to support fast customization of industry-specific AI avatar solutions.
- Android 开发者：[Duix Mobile SDK for Android](./duix-android/dh_aigc_android/README.md)
- iOS 开发者：[Duix Mobile SDK for iOS](./duix-ios/GJLocalDigitalDemo/README.md)
## 📑 Development Documentation
## 💚 实际部署案例
- For Android Developers: [Duix Mobile SDK for Android](./duix-android/dh_aigc_android/README.md)
- For iOS Developers: [Duix Mobile SDK for iOS](./duix-ios/GJLocalDigitalDemo/README.md)
前往哔哩哔哩查看：
- [《程序员与奶奶的虚拟重逢》](https://www.bilibili.com/video/BV1QSgczPESS)
- [《Grok 遇见 Duix，谁才是你的真女友？》](https://www.bilibili.com/video/BV1Dbg3zbExC/)
## ✨ Public AI avatar Downloads
## ✨ 公用数字人下载
- 以下是 Duix 提供的 8 个公有数字人，可供下载和集成。
- 4 public AI avatars provided by Duix, available for download and integration.
<table>
<tr>
<td align="center">
<img src="./res/avatar/1.png" alt="Model 1" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/guilv0515_20240516_optim_m80.zip"><button>下载</button></a>
</td>
<td align="center">
<img src="./res/avatar/2.png" alt="Model 2" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/guilv3_20240511_optim_m80.zip"><button>下载</button></a>
</td>
<td align="center">
<img src="./res/avatar/3.png" alt="Model 3" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/wuhao_20240418_optim_m80.zip"><button>下载</button></a>
</td>
<td align="center">
<img src="./res/avatar/8.png" alt="Model 8" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/siyao_20240418_optim_m80.zip"><button>下载</button></a>
</td>
</tr>
<tr>
<td align="center">
<img src="./res/avatar/5.jpg" alt="Model 5" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/696309955760197_fdc4a25a012c99789cd1ec95f5faf0de_optim_m80.zip"><button>下载</button></a>
</td>
<td align="center">
<img src="./res/avatar/6.png" alt="Model 6" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/696303589556293_268307125eeeff7e2c85461dd8c3ac52_optim_m80.zip"><button>下载</button></a>
</td>
<td align="center">
<img src="./res/avatar/7.jpg" alt="Model 7" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/696326678212677_9d4da9041e81466f5dadc99ddd1e3bd9.zip"><button>下载</button></a>
</td>
<td align="center">
<img src="./res/avatar/4.png" alt="Model 4" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/651686686687301_846161843f9ffdaaeace716bf3436be5_optim_m80.zip"><button>下载</button></a>
</td>
</tr>
</table>
<tr>
<td align="center">
<img src="./res/avatar/Leo.jpg" alt="Model 5" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v2.0.1/Leo.zip">Download</a>
</td>
<td align="center">
<img src="./res/avatar/Oliver.jpg" alt="Model 6" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v2.0.1/Oliver.zip">Download</a>
</td>
<td align="center">
<img src="./res/avatar/Sofia.jpg" alt="Model 6" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v2.0.1/Sofia.zip">Download</a>
</td>
<td align="center">
<img src="./res/avatar/Lily.jpg" alt="Model 6" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v2.0.1/Lily.zip">Download</a>
</td>
</tr>
</table>
## 🤗 如何定制私有数字人？
View more AI avatars online：[www.duix.com](http://www.duix.com)
- 部署遇到问题？想要定制私有化数字人？
- 请发邮件至邮箱：`amos.young@duix.com`
- 或者加入技术支持群：
## 🤗 How to Customize Private AI avatars?
<img src="./res/contact.png" alt="企业微信" width="260">
- Please send an email to: [support@duix.com](mailto:support@duix.com)
## 🙌 常见问题解答
## 🙌 Frequently Asked Questions
<details>
<summary>我可以集成自己的大模型（LLM）、语音识别（ASR）和语音合成（TTS）吗？</summary>
<summary>Can I integrate my own Large Language Model (LLM), Speech-to-Text(ASR), and Text-to-Speech (TTS)?</summary>
当然可以，你可以将 Duix Mobile 的数字人与你的自己 LLM、ASR 和 TTS 进行集成。
Yes, Duix Mobile supports full integration with custom or third-party LLM, ASR, and TTS services.
</details>
<details>
<summary>Does it support "lip synchronization"?</summary>
Yes, it does.
</details>
<details>
<summary>Does it support "multilingual subtitles"?</summary>
Yes, it does.
</details>
<details>
<summary>How can I create custom AI avatars?</summary>
We offer 4 public avatar models. For custom avatars, please contact us via the email address above.
Usually, providing a 15-second to 2-minute video is typically sufficient for customization.
</details>
<details>
<summary>Is streaming audio supported?</summary>
Yes, streaming audio with barge-in support is available from the July 17, 2025 release.
</details>
<details>
<summary>Are voice start/end callbacks available?</summary>
Yes, callback events for voice start and end are fully documented.
</details>
<details>
<summary>是否支持「唇动同步」？</summary>
支持。
</details>
<details>
<summary>是否支持「多语种字幕」？</summary>
支持。
</details>
<details>
<summary>我如何创建自定义数字人？</summary>
我们提供了 8 个公有数字人，如需额外定制，请联系上方的企业微信。
通常录制 15 秒至 2 分钟的视频即可完成定制过程，简单便捷。
</details>
<details>
<summary>是否支持流式音频？</summary>
支持，流式音频已于 2025 年 7 月 17 日版本更新中上线。
</details>
<details>
<summary>是否提供数字人语音开始和结束的回调？</summary>
是的，我们提供语音开始和结束的回调文档。
</details>
## 💡 版本计划
- [x] 流式音频能力，2025 年 7 月 16 日上线
- [ ] 算法响应优化，预计时间：2025 年 8 月 30 日前
## 📚 相关开源仓库
- GitHub: https://github.com/duixcom/Duix-Mobile
- Gitee: https://gitee.com/duix/Duix-Mobile
- GitCode: https://gitcode.com/openguiji/duix-mobile
## ❇️ Other projects by Duix
- [Duix.com](http://Duix.com) - Easily integrable cloud-based real-time interactive AI avatar
- [Duix-Avatar](https://github.com/duixcom/Duix.Avatar) - The true open-source AI avatar video production
- [Duix-Reface](https://github.com/duixcom/Duix-Reface) - Truly open-source real-time, high-fidelity face-swap engine for AI avatar

152
README_en.md
View File

@@ -1,152 +0,0 @@
[简体中文](/README.md) | English
[![Duix Mobile 视频封面](./res/main_video_thumbnail.jpg)](https://www.bilibili.com/video/BV1t2g7z3ERK/)
# 🚀🚀🚀 Duix Mobile — The Best Real-time Interactive Digital Human Solution for Mobile Devices
**📱 Cross-platform support: iOS / Android / Tablet / Automotive / VR / IoT / Large Screen Interaction, etc.**
## 😎 What is Duix Mobile?
Duix Mobile, open-sourced by GuijiAI, is a **real-time conversational digital human SDK that can be deployed on mobile phones or embedded screens**.
Developers can easily integrate their own or third-party Large Language Models (LLM), Automatic Speech Recognition (ASR), and Text-to-Speech (TTS) services to quickly build digital human interfaces that can naturally converse with users.
Duix Mobile supports one-click cross-platform deployment (Android/iOS), has a low learning curve, and is suitable for various application scenarios such as intelligent customer service, virtual doctors, virtual lawyers, virtual companionship, and virtual teaching.
Start building your own interactive digital human now and significantly boost your product performance!
## 🤩 Application Scenarios
- Duix Mobile supports various practical application scenarios across Android/iOS/Pad/large screen devices;
- Significantly enhance your product performance and boost your revenue levels.
![](./res/example.png)
## 🥳 Advantages
- **Realistic Digital Human Experience**: Natural presentation of facial expressions, tone, and emotional resonance, creating "human-like" AI conversations.
- **Streaming Audio Support**: Synthesize and speak simultaneously, supports interruption and barge-in, making digital humans not only talk but also behave more "human-like".
- **Ultimate Response Speed**: Digital human response latency under 120ms (tested on Snapdragon® 8 Gen 2 SoC), delivering millisecond-level smooth interaction experience.
- **Cost-Friendly, Deploy Anywhere**: Lightweight operation, extremely low resource consumption, easily adaptable to phones, tablets, smart screens, and other terminals.
- **Resilient in Weak Network Environments**: Core processing completed locally, minimal network dependency, especially suitable for scenarios requiring high stability like finance, government, and legal sectors.
- **Comprehensive Industry Adaptation**: Modular design, supports rapid customization, easily create industry-specific digital human solutions.
## 📑 Development Documentation
- For Android Developers: [Duix Mobile SDK for Android](./duix-android/dh_aigc_android/README_en.md)
- For iOS Developers: [Duix Mobile SDK for iOS](./duix-ios/GJLocalDigitalDemo/README_en.md)
## 💚 Real Deployment Cases
See on Bilibili:
- ["A Programmer's Virtual Reunion with Grandma"](https://www.bilibili.com/video/BV1QSgczPESS)
- ["Grok Meets Duix: Who Is Your Real Girlfriend?"](https://www.bilibili.com/video/BV1Dbg3zbExC/)
## ✨ Public Digital Human Downloads
- Below are 8 public digital humans provided by Duix, available for download and integration.
<table>
<tr>
<td align="center">
<img src="./res/avatar/1.png" alt="Model 1" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/guilv0515_20240516_optim_m80.zip"><button>Download</button></a>
</td>
<td align="center">
<img src="./res/avatar/2.png" alt="Model 2" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/guilv3_20240511_optim_m80.zip"><button>Download</button></a>
</td>
<td align="center">
<img src="./res/avatar/3.png" alt="Model 3" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/wuhao_20240418_optim_m80.zip"><button>Download</button></a>
</td>
<td align="center">
<img src="./res/avatar/8.png" alt="Model 8" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/siyao_20240418_optim_m80.zip"><button>Download</button></a>
</td>
</tr>
<tr>
<td align="center">
<img src="./res/avatar/5.jpg" alt="Model 5" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/696309955760197_fdc4a25a012c99789cd1ec95f5faf0de_optim_m80.zip"><button>Download</button></a>
</td>
<td align="center">
<img src="./res/avatar/6.png" alt="Model 6" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/696303589556293_268307125eeeff7e2c85461dd8c3ac52_optim_m80.zip"><button>Download</button></a>
</td>
<td align="center">
<img src="./res/avatar/7.jpg" alt="Model 7" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/696326678212677_9d4da9041e81466f5dadc99ddd1e3bd9.zip"><button>Download</button></a>
</td>
<td align="center">
<img src="./res/avatar/4.png" alt="Model 4" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v1.0.0/651686686687301_846161843f9ffdaaeace716bf3436be5_optim_m80.zip"><button>Download</button></a>
</td>
</tr>
</table>
## 🤗 How to Customize Private Digital Humans?
- Having deployment issues? Want to customize private digital humans?
- Please send an email to: `amos.young@duix.com`
- Or add our enterprise WeChat:
<img src="./res/contact.png" alt="Enterprise WeChat" width="260">
## 🙌 Frequently Asked Questions
<details>
<summary>Can I integrate my own Large Language Model (LLM), Speech Recognition (ASR), and Text-to-Speech (TTS)?</summary>
Yes, you can integrate Duix-Mobile's digital humans with your own LLM, ASR, and TTS.
</details>
<details>
<summary>Does it support "lip synchronization"?</summary>
Yes, it does.
</details>
<details>
<summary>Does it support "multilingual subtitles"?</summary>
Yes, it does.
</details>
<details>
<summary>How do I create custom digital humans?</summary>
We provide 8 public digital humans. For additional customization, please contact the enterprise WeChat above.
Usually, recording a 15-second to 2-minute video is sufficient to complete the customization process, making it simple and convenient.
</details>
<details>
<summary>Is streaming audio supported?</summary>
Yes, streaming audio was released in the July 17, 2025 update.
</details>
<details>
<summary>Are callbacks provided for digital human voice start and end?</summary>
Yes, we provide documentation for voice start and end callbacks.
</details>
## 💡 Version Roadmap
- [x] Streaming audio capability, completed by July 16, 2025
- [ ] Algorithm response optimization, expected by August 30, 2025
## 📚 Related Open Source Repositories
- GitHub: https://github.com/duixcom/Duix-Mobile
- Gitee: https://gitee.com/duix/Duix-Mobile
- GitCode: https://gitcode.com/openguiji/duix-mobile

138
README_zh.md Normal file
View File

@@ -0,0 +1,138 @@
# Duix-Mobile 中文文档
中文 | [English](/README.md)
<a href="https://www.bilibili.com/video/BV1t2g7z3ERK/" target="_blank">
<img src="./res/main_video_thumbnail.webp" alt="Duix Mobile thumbnail">
</a>
# 🚀🚀🚀 Duix Mobile —— 全网效果最好的移动端【实时对话数字人】
🔗 **官方网站**：[www.duix.com](http://www.duix.com)
**📱 跨平台支持：iOS / Android / 平板 / 车载系统 / VR设备 / IoT终端 / 大屏交互等**
## 😎 Duix Mobile 是什么？
本次由硅基智能开源的 Duix Mobile，是一个**可部署在手机或嵌入式屏幕的实时对话数字人 SDK**。
开发者可以轻松集成自有或第三方的大语言模型（LLM）、语音识别（ASR）和语音合成（TTS）服务，快速构建能与用户自然对话的数字人界面。
Duix Mobile 支持一键跨平台部署（Android/iOS），上手门槛低，适用于智能客服、虚拟医生、虚拟律师、虚拟陪伴、虚拟教学等多种应用场景。
现在就开始构建你自己的交互数字人，大幅提升你的产品业绩吧！
## 🤩 有哪些应用场景？
- Duix Mobile 在 Andorid/iOS/Pad/大屏等设备下可以支持到多种实际应用场景；
- 大幅度提升你的产品表现力，从而提升你的营收水平。
![](./res/example.png)
## 🥳 有什么优势？
- **仿真数字人体验**：自然呈现面部表情、语调和情绪共鸣，打造「像人一样」的 AI 对话。
- **支持流式音频**：边合成、边说话，支持中途打断、抢话，让数字人不仅会说话，而且更像「人」。
- **极致响应速度**：数字人响应延迟低于 120ms（测试设备为骁龙® 8 Gen 2 SoC），带来毫秒级流畅互动体验。
- **成本友好，随处部署**：轻量化运行，资源占用极低，轻松适配手机、平板、智能屏等终端。
- **无惧弱网环境**：核心处理本地完成，对网络依赖极低，尤其适合金融、政务、法律等高稳定性场景。
- **全行业适配**：模块化设计，支持快速定制，轻松打造各行业专属数字人解决方案。
## 📑 开发文档
- Android 开发者：[Duix Mobile SDK for Android](./duix-android/dh_aigc_android/README_zh.md)
- iOS 开发者：[Duix Mobile SDK for iOS](./duix-ios/GJLocalDigitalDemo/README_zh.md)
## 💚 实际部署案例
前往哔哩哔哩查看：
- [《程序员与奶奶的虚拟重逢》](https://www.bilibili.com/video/BV1QSgczPESS)
- [《Grok 遇见 Duix，谁才是你的真女友？》](https://www.bilibili.com/video/BV1Dbg3zbExC/)
## ✨ 公用数字人下载
- 以下是 Duix 提供的 4 个公有数字人，可供下载和集成。
<table>
<tr>
<td align="center">
<img src="./res/avatar/Leo.jpg" alt="Model 5" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v2.0.1/Leo.zip">下载</a>
</td>
<td align="center">
<img src="./res/avatar/Oliver.jpg" alt="Model 6" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v2.0.1/Oliver.zip">下载</a>
</td>
<td align="center">
<img src="./res/avatar/Sofia.jpg" alt="Model 6" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v2.0.1/Sofia.zip">下载</a>
</td>
<td align="center">
<img src="./res/avatar/Lily.jpg" alt="Model 6" width="100%"><br>
<a href="https://github.com/duixcom/Duix.mobile/releases/download/v2.0.1/Lily.zip">下载</a>
</td>
</tr>
</table>
## 🤗 如何定制私有数字人？
- 请发邮件至邮箱：`support@duix.com`
## 🙌 常见问题解答
<details>
<summary>我可以集成自己的大模型（LLM）、语音识别（ASR）和语音合成（TTS）吗？</summary>
当然可以，你可以将 Duix Mobile 的数字人与你的自己 LLM、ASR 和 TTS 进行集成。
</details>
<details>
<summary>是否支持「唇动同步」？</summary>
支持。
</details>
<details>
<summary>是否支持「多语种字幕」？</summary>
支持。
</details>
<details>
<summary>我如何创建自定义数字人？</summary>
我们提供了 8 个公有数字人，如需额外定制，请联系上方的企业微信。
通常录制 15 秒至 2 分钟的视频即可完成定制过程，简单便捷。
</details>
<details>
<summary>是否支持流式音频？</summary>
支持，流式音频已于 2025 年 7 月 17 日版本更新中上线。
</details>
<details>
<summary>是否提供数字人语音开始和结束的回调？</summary>
是的，我们提供语音开始和结束的回调文档。
</details>
## 📚 相关开源仓库
- GitHub: https://github.com/duixcom/Duix-Mobile
- Gitee: https://gitee.com/duix/Duix-Mobile
- GitCode: https://gitcode.com/openguiji/duix-mobile
## ❇️ Duix其他项目
- [Duix.com](http://Duix.com) - Easily integrable cloud-based real-time interactive AI avatar
- [Duix-Avatar](https://github.com/duixcom/Duix.Avatar) - The true open-source AI avatar video production
- [Duix-Reface](https://github.com/duixcom/Duix-Reface) - Truly open-source real-time, high-fidelity face-swap engine for AI avatar

364
duix-android/dh_aigc_android/README.md
View File

@@ -1,53 +1,51 @@
# Duix Mobile for Android SDK 文档
# Duix Mobile for Android SDK Documentation
简体中文 | [English](./README_en.md)
English | [中文](./README_zh.md)
## 一、产品介绍
## 1. Product Overview
`Duix Mobile for Android` 是一套轻量级、纯离线的 Android 平台 2D 虚拟人解决方案，支持通过语音音频驱动数字人形象并进行实时渲染。
`Duix Mobile for Android` is a lightweight, fully offline 2D digital human solution for Android, supporting real-time rendering of digital avatars driven by voice audio.
### 1.1 应用场景
### 1.1 Application Scenarios
- **部署成本低**：适用于大屏终端、政务展厅、银行等无人值守场景。
- **网络依赖小**：完全本地运行，无需联网，可在地铁、偏远地区稳定运行。
- **功能多样化**：可服务于导览讲解、问答客服、智能陪伴等多种业务形态。
- **Low deployment cost**: Suitable for unattended scenarios such as large-screen terminals, government halls, and banks.
- **Minimal network dependency**: Runs entirely locally, no internet required, stable operation in subways and remote areas.
- **Diverse functionality**: Can serve as a guide, Q&A customer service, intelligent companion, and more.
### 1.2 核心功能
### 1.2 Core Features
- 数字人形象定制与本地渲染
- 实时语音驱动播报（支持 WAV 播放和 PCM 推送）
- 动作播放控制（指定动作、随机动作）
- 资源自动下载管理
- Customizable digital avatar and local rendering
- Real-time voice-driven playback (supports WAV playback and PCM streaming)
- Motion playback control (specific or random actions)
- Automatic resource download management
---
## 二、术语说明
## 2. Terminology
| 术语 | 含义 |
|-------------------|------------------------------------------------------------------------|
| PCM | Pulse-Code Modulation，16kHz 采样率、16bit 位深、Mono 单通道的原始音频流 |
| WAV | 一种音频文件格式，支持 PCM 编码，适合短语音播放 |
| RenderSink | 渲染数据接收接口，由 SDK 提供实现，可用于自定义渲染或默认展示 |
| DUIX | 数字人主控对象，集成了模型加载、渲染、播报、动作等能力 |
| GLES | OpenGL ES，Android 渲染图像用到的图形接口 |
| SpecialAction | 模型附带的 JSON 文件，标注动作区间（例如打招呼、挥手等） |
| Term | Meaning |
|--------------------|----------------------------------------------------------------------------|
| PCM | Pulse-Code Modulation, raw audio stream with 16kHz sample rate, 16-bit depth, Mono channel |
| WAV | An audio file format that supports PCM encoding, suitable for short voice playback |
| RenderSink | Rendering data reception interface, implemented by the SDK, can be used for custom rendering or default display |
| DUIX | Main control object of the digital human, integrates model loading, rendering, broadcasting, and motion control |
| GLES | OpenGL ES, a graphics interface for rendering images on Android |
| SpecialAction | A JSON file attached to the model that marks action intervals (e.g., greetings, waving) |
---
## 三、SDK 获取方式
## 3. SDK Access
> ✅ 当前 SDK 以本地 Module 或 AAR 方式提供。可联系商务或技术支持获取 SDK 包。
### 3.1 Module Reference (Recommended)
### 3.1 Module 引用（推荐）
1. 获取完整源码包，解压后将 `duix-sdk` 目录复制到项目根目录下。
2. 在项目 `settings.gradle` 中添加：
1. Obtain the complete source package, unzip it, and copy the `duix-sdk` directory to the project root directory.
2. In the project `settings.gradle`, add:
```gradle
include ':duix-sdk'
```
3. 在模块 `build.gradle` 中添加依赖：
3. In the module's `build.gradle`, add the dependency:
```gradle
dependencies {
@@ -55,10 +53,10 @@ dependencies {
}
```
### 3.2 AAR 引用（可选）
### 3.2 AAR Reference (Optional)
1. 将duix-sdk模块编译的 `duix-sdk-release.aar` 放入 `libs/` 目录。
2. 添加依赖：
1. Place the compiled `duix-sdk-release.aar` module into the `libs/` directory.
2. Add the dependency:
```gradle
dependencies {
@@ -68,75 +66,71 @@ dependencies {
---
## 四、集成要求
## 4. Integration Requirements
| 项目 | 描述 |
|--------|----------------------------------------------------|
| 系统 | 支持 Android 10+ 系统。 |
| CPU架构 | armeabi-v7a, arm64-v8a |
| 硬件要求 | 要求设备 CPU8 核及以上(骁龙8 Gen2),内存 8G 及以上。可用存储空间 1GB 及以上。 |
| 网络 | 无（完全本地运行） |
| 开发 IDE | Android Studio Giraffe 2022.3.1 Patch 2 |
| 内存要求 | 可用于数字人的内存 >= 800MB |
**编译项目的Gradle使用的JDK版本为17,需要在File->Setting->Build,Execution,Deployment->Grade Projects->Gradle JDK: ${选择一个17版本的JDK}**
| Item | Description |
|----------------|-----------------------------------------------------------------|
| System | Supports Android 10+ systems. |
| CPU Architecture | armeabi-v7a, arm64-v8a |
| Hardware Requirements | Device CPU with 8 or more cores (Snapdragon 8 Gen 2), 8GB or more memory, available storage space of 1GB or more |
| Network | None (Fully local operation) |
| Development IDE | Android Studio Giraffe 2022.3.1 Patch 2 |
| Memory Requirements | Minimum 800MB memory available for the digital human |
---
## 五、使用流程概览
## 5. Usage Flow Overview
```mermaid
graph TD
A[检查配置与模型] --> B[构建 DUIX 实例]
B --> C[调用 init 初始化]
C --> D[展示形象 / 渲染]
D --> E[PCM 或 WAV 音频驱动]
E --> F[播放控制与动作触发]
F --> G[资源释放]
A[Check Configuration and Models] --> B[Build DUIX Instance]
B --> C[Call init to Initialize]
C --> D[Display Avatar / Render]
D --> E[PCM or WAV Audio Driving]
E --> F[Playback Control & Motion Triggering]
F --> G[Resource Release]
```
---
## 六、关键接口与调用示例
## 6. Key Interfaces and Example Calls
### 6.1. 模型检查及下载
### 6.1 Model Check and Download
使用渲染服务前需要将基础配置及模型文件同步到本地存储中,SDK中提供了VirtualModelUtil简单演示了模型下载解压流程。
若模型下载过慢或无法下载，开发者可以选择将模型包缓存到自己的存储服务。
Before using the rendering service, ensure that the basic configuration and model files are synchronized to local storage. The SDK provides a simple demonstration of the model download and decompression process using `VirtualModelUtil`. If model download is slow or fails, developers can choose to cache the model package to their own storage service.
> 函数定义: `ai.guiji.duix.sdk.client.VirtualModelUtil`
> Function Definition: `ai.guiji.duix.sdk.client.VirtualModelUtil`
```
// 检查基础配置是否已下载
// Check if base configuration is downloaded
boolean checkBaseConfig(Context context)
// 检查模型是否已下载
// Check if the model is downloaded
boolean checkModel(Context context, String name)
// 基础配置下载
// Base configuration download
void baseConfigDownload(Context context, String url, ModelDownloadCallback callback)
// 模型下载
// Model download
void modelDownload(Context context, String modelUrl, ModelDownloadCallback callback)
```
`ModelDownloadCallback` 包含进度、完成、失败等回调，详见 SDK 定义。
`ModelDownloadCallback` includes progress, completion, failure callbacks, etc., as defined in the SDK.
```
interface ModelDownloadCallback {
// 下载进度
// Download progress
void onDownloadProgress(String url, long current, long total);
// 解压进度
// Unzip progress
void onUnzipProgress(String url, long current, long total);
// 下载解压完成
// Download and unzip complete
void onDownloadComplete(String url, File dir);
// 下载解压失败
// Download and unzip failed
void onDownloadFail(String url, int code, String msg);
}
```
**调用示例**:
**Call Example**:
```kotlin
if (!VirtualModelUtil.checkBaseConfig(mContext)){
@@ -148,36 +142,34 @@ if (!VirtualModelUtil.checkBaseConfig(mContext)){
if (!VirtualModelUtil.checkModel(mContext, modelUrl)){
VirtualModelUtil.modelDownload(mContext, modelUrl, callback)
}
```
---
### 6.2. 初始化与渲染启动
### 6.2 Initialization and Rendering Start
在渲染页onCreate()阶段构建DUIX对象并调用init接口
In the `onCreate()` stage of the rendering page, build the DUIX object and call the init interface.
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
> Function Definition: `ai.guiji.duix.sdk.client.DUIX`
```
// 构建DUIX对象
// Build DUIX object
public DUIX(Context context, String modelName, RenderSink sink, Callback callback)
// 初始化DUIX服务
// Initialize DUIX service
void init()
```
**DUIX对象构建说明**:
**DUIX Object Construction Explanation**:
| 参数 | 类型 | 描述 |
|------------|------------|-------------------------------------|
| context | Context | 系统上下文 |
| modelName | String | 可以传递模型下载的URL(已下载完成)或缓存的文件名 |
| render | RenderSink | 渲染数据接口，sdk提供了默认的渲染组件继承自该接口，也可以自己实现 |
| callback | Callback | SDK处理的各种回调事件 |
| Parameter | Type | Description |
|---------------|-----------|----------------------------------------------------------------|
| context | Context | System context |
| modelName | String | Can pass the model download URL (if downloaded) or cached filename |
| render | RenderSink| Rendering data interface, SDK provides a default rendering component inheriting from this interface, or you can implement it yourself |
| callback | Callback | Various callback events handled by the SDK |
其中**Callback**的定义: `ai.guiji.duix.sdk.client.Callback`
Where **Callback** is defined as: `ai.guiji.duix.sdk.client.Callback`
```
interface Callback {
@@ -185,7 +177,7 @@ interface Callback {
}
```
**调用示例**:
**Call Example**:
```kotlin
duix = DUIX(mContext, modelUrl, mDUIXRender) { event, msg, info ->
@@ -198,38 +190,37 @@ duix = DUIX(mContext, modelUrl, mDUIXRender) { event, msg, info ->
initError()
}
// ...
}
}
// 异步回调结果
// Asynchronous callback result
duix?.init()
```
在init回调中确认初始化结果
In the `init` callback, confirm the initialization result.
---
### 6.3. 数字人形象展示
### 6.3 Digital Human Avatar Display
使用 SDK 提供的 `DUIXRenderer` 和 `DUIXTextureView` 可快速实现支持透明通道的渲染。也可以自己实现RenderSink接口自定义渲染逻辑。
Use the SDK-provided `DUIXRenderer` and `DUIXTextureView` to quickly implement rendering with transparency support. Alternatively, you can implement the `RenderSink` interface to customize the rendering logic.
其中**RenderSink**的定义如下: `ai.guiji.duix.sdk.client.render.RenderSink`
The **RenderSink** definition is as follows: `ai.guiji.duix.sdk.client.render.RenderSink`
```java
/**
* 渲染管道，通过该接口返回渲染数据
* Rendering pipeline, returns rendering data through this interface
*/
public interface RenderSink {
// frame中的buffer数据以bgr顺序排列
// The frame's buffer data is arranged in BGR order
void onVideoFrame(ImageFrame imageFrame);
}
```
**调用示例**:
**Call Example**:
使用DUIXRenderer及DUIXTextureView控件简单实现渲染展示,该控件支持透明通道可以自由设置背景及前景
Use `DUIXRenderer` and `DUIXTextureView` to quickly implement rendering. These controls support transparency and can freely set the background and foreground.
```kotlin
override fun onCreate(savedInstanceState: Bundle?) {
@@ -242,11 +233,11 @@ override fun onCreate(savedInstanceState: Bundle?) {
)
binding.glTextureView.setEGLContextClientVersion(GL_CONTEXT_VERSION)
binding.glTextureView.setEGLConfigChooser(8, 8, 8, 8, 16, 0) // 透明
binding.glTextureView.isOpaque = false // 透明
binding.glTextureView.setEGLConfigChooser(8, 8, 8, 8, 16, 0) // Transparency
binding.glTextureView.isOpaque = false // Transparency
binding.glTextureView.setRenderer(mDUIXRender)
binding.glTextureView.renderMode =
GLSurfaceView.RENDERMODE_WHEN_DIRTY // 一定要在设置完Render之后再调用
GLSurfaceView.RENDERMODE_WHEN_DIRTY // Must be called after setting the renderer
duix = DUIX(mContext, modelUrl, mDUIXRender) { event, msg, _ ->
}
@@ -256,31 +247,30 @@ override fun onCreate(savedInstanceState: Bundle?) {
---
### 6.4 播报控制
### 6.4 Broadcasting Control
#### 使用流式推送PCM驱动数字人播报
#### Use Streaming PCM to Drive Digital Human Broadcasting
**PCM格式:16k采样率单通道16位深**
**PCM Format: 16kHz sample rate, single channel, 16-bit depth**
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
> Function Definition: `ai.guiji.duix.sdk.client.DUIX`
```
// 通知服务开始推送音频
// Notify service to start pushing audio
void startPush()
// 推送PCM数据
// Push PCM data
void pushPcm(byte[] buffer)
// 完成一段音频推送(音频推送完就调要该函数，而不是等播放完成再调用。)
// Finish a segment of audio push (Call this after the audio push is complete, not after playback finishes)
void stopPush()
```
startPush、pushPcm、stopPush需要成对调用，pushPcm不宜过长。可以在一整段音频推送完后调用stopPush结束当前会话，下一段音频再使用startPush重新开启推送。
`startPush`, `pushPcm`, and `stopPush` need to be called in pairs. `pushPcm` should not be too long. After pushing the entire audio, call `stopPush` to end the session. Use `startPush` again for the next audio.
**每段startPush到stopPush中间的音频数据最少要1秒(32000字节)否则无法触发口型驱动，可以自行使用空白帧填充。**
**The audio data between each startPush and stopPush segment should be at least 1 second (32000 bytes), otherwise the mouth shape driver cannot be triggered, and blank frames can be used to fill in.**
**调用示例**:
**Call Example**:
```kotlin
val thread = Thread {
@@ -300,112 +290,42 @@ thread.start()
---
#### WAV 播放驱动
### 6.5 Motion Control
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
#### Play Specific Motion Interval
```
void playAudio(String wavPath)
```
The model supports new motion intervals marked in `SpecialAction.json`
**该函数兼容旧的wav驱动数字人接口，在内部实际是调用了PCM推流方式实现驱动。**
**参数说明**:
| 参数 | 类型 | 描述 |
|---------|--------|-----------------------|
| wavPath | String | 16k采样率单通道16位深的wav本地文件 |
**调用示例**:
```kotlin
duix?.playAudio(wavPath)
```
音频播放状态及进度回调:
```kotlin
object : Callback {
fun onEvent(event: String, msg: String, info: Object) {
when (event) {
// ...
"play.start" -> {
// 开始播放音频
}
"play.end" -> {
// 完成播放音频
}
"play.error" -> {
// 音频播放异常
}
}
}
}
```
---
#### 终止当前播报
当数字人正在播报时调用该接口终止播报。
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
```
boolean stopAudio();
```
**调用示例如下**：
```kotlin
duix?.stopAudio()
```
---
### 6.5. 动作控制
#### 播放指定动作区间
模型中支持新的动作区间标注(SpecialAction.json)
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
> Function Definition: `ai.guiji.duix.sdk.client.DUIX`
```
/**
* 播放指定动作区间
* @param name 动作区间名称，在init成功回调时，可以在@{ModelInfo.getSilenceRegion()}中获取到可用的动作区间
* @param now 是否立即播放 true: 立即播放; false: 等待当前静默区间或动作区间播放完毕后播放
* Play specific motion interval
* @param name The motion interval name, which can be obtained from @{ModelInfo.getSilenceRegion()} after init callback
* @param now Whether to play immediately: true: play now; false: wait for current silent or motion interval to finish
*/
void startMotion(String name, boolean now)
```
**调用示例如下**：
**Call Example**:
```kotlin
duix?.startMotion("打招呼", true)
duix?.startMotion("Greeting", true)
```
#### 随机播放动作区间
#### Randomly Play Motion Interval
随机播放场景及旧的标注协议(config.json)
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
> Function Definition: `ai.guiji.duix.sdk.client.DUIX`
```
/**
* 随机播放一个动作区间
* @param now 是否立即播放 true: 立即播放; false: 等待当前静默区间或动作区间播放完毕后播放
* Randomly play a motion interval
* @param now Whether to play immediately: true: play now; false: wait for current silent or motion interval to finish
*/
void startRandomMotion(boolean now);
```
**调用示例如下**：
**Call Example**:
```kotlin
duix?.startRandomMotion(true)
@@ -413,9 +333,9 @@ duix?.startRandomMotion(true)
---
## 七. Proguard配置
## 7. Proguard Configuration
如果代码使用了混淆，请在proguard-rules.pro中配置：
If using obfuscation, add the following in `proguard-rules.pro`:
```proguard
-keep class ai.guiji.duix.DuixNcnn{*; }
@@ -423,65 +343,63 @@ duix?.startRandomMotion(true)
---
## 八、注意事项
## 8. Precautions
1. 驱动渲染初始化前需要确保基础配置文件及模型下载到指定位置。
2. 播放的PCM音频不宜过长，播放的PCM缓存在内存中，过长的音频流可能导致内存溢出。
3. 替换预览模型可以在MainActivity.kt文件中修改modelUrl的值，使用SDK中自带的文件下载解压管理以获得完整的模型文件。
4. 音频驱动的格式: 16k采样率单通道16位深度
5. 设备性能不足时可能导致音频特征提取的速度跟不上音频播放的速度，可以使用duix?.setReporter()函数添加一个监控观察帧渲染返回的信息。
6. 每段startPush到stopPush中间的音频数据最少要1秒(32000字节)否则无法触发口型驱动，可以自行使用空白帧填充。
1. Ensure that the base configuration file and model are downloaded to the specified location before driving rendering initialization.
2. PCM audio should not be too long, as PCM buffers are cached in memory; long audio streams may cause memory overflow.
3. To replace the preview model, modify the `modelUrl` value in `MainActivity.kt` and use the SDK's built-in file download and decompression management to obtain the complete model files.
4. Audio driving format: 16kHz sample rate, single channel, 16-bit depth.
5. Insufficient device performance may result in the audio feature extraction speed not matching the playback speed. You can use `duix?.setReporter()` to monitor frame rendering information.
---
## 九、常见问题与排查指南
## 9. FAQ and Troubleshooting Guide
| 问题现象 | 可能原因 | 解决方案 |
|---------------------|--------------------------|------------------------|
| init 回调失败 | 模型路径错误或未下载完成 | 使用 `checkModel` 检查模型状态 |
| 渲染黑屏 | EGL 配置或纹理视图设置错误 | 使用 SDK 提供示例中的设置方法 |
| PCM 无播报效果 | 格式不符或未调用 startPush | 确保音频格式正确并调用推送方法 |
| 模型下载过慢 | 网络不稳定或 CDN 受限 | 支持自建模型文件托管服务 |
| Issue | Possible Cause | Solution |
|---------------------------------|------------------------------|------------------------------|
| init callback failed | Model path error or model not downloaded | Use `checkModel` to check model status |
| Rendering black screen | EGL configuration or texture view error | Use SDK-provided example settings |
| No PCM playback effect | Incorrect format or `startPush` not called | Ensure audio format is correct and call push method |
| Model download slow | Unstable network or restricted CDN | Support self-hosted model file storage service |
---
## 十、版本记录
## 10. Version History
**<a>4.0.1</a>**
```text
1. 支持PCM音频流驱动数字人，提升音频播放响应速度。
2. 优化动作区间播放，可根据模型配置指定播放动作区间。
3. 自定义音频播放器，去除Exoplayer播放依赖
4. 提供简洁的模型下载同步管理工具
```
1. Supports PCM audio stream driving the digital human, improving audio playback response speed.
2. Optimized motion interval playback, allowing specific motion intervals based on model configuration.
3. Custom audio player, removed Exoplayer playback dependency.
4. Provided simplified model download synchronization management tools.
5. The audio data between each startPush and stopPush segment should be at least 1 second (32000 bytes), otherwise the mouth shape driver cannot be triggered, and blank frames can be used to fill in.
**<a>3.0.5</a>**
```text
1. 更新arm32位cpu的libonnxruntime.so版本以修复兼容问题。
2. 修改动作区间播放函数，可以使用随机播放和顺序播放，需要主动调用停止播放动作区间以回到静默区间。
1. Updated arm32 CPU libonnxruntime.so version to fix compatibility issues.
2. Modified motion interval playback function, supports random and sequential playback, requires manual call to stop playback to return to silent interval.
```
**<a>3.0.4</a>**
```text
1. 修复部分设备gl默认float低精度导致无法正常显示形象问题。
1. Fixed model display issue due to low float precision on some devices.
```
**<a>3.0.3</a>**
```text
1. 优化本地渲染。
1. Optimized local rendering.
```
## 十一、🔗 开源依赖
## 11. 🔗 Open-source Dependencies
| 模块 | 描述 |
|-------------------------------------------|-------------------|
| [onnx](https://github.com/onnx/onnx) | 通用AI模型标准格式 |
| [ncnn](https://github.com/Tencent/ncnn) | 高性能神经网络计算框架（腾讯） |
| Module | Description |
|------------------------------------------|--------------------------------|
| [onnx](https://github.com/onnx/onnx) | General AI model standard format |
| [ncnn](https://github.com/Tencent/ncnn) | High-performance neural network computing framework (Tencent) |
---
如需更多帮助，请联系技术支持团队。
For more help, please contact the technical support team.

407
duix-android/dh_aigc_android/README_en.md
View File

@@ -1,407 +0,0 @@
# Duix Mobile for Android SDK Documentation
[简体中文](./README.md) | English
## 1. Product Overview
`Duix Mobile for Android` is a lightweight, fully offline 2D digital human solution for Android, supporting real-time rendering of digital avatars driven by voice audio.
### 1.1 Application Scenarios
- **Low deployment cost**: Suitable for unattended scenarios such as large-screen terminals, government halls, and banks.
- **Minimal network dependency**: Runs entirely locally, no internet required, stable operation in subways and remote areas.
- **Diverse functionality**: Can serve as a guide, Q&A customer service, intelligent companion, and more.
### 1.2 Core Features
- Customizable digital avatar and local rendering
- Real-time voice-driven playback (supports WAV playback and PCM streaming)
- Motion playback control (specific or random actions)
- Automatic resource download management
---
## 2. Terminology
| Term | Meaning |
|--------------------|----------------------------------------------------------------------------|
| PCM | Pulse-Code Modulation, raw audio stream with 16kHz sample rate, 16-bit depth, Mono channel |
| WAV | An audio file format that supports PCM encoding, suitable for short voice playback |
| RenderSink | Rendering data reception interface, implemented by the SDK, can be used for custom rendering or default display |
| DUIX | Main control object of the digital human, integrates model loading, rendering, broadcasting, and motion control |
| GLES | OpenGL ES, a graphics interface for rendering images on Android |
| SpecialAction | A JSON file attached to the model that marks action intervals (e.g., greetings, waving) |
---
## 3. SDK Access
> ✅ The SDK is currently provided as a local module or AAR file. You can contact business or technical support to obtain the SDK package.
### 3.1 Module Reference (Recommended)
1. Obtain the complete source package, unzip it, and copy the `duix-sdk` directory to the project root directory.
2. In the project `settings.gradle`, add:
```gradle
include ':duix-sdk'
```
3. In the module's `build.gradle`, add the dependency:
```gradle
dependencies {
api project(":duix-sdk")
}
```
### 3.2 AAR Reference (Optional)
1. Place the compiled `duix-sdk-release.aar` module into the `libs/` directory.
2. Add the dependency:
```gradle
dependencies {
api fileTree(include: ['*.jar', '*.aar'], dir: 'libs')
}
```
---
## 4. Integration Requirements
| Item | Description |
|----------------|-----------------------------------------------------------------|
| System | Supports Android 10+ systems. |
| CPU Architecture | armeabi-v7a, arm64-v8a |
| Hardware Requirements | Device CPU with 8 or more cores (Snapdragon 8 Gen 2), 8GB or more memory, available storage space of 1GB or more |
| Network | None (Fully local operation) |
| Development IDE | Android Studio Giraffe 2022.3.1 Patch 2 |
| Memory Requirements | Minimum 800MB memory available for the digital human |
---
## 5. Usage Flow Overview
```mermaid
graph TD
A[Check Configuration and Models] --> B[Build DUIX Instance]
B --> C[Call init to Initialize]
C --> D[Display Avatar / Render]
D --> E[PCM or WAV Audio Driving]
E --> F[Playback Control & Motion Triggering]
F --> G[Resource Release]
```
---
## 6. Key Interfaces and Example Calls
### 6.1 Model Check and Download
Before using the rendering service, ensure that the basic configuration and model files are synchronized to local storage. The SDK provides a simple demonstration of the model download and decompression process using `VirtualModelUtil`. If model download is slow or fails, developers can choose to cache the model package to their own storage service.
> Function Definition: `ai.guiji.duix.sdk.client.VirtualModelUtil`
```
// Check if base configuration is downloaded
boolean checkBaseConfig(Context context)
// Check if the model is downloaded
boolean checkModel(Context context, String name)
// Base configuration download
void baseConfigDownload(Context context, String url, ModelDownloadCallback callback)
// Model download
void modelDownload(Context context, String modelUrl, ModelDownloadCallback callback)
```
`ModelDownloadCallback` includes progress, completion, failure callbacks, etc., as defined in the SDK.
```
interface ModelDownloadCallback {
// Download progress
void onDownloadProgress(String url, long current, long total);
// Unzip progress
void onUnzipProgress(String url, long current, long total);
// Download and unzip complete
void onDownloadComplete(String url, File dir);
// Download and unzip failed
void onDownloadFail(String url, int code, String msg);
}
```
**Call Example**:
```kotlin
if (!VirtualModelUtil.checkBaseConfig(mContext)){
VirtualModelUtil.baseConfigDownload(mContext, baseConfigUrl, callback)
}
```
```kotlin
if (!VirtualModelUtil.checkModel(mContext, modelUrl)){
VirtualModelUtil.modelDownload(mContext, modelUrl, callback)
}
```
---
### 6.2 Initialization and Rendering Start
In the `onCreate()` stage of the rendering page, build the DUIX object and call the init interface.
> Function Definition: `ai.guiji.duix.sdk.client.DUIX`
```
// Build DUIX object
public DUIX(Context context, String modelName, RenderSink sink, Callback callback)
// Initialize DUIX service
void init()
```
**DUIX Object Construction Explanation**:
| Parameter | Type | Description |
|---------------|-----------|----------------------------------------------------------------|
| context | Context | System context |
| modelName | String | Can pass the model download URL (if downloaded) or cached filename |
| render | RenderSink| Rendering data interface, SDK provides a default rendering component inheriting from this interface, or you can implement it yourself |
| callback | Callback | Various callback events handled by the SDK |
Where **Callback** is defined as: `ai.guiji.duix.sdk.client.Callback`
```
interface Callback {
void onEvent(String event, String msg, Object info);
}
```
**Call Example**:
```kotlin
duix = DUIX(mContext, modelUrl, mDUIXRender) { event, msg, info ->
when (event) {
ai.guiji.duix.sdk.client.Constant.CALLBACK_EVENT_INIT_READY -> {
initOK()
}
ai.guiji.duix.sdk.client.Constant.CALLBACK_EVENT_INIT_ERROR -> {
initError()
}
// ...
}
}
// Asynchronous callback result
duix?.init()
```
In the `init` callback, confirm the initialization result.
---
### 6.3 Digital Human Avatar Display
Use the SDK-provided `DUIXRenderer` and `DUIXTextureView` to quickly implement rendering with transparency support. Alternatively, you can implement the `RenderSink` interface to customize the rendering logic.
The **RenderSink** definition is as follows: `ai.guiji.duix.sdk.client.render.RenderSink`
```java
/**
* Rendering pipeline, returns rendering data through this interface
*/
public interface RenderSink {
// The frame's buffer data is arranged in BGR order
void onVideoFrame(ImageFrame imageFrame);
}
```
**Call Example**:
Use `DUIXRenderer` and `DUIXTextureView` to quickly implement rendering. These controls support transparency and can freely set the background and foreground.
```kotlin
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
// ...
mDUIXRender =
DUIXRenderer(
mContext,
binding.glTextureView
)
binding.glTextureView.setEGLContextClientVersion(GL_CONTEXT_VERSION)
binding.glTextureView.setEGLConfigChooser(8, 8, 8, 8, 16, 0) // Transparency
binding.glTextureView.isOpaque = false // Transparency
binding.glTextureView.setRenderer(mDUIXRender)
binding.glTextureView.renderMode =
GLSurfaceView.RENDERMODE_WHEN_DIRTY // Must be called after setting the renderer
duix = DUIX(mContext, modelUrl, mDUIXRender) { event, msg, _ ->
}
// ...
}
```
---
### 6.4 Broadcasting Control
#### Use Streaming PCM to Drive Digital Human Broadcasting
**PCM Format: 16kHz sample rate, single channel, 16-bit depth**
> Function Definition: `ai.guiji.duix.sdk.client.DUIX`
```
// Notify service to start pushing audio
void startPush()
// Push PCM data
void pushPcm(byte[] buffer)
// Finish a segment of audio push (Call this after the audio push is complete, not after playback finishes)
void stopPush()
```
`startPush`, `pushPcm`, and `stopPush` need to be called in pairs. `pushPcm` should not be too long. After pushing the entire audio, call `stopPush` to end the session. Use `startPush` again for the next audio.
**The audio data between each startPush and stopPush segment should be at least 1 second (32000 bytes), otherwise the mouth shape driver cannot be triggered, and blank frames can be used to fill in.**
**Call Example**:
```kotlin
val thread = Thread {
duix?.startPush()
val inputStream = assets.open("pcm/2.pcm")
val buffer = ByteArray(320)
var length = 0
while (inputStream.read(buffer).also { length = it } > 0){
val data = buffer.copyOfRange(0, length)
duix?.pushPcm(data)
}
duix?.stopPush()
inputStream.close()
}
thread.start()
```
---
### 6.5 Motion Control
#### Play Specific Motion Interval
The model supports new motion intervals marked in `SpecialAction.json`
> Function Definition: `ai.guiji.duix.sdk.client.DUIX`
```
/**
* Play specific motion interval
* @param name The motion interval name, which can be obtained from @{ModelInfo.getSilenceRegion()} after init callback
* @param now Whether to play immediately: true: play now; false: wait for current silent or motion interval to finish
*/
void startMotion(String name, boolean now)
```
**Call Example**:
```kotlin
duix?.startMotion("Greeting", true)
```
#### Randomly Play Motion Interval
> Function Definition: `ai.guiji.duix.sdk.client.DUIX`
```
/**
* Randomly play a motion interval
* @param now Whether to play immediately: true: play now; false: wait for current silent or motion interval to finish
*/
void startRandomMotion(boolean now);
```
**Call Example**:
```kotlin
duix?.startRandomMotion(true)
```
---
## 7. Proguard Configuration
If using obfuscation, add the following in `proguard-rules.pro`:
```proguard
-keep class ai.guiji.duix.DuixNcnn{*; }
```
---
## 8. Precautions
1. Ensure that the base configuration file and model are downloaded to the specified location before driving rendering initialization.
2. PCM audio should not be too long, as PCM buffers are cached in memory; long audio streams may cause memory overflow.
3. To replace the preview model, modify the `modelUrl` value in `MainActivity.kt` and use the SDK's built-in file download and decompression management to obtain the complete model files.
4. Audio driving format: 16kHz sample rate, single channel, 16-bit depth.
5. Insufficient device performance may result in the audio feature extraction speed not matching the playback speed. You can use `duix?.setReporter()` to monitor frame rendering information.
---
## 9. FAQ and Troubleshooting Guide
| Issue | Possible Cause | Solution |
|---------------------------------|------------------------------|------------------------------|
| init callback failed | Model path error or model not downloaded | Use `checkModel` to check model status |
| Rendering black screen | EGL configuration or texture view error | Use SDK-provided example settings |
| No PCM playback effect | Incorrect format or `startPush` not called | Ensure audio format is correct and call push method |
| Model download slow | Unstable network or restricted CDN | Support self-hosted model file storage service |
---
## 10. Version History
**<a>4.0.1</a>**
1. Supports PCM audio stream driving the digital human, improving audio playback response speed.
2. Optimized motion interval playback, allowing specific motion intervals based on model configuration.
3. Custom audio player, removed Exoplayer playback dependency.
4. Provided simplified model download synchronization management tools.
5. The audio data between each startPush and stopPush segment should be at least 1 second (32000 bytes), otherwise the mouth shape driver cannot be triggered, and blank frames can be used to fill in.
**<a>3.0.5</a>**
```text
1. Updated arm32 CPU libonnxruntime.so version to fix compatibility issues.
2. Modified motion interval playback function, supports random and sequential playback, requires manual call to stop playback to return to silent interval.
```
**<a>3.0.4</a>**
```text
1. Fixed model display issue due to low float precision on some devices.
```
**<a>3.0.3</a>**
```text
1. Optimized local rendering.
```
## 11. 🔗 Open-source Dependencies
| Module | Description |
|------------------------------------------|--------------------------------|
| [onnx](https://github.com/onnx/onnx) | General AI model standard format |
| [ncnn](https://github.com/Tencent/ncnn) | High-performance neural network computing framework (Tencent) |
---
For more help, please contact the technical support team.

485
duix-android/dh_aigc_android/README_zh.md Normal file
View File

@@ -0,0 +1,485 @@
# Duix Mobile for Android SDK 文档
中文 | [English](./README.md)
## 一、产品介绍
`Duix Mobile for Android` 是一套轻量级、纯离线的 Android 平台 2D 虚拟人解决方案，支持通过语音音频驱动数字人形象并进行实时渲染。
### 1.1 应用场景
- **部署成本低**：适用于大屏终端、政务展厅、银行等无人值守场景。
- **网络依赖小**：完全本地运行，无需联网，可在地铁、偏远地区稳定运行。
- **功能多样化**：可服务于导览讲解、问答客服、智能陪伴等多种业务形态。
### 1.2 核心功能
- 数字人形象定制与本地渲染
- 实时语音驱动播报（支持 WAV 播放和 PCM 推送）
- 动作播放控制（指定动作、随机动作）
- 资源自动下载管理
---
## 二、术语说明
| 术语 | 含义 |
|-------------------|------------------------------------------------------------------------|
| PCM | Pulse-Code Modulation，16kHz 采样率、16bit 位深、Mono 单通道的原始音频流 |
| WAV | 一种音频文件格式，支持 PCM 编码，适合短语音播放 |
| RenderSink | 渲染数据接收接口，由 SDK 提供实现，可用于自定义渲染或默认展示 |
| DUIX | 数字人主控对象，集成了模型加载、渲染、播报、动作等能力 |
| GLES | OpenGL ES，Android 渲染图像用到的图形接口 |
| SpecialAction | 模型附带的 JSON 文件，标注动作区间（例如打招呼、挥手等） |
---
## 三、SDK 获取方式
### 3.1 Module 引用（推荐）
1. 获取完整源码包，解压后将 `duix-sdk` 目录复制到项目根目录下。
2. 在项目 `settings.gradle` 中添加：
```gradle
include ':duix-sdk'
```
3. 在模块 `build.gradle` 中添加依赖：
```gradle
dependencies {
api project(":duix-sdk")
}
```
### 3.2 AAR 引用（可选）
1. 将duix-sdk模块编译的 `duix-sdk-release.aar` 放入 `libs/` 目录。
2. 添加依赖：
```gradle
dependencies {
api fileTree(include: ['*.jar', '*.aar'], dir: 'libs')
}
```
---
## 四、集成要求
| 项目 | 描述 |
|--------|----------------------------------------------------|
| 系统 | 支持 Android 10+ 系统。 |
| CPU架构 | armeabi-v7a, arm64-v8a |
| 硬件要求 | 要求设备 CPU8 核及以上(骁龙8 Gen2),内存 8G 及以上。可用存储空间 1GB 及以上。 |
| 网络 | 无（完全本地运行） |
| 开发 IDE | Android Studio Giraffe 2022.3.1 Patch 2 |
| 内存要求 | 可用于数字人的内存 >= 800MB |
**编译项目的Gradle使用的JDK版本为17,需要在File->Setting->Build,Execution,Deployment->Grade Projects->Gradle JDK: ${选择一个17版本的JDK}**
---
## 五、使用流程概览
```mermaid
graph TD
A[检查配置与模型] --> B[构建 DUIX 实例]
B --> C[调用 init 初始化]
C --> D[展示形象 / 渲染]
D --> E[PCM 或 WAV 音频驱动]
E --> F[播放控制与动作触发]
F --> G[资源释放]
```
---
## 六、关键接口与调用示例
### 6.1. 模型检查及下载
使用渲染服务前需要将基础配置及模型文件同步到本地存储中,SDK中提供了VirtualModelUtil简单演示了模型下载解压流程。
若模型下载过慢或无法下载，开发者可以选择将模型包缓存到自己的存储服务。
> 函数定义: `ai.guiji.duix.sdk.client.VirtualModelUtil`
```
// 检查基础配置是否已下载
boolean checkBaseConfig(Context context)
// 检查模型是否已下载
boolean checkModel(Context context, String name)
// 基础配置下载
void baseConfigDownload(Context context, String url, ModelDownloadCallback callback)
// 模型下载
void modelDownload(Context context, String modelUrl, ModelDownloadCallback callback)
```
`ModelDownloadCallback` 包含进度、完成、失败等回调，详见 SDK 定义。
```
interface ModelDownloadCallback {
// 下载进度
void onDownloadProgress(String url, long current, long total);
// 解压进度
void onUnzipProgress(String url, long current, long total);
// 下载解压完成
void onDownloadComplete(String url, File dir);
// 下载解压失败
void onDownloadFail(String url, int code, String msg);
}
```
**调用示例**:
```kotlin
if (!VirtualModelUtil.checkBaseConfig(mContext)){
VirtualModelUtil.baseConfigDownload(mContext, baseConfigUrl, callback)
}
```
```kotlin
if (!VirtualModelUtil.checkModel(mContext, modelUrl)){
VirtualModelUtil.modelDownload(mContext, modelUrl, callback)
}
```
---
### 6.2. 初始化与渲染启动
在渲染页onCreate()阶段构建DUIX对象并调用init接口
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
```
// 构建DUIX对象
public DUIX(Context context, String modelName, RenderSink sink, Callback callback)
// 初始化DUIX服务
void init()
```
**DUIX对象构建说明**:
| 参数 | 类型 | 描述 |
|------------|------------|-------------------------------------|
| context | Context | 系统上下文 |
| modelName | String | 可以传递模型下载的URL(已下载完成)或缓存的文件名 |
| render | RenderSink | 渲染数据接口，sdk提供了默认的渲染组件继承自该接口，也可以自己实现 |
| callback | Callback | SDK处理的各种回调事件 |
其中**Callback**的定义: `ai.guiji.duix.sdk.client.Callback`
```
interface Callback {
void onEvent(String event, String msg, Object info);
}
```
**调用示例**:
```kotlin
duix = DUIX(mContext, modelUrl, mDUIXRender) { event, msg, info ->
when (event) {
ai.guiji.duix.sdk.client.Constant.CALLBACK_EVENT_INIT_READY -> {
initOK()
}
ai.guiji.duix.sdk.client.Constant.CALLBACK_EVENT_INIT_ERROR -> {
initError()
}
// ...
}
}
// 异步回调结果
duix?.init()
```
在init回调中确认初始化结果
---
### 6.3. 数字人形象展示
使用 SDK 提供的 `DUIXRenderer` 和 `DUIXTextureView` 可快速实现支持透明通道的渲染。也可以自己实现RenderSink接口自定义渲染逻辑。
其中**RenderSink**的定义如下: `ai.guiji.duix.sdk.client.render.RenderSink`
```java
/**
* 渲染管道，通过该接口返回渲染数据
*/
public interface RenderSink {
// frame中的buffer数据以bgr顺序排列
void onVideoFrame(ImageFrame imageFrame);
}
```
**调用示例**:
使用DUIXRenderer及DUIXTextureView控件简单实现渲染展示,该控件支持透明通道可以自由设置背景及前景
```kotlin
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
// ...
mDUIXRender =
DUIXRenderer(
mContext,
binding.glTextureView
)
binding.glTextureView.setEGLContextClientVersion(GL_CONTEXT_VERSION)
binding.glTextureView.setEGLConfigChooser(8, 8, 8, 8, 16, 0) // 透明
binding.glTextureView.isOpaque = false // 透明
binding.glTextureView.setRenderer(mDUIXRender)
binding.glTextureView.renderMode =
GLSurfaceView.RENDERMODE_WHEN_DIRTY // 一定要在设置完Render之后再调用
duix = DUIX(mContext, modelUrl, mDUIXRender) { event, msg, _ ->
}
// ...
}
```
---
### 6.4 播报控制
#### 使用流式推送PCM驱动数字人播报
**PCM格式:16k采样率单通道16位深**
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
```
// 通知服务开始推送音频
void startPush()
// 推送PCM数据
void pushPcm(byte[] buffer)
// 完成一段音频推送(音频推送完就调要该函数，而不是等播放完成再调用。)
void stopPush()
```
startPush、pushPcm、stopPush需要成对调用，pushPcm不宜过长。可以在一整段音频推送完后调用stopPush结束当前会话，下一段音频再使用startPush重新开启推送。
**每段startPush到stopPush中间的音频数据最少要1秒(32000字节)否则无法触发口型驱动，可以自行使用空白帧填充。**
**调用示例**:
```kotlin
val thread = Thread {
duix?.startPush()
val inputStream = assets.open("pcm/2.pcm")
val buffer = ByteArray(320)
var length = 0
while (inputStream.read(buffer).also { length = it } > 0){
val data = buffer.copyOfRange(0, length)
duix?.pushPcm(data)
}
duix?.stopPush()
inputStream.close()
}
thread.start()
```
---
#### WAV 播放驱动
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
```
void playAudio(String wavPath)
```
**该函数兼容旧的wav驱动数字人接口，在内部实际是调用了PCM推流方式实现驱动。**
**参数说明**:
| 参数 | 类型 | 描述 |
|---------|--------|-----------------------|
| wavPath | String | 16k采样率单通道16位深的wav本地文件 |
**调用示例**:
```kotlin
duix?.playAudio(wavPath)
```
音频播放状态及进度回调:
```kotlin
object : Callback {
fun onEvent(event: String, msg: String, info: Object) {
when (event) {
// ...
"play.start" -> {
// 开始播放音频
}
"play.end" -> {
// 完成播放音频
}
"play.error" -> {
// 音频播放异常
}
}
}
}
```
---
#### 终止当前播报
当数字人正在播报时调用该接口终止播报。
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
```
boolean stopAudio();
```
**调用示例如下**：
```kotlin
duix?.stopAudio()
```
---
### 6.5. 动作控制
#### 播放指定动作区间
模型中支持新的动作区间标注(SpecialAction.json)
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
```
/**
* 播放指定动作区间
* @param name 动作区间名称，在init成功回调时，可以在@{ModelInfo.getSilenceRegion()}中获取到可用的动作区间
* @param now 是否立即播放 true: 立即播放; false: 等待当前静默区间或动作区间播放完毕后播放
*/
void startMotion(String name, boolean now)
```
**调用示例如下**：
```kotlin
duix?.startMotion("打招呼", true)
```
#### 随机播放动作区间
随机播放场景及旧的标注协议(config.json)
> 函数定义: `ai.guiji.duix.sdk.client.DUIX`
```
/**
* 随机播放一个动作区间
* @param now 是否立即播放 true: 立即播放; false: 等待当前静默区间或动作区间播放完毕后播放
*/
void startRandomMotion(boolean now);
```
**调用示例如下**：
```kotlin
duix?.startRandomMotion(true)
```
---
## 七. Proguard配置
如果代码使用了混淆，请在proguard-rules.pro中配置：
```proguard
-keep class ai.guiji.duix.DuixNcnn{*; }
```
---
## 八、注意事项
1. 驱动渲染初始化前需要确保基础配置文件及模型下载到指定位置。
2. 播放的PCM音频不宜过长，播放的PCM缓存在内存中，过长的音频流可能导致内存溢出。
3. 替换预览模型可以在MainActivity.kt文件中修改modelUrl的值，使用SDK中自带的文件下载解压管理以获得完整的模型文件。
4. 音频驱动的格式: 16k采样率单通道16位深度
5. 设备性能不足时可能导致音频特征提取的速度跟不上音频播放的速度，可以使用duix?.setReporter()函数添加一个监控观察帧渲染返回的信息。
6. 每段startPush到stopPush中间的音频数据最少要1秒(32000字节)否则无法触发口型驱动，可以自行使用空白帧填充。
---
## 九、常见问题与排查指南
| 问题现象 | 可能原因 | 解决方案 |
|---------------------|--------------------------|------------------------|
| init 回调失败 | 模型路径错误或未下载完成 | 使用 `checkModel` 检查模型状态 |
| 渲染黑屏 | EGL 配置或纹理视图设置错误 | 使用 SDK 提供示例中的设置方法 |
| PCM 无播报效果 | 格式不符或未调用 startPush | 确保音频格式正确并调用推送方法 |
| 模型下载过慢 | 网络不稳定或 CDN 受限 | 支持自建模型文件托管服务 |
---
## 十、版本记录
**<a>4.0.1</a>**
```text
1. 支持PCM音频流驱动数字人，提升音频播放响应速度。
2. 优化动作区间播放，可根据模型配置指定播放动作区间。
3. 自定义音频播放器，去除Exoplayer播放依赖
4. 提供简洁的模型下载同步管理工具
```
**<a>3.0.5</a>**
```text
1. 更新arm32位cpu的libonnxruntime.so版本以修复兼容问题。
2. 修改动作区间播放函数，可以使用随机播放和顺序播放，需要主动调用停止播放动作区间以回到静默区间。
```
**<a>3.0.4</a>**
```text
1. 修复部分设备gl默认float低精度导致无法正常显示形象问题。
```
**<a>3.0.3</a>**
```text
1. 优化本地渲染。
```
## 十一、🔗 开源依赖
| 模块 | 描述 |
|-------------------------------------------|-------------------|
| [onnx](https://github.com/onnx/onnx) | 通用AI模型标准格式 |
| [ncnn](https://github.com/Tencent/ncnn) | 高性能神经网络计算框架（腾讯） |
---
如需更多帮助，请联系技术支持团队。

BIN
duix-android/duix_mobile_test_release_2025-08-19-17-20_4.1.1.apk
View File

Binary file not shown.

12
duix-ios/GJLocalDigitalDemo/GJLocalDigitalDemo.xcodeproj/project.pbxproj
View File

@@ -651,7 +651,7 @@
ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor;
CODE_SIGN_ENTITLEMENTS = GJLocalDigitalDemo/GJLocalDigitalDemo.entitlements;
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
CURRENT_PROJECT_VERSION = 1.2.3;
DEVELOPMENT_TEAM = RHWTKM535Q;
ENABLE_USER_SCRIPT_SANDBOXING = NO;
FRAMEWORK_SEARCH_PATHS = (
@@ -661,7 +661,7 @@
);
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = GJLocalDigitalDemo/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = SDKDemo1.2.2;
INFOPLIST_KEY_CFBundleDisplayName = SDKDemo1.2.3;
INFOPLIST_KEY_NSMicrophoneUsageDescription = "需要录音权限来语音识别";
INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents = YES;
INFOPLIST_KEY_UILaunchStoryboardName = LaunchScreen;
@@ -676,7 +676,7 @@
"$(inherited)",
"$(PROJECT_DIR)/GJLocalDigitalSDK/GJLocalDigitalSDK/libjpeg-turbo/libs",
);
MARKETING_VERSION = 1.2.2;
MARKETING_VERSION = 1.2.3;
OTHER_LDFLAGS = "$(inherited)";
PRODUCT_BUNDLE_IDENTIFIER = com.wanruyi.sdkdemo;
PRODUCT_NAME = "$(TARGET_NAME)";
@@ -694,7 +694,7 @@
ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor;
CODE_SIGN_ENTITLEMENTS = GJLocalDigitalDemo/GJLocalDigitalDemo.entitlements;
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1;
CURRENT_PROJECT_VERSION = 1.2.3;
DEVELOPMENT_TEAM = RHWTKM535Q;
ENABLE_USER_SCRIPT_SANDBOXING = NO;
FRAMEWORK_SEARCH_PATHS = (
@@ -704,7 +704,7 @@
);
GENERATE_INFOPLIST_FILE = YES;
INFOPLIST_FILE = GJLocalDigitalDemo/Info.plist;
INFOPLIST_KEY_CFBundleDisplayName = SDKDemo1.2.2;
INFOPLIST_KEY_CFBundleDisplayName = SDKDemo1.2.3;
INFOPLIST_KEY_NSMicrophoneUsageDescription = "需要录音权限来语音识别";
INFOPLIST_KEY_UIApplicationSupportsIndirectInputEvents = YES;
INFOPLIST_KEY_UILaunchStoryboardName = LaunchScreen;
@@ -719,7 +719,7 @@
"$(inherited)",
"$(PROJECT_DIR)/GJLocalDigitalSDK/GJLocalDigitalSDK/libjpeg-turbo/libs",
);
MARKETING_VERSION = 1.2.2;
MARKETING_VERSION = 1.2.3;
OTHER_LDFLAGS = "$(inherited)";
PRODUCT_BUNDLE_IDENTIFIER = com.wanruyi.sdkdemo;
PRODUCT_NAME = "$(TARGET_NAME)";

4
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK.md
View File

@@ -338,6 +338,10 @@ if (result == 1) {
## 十、版本更新记录
### v1.2.3
- 新增 支持 128模型
### v1.2.0
- 新增 PCM 推流支持

8
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK.xcodeproj/project.pbxproj
View File

@@ -812,7 +812,7 @@
buildSettings = {
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1.2.2;
CURRENT_PROJECT_VERSION = 1.2.3;
DEFINES_MODULE = YES;
DEVELOPMENT_TEAM = RHWTKM535Q;
DYLIB_COMPATIBILITY_VERSION = 1;
@@ -840,7 +840,7 @@
"$(PROJECT_DIR)/GJLocalDigitalSDK/libjpeg-turbo/libs",
);
MACH_O_TYPE = mh_dylib;
MARKETING_VERSION = 1.2.2;
MARKETING_VERSION = 1.2.3;
MODULE_VERIFIER_SUPPORTED_LANGUAGES = "objective-c objective-c++";
MODULE_VERIFIER_SUPPORTED_LANGUAGE_STANDARDS = "gnu17 gnu++20";
ONLY_ACTIVE_ARCH = YES;
@@ -861,7 +861,7 @@
buildSettings = {
CLANG_CXX_LANGUAGE_STANDARD = "gnu++20";
CODE_SIGN_STYLE = Automatic;
CURRENT_PROJECT_VERSION = 1.2.2;
CURRENT_PROJECT_VERSION = 1.2.3;
DEFINES_MODULE = YES;
DEVELOPMENT_TEAM = RHWTKM535Q;
DYLIB_COMPATIBILITY_VERSION = 1;
@@ -889,7 +889,7 @@
"$(PROJECT_DIR)/GJLocalDigitalSDK/libjpeg-turbo/libs",
);
MACH_O_TYPE = mh_dylib;
MARKETING_VERSION = 1.2.2;
MARKETING_VERSION = 1.2.3;
MODULE_VERIFIER_SUPPORTED_LANGUAGES = "objective-c objective-c++";
MODULE_VERIFIER_SUPPORTED_LANGUAGE_STANDARDS = "gnu17 gnu++20";
ONLY_ACTIVE_ARCH = NO;

117
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/DIMetalView/DIMetalView.mm
View File

@@ -146,43 +146,86 @@ typedef NS_ENUM(NSInteger, JPMetalViewContentMode) {
- (void)setupVertex
{
float heightScaling = 1.0;
float widthScaling = 1.0;
CGSize currentViewSize=CGSizeMake(self.mtkView.frame.size.width, self.mtkView.frame.size.height);
CGRect insetRect = AVMakeRectWithAspectRatioInsideRect(CGSizeMake([DigitalHumanDriven manager].configModel.width, [DigitalHumanDriven manager].configModel.height), CGRectMake(0, 0, currentViewSize.width, currentViewSize.height));
switch (_fillMode) {
case JPMetalViewContentModeStretch: {
widthScaling = 1.0;
heightScaling = 1.0;
break;
}
case JPMetalViewContentModeFit: {
widthScaling = insetRect.size.width / currentViewSize.width;
heightScaling = insetRect.size.height / currentViewSize.height;
break;
}
case JPMetalViewContentModeFill: {
widthScaling = currentViewSize.height / insetRect.size.height;
heightScaling = currentViewSize.width / insetRect.size.width;
break;
}
}
static const LYVertex quadVertices[] =
{ // 顶点坐标，分别是x、y、z、w； 纹理坐标，x、y；
{ { widthScaling, -heightScaling, 0.0, 1.0 }, { 1.f, 1.f } },
{ { -widthScaling, -heightScaling, 0.0, 1.0 }, { 0.f, 1.f } },
{ { -widthScaling, heightScaling, 0.0, 1.0 }, { 0.f, 0.f } },
{ { widthScaling, -heightScaling, 0.0, 1.0 }, { 1.f, 1.f } },
{ { -widthScaling, heightScaling, 0.0, 1.0 }, { 0.f, 0.f } },
{ { widthScaling, heightScaling, 0.0, 1.0 }, { 1.f, 0.f } },
};
self.vertices = [self.mtkView.device newBufferWithBytes:quadVertices
length:sizeof(quadVertices)
options:MTLResourceStorageModeShared]; // 创建顶点缓存
self.numVertices = sizeof(quadVertices) / sizeof(LYVertex); // 顶点个数
// 获取视图和纹理的尺寸
CGSize viewSize =CGSizeMake(self.mtkView.frame.size.width, self.mtkView.frame.size.height);
CGSize textureSize = CGSizeMake([DigitalHumanDriven manager].configModel.width, [DigitalHumanDriven manager].configModel.height);
// 计算宽高比
float textureAspect = textureSize.width / textureSize.height;
float viewAspect = viewSize.width / viewSize.height;
// 计算缩放因子和纹理坐标
float widthScaling = 1.0;
float heightScaling = 1.0;
float texLeft = 0.0;
float texRight = 1.0;
float texTop = 0.0;
float texBottom = 1.0;
switch (_fillMode) {
case JPMetalViewContentModeStretch: {
// 直接拉伸，不保持宽高比
widthScaling = 1.0;
heightScaling = 1.0;
// 纹理坐标保持不变 (0,0) 到 (1,1)
break;
}
case JPMetalViewContentModeFit: {
// 保持宽高比，适应视图（可能留有黑边）
if (textureAspect > viewAspect) {
// 纹理比视图宽，以宽度为基准缩放
widthScaling = 1.0;
heightScaling = (viewSize.width / textureAspect) / viewSize.height;
} else {
// 纹理比视图高，以高度为基准缩放
widthScaling = (viewSize.height * textureAspect) / viewSize.width;
heightScaling = 1.0;
}
// 纹理坐标保持不变 (0,0) 到 (1,1)
break;
}
case JPMetalViewContentModeFill: {
// 保持宽高比，填充视图（可能裁剪部分内容）
if (textureAspect > viewAspect) {
// 纹理比视图宽，需要裁剪左右
widthScaling = 1.0;
heightScaling = 1.0;
// 调整纹理坐标以裁剪左右
float texWidth = viewAspect / textureAspect;
texLeft = (1.0 - texWidth) / 2.0;
texRight = 1.0 - texLeft;
} else {
// 纹理比视图高，需要裁剪上下
widthScaling = 1.0;
heightScaling = 1.0;
// 调整纹理坐标以裁剪上下
float texHeight = textureAspect / viewAspect;
texTop = (1.0 - texHeight) / 2.0;
texBottom = 1.0 - texTop;
}
break;
}
}
// 创建顶点数据
LYVertex quadVertices[] = {
// 顶点坐标(x, y, z, w) 纹理坐标(x, y)
{ { widthScaling, -heightScaling, 0.0, 1.0 }, { texRight, texBottom } },
{ { -widthScaling, -heightScaling, 0.0, 1.0 }, { texLeft, texBottom } },
{ { -widthScaling, heightScaling, 0.0, 1.0 }, { texLeft, texTop } },
{ { widthScaling, -heightScaling, 0.0, 1.0 }, { texRight, texBottom } },
{ { -widthScaling, heightScaling, 0.0, 1.0 }, { texLeft, texTop } },
{ { widthScaling, heightScaling, 0.0, 1.0 }, { texRight, texTop } },
};
// 创建顶点缓冲区
self.vertices = [self.mtkView.device newBufferWithBytes:quadVertices
length:sizeof(quadVertices)
options:MTLResourceStorageModeShared];
self.numVertices = sizeof(quadVertices) / sizeof(LYVertex);
}
- (void)setupTexture {

2
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/DigitalHumanDriven/DigitalConfigModel.h
View File

@@ -27,6 +27,8 @@
//多动作
@property(nonatomic,strong) NSMutableArray * actions;
//128和168口型 默认168
@property(nonatomic,assign)NSInteger modelkind;
@end
@interface DigitalRangeModel : NSObject

1
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/DigitalHumanDriven/DigitalConfigModel.m
View File

@@ -15,6 +15,7 @@
{
self.width=540;
self.height=960;
self.modelkind=168;
self.res_fmt=@"";
self.ranges=[[NSMutableArray alloc] init];
self.reverses=[[NSMutableArray alloc] init];

2
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/DigitalHumanDriven/DigitalHumanDriven.mm
View File

@@ -138,7 +138,7 @@ static DigitalHumanDriven *manager = nil;
const char *cBinPath2 = [binPath2 UTF8String];
char *binPathStr2 = (char *)cBinPath2;
rst = dhduix_initMunet(gjduix_s,paramPathStr,binPathStr,binPathStr2);
rst = dhduix_initMunetex(gjduix_s,paramPathStr,binPathStr,binPathStr2,(int)self.configModel.modelkind);
return rst;
}

BIN
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/GJFrameWork/GJLDecry.framework/GJLDecry
View File

Binary file not shown.

BIN
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/GJFrameWork/GJLDecry.framework/Info.plist
View File

Binary file not shown.

5
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/GJLDigitalManager.mm
View File

@@ -330,6 +330,11 @@ static GJLDigitalManager * manager = nil;
[DigitalHumanDriven manager].configModel.width=width>0?width:540;
[DigitalHumanDriven manager].configModel.height=height>0?height:960;
[DigitalHumanDriven manager].need_png=[[config_dic valueForKey:@"need_png"]?:@"" integerValue];
if([config_dic.allKeys containsObject:@"modelkind"])
{
[DigitalHumanDriven manager].configModel.modelkind=[[config_dic valueForKey:@"modelkind"]?:@"" integerValue];
}
//NSDictionary *mydic= @{@"ranges":@[@{ @"min": @"1",@"max": @"64", @"type": @"0"}, @{@"min": @"65",@"max": @"125", @"type": @"1"}]};
[[DigitalHumanDriven manager].configModel.ranges removeAllObjects];

126
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/duixsdka/dhunet/malpha.cpp
View File

@@ -1,6 +1,6 @@
#include "malpha.h"
MWorkMat::MWorkMat(JMat* pic,JMat* msk,const int* boxs){
MWorkMat::MWorkMat(JMat* pic,JMat* msk,const int* boxs,int kind){
m_boxx = boxs[0];
m_boxy=boxs[1];
m_boxwidth=boxs[2]-m_boxx;
@@ -9,73 +9,93 @@ MWorkMat::MWorkMat(JMat* pic,JMat* msk,const int* boxs){
m_pic = pic;
m_msk = msk;
pic_real160 = new JMat(160,160,3,0,1);
pic_mask160 = new JMat(160,160,3,0,1);
//pic_crop160 = new JMat(160,160,3,0,1);
msk_real160 = new JMat(160,160,1,0,1);
if(kind==168){
//msk_mask160 = new JMat(160,160,3,0,1);
srcw = 168;
edge = 4;
adjw = 160;
mskx = 5;
msky = 5;
mskw = 150;
mskh = 145;
}else if(kind==128){
srcw = 134;
edge = 3;
adjw = 128;
mskx = 4;
msky = 4;
mskw = 120;
mskh = 120;
}
pic_realadjw = new JMat(adjw,adjw,3,0,1);
pic_maskadjw = new JMat(adjw,adjw,3,0,1);
//pic_cropadjw = new JMat(adjw,adjw,3,0,1);
msk_realadjw = new JMat(adjw,adjw,1,0,1);
}
MWorkMat::~MWorkMat(){
matpic_org168.release();
matpic_orgsrcw.release();
matpic_roirst.release();
delete pic_real160;
delete pic_mask160;
delete msk_real160;
if(pic_clone160) delete pic_clone160;
delete pic_realadjw;
delete pic_maskadjw;
delete msk_realadjw;
if(pic_cloneadjw) delete pic_cloneadjw;
}
int MWorkMat::munet(JMat** ppic,JMat** pmsk){
*ppic = pic_real160;
*pmsk = pic_mask160;
*ppic = pic_realadjw;
*pmsk = pic_maskadjw;
return 0;
}
int MWorkMat::premunet(){
matpic_roisrc = cv::Mat(m_pic->cvmat(),cv::Rect(m_boxx,m_boxy,m_boxwidth,m_boxheight));
cv::resize(matpic_roisrc , matpic_org168, cv::Size(168, 168), cv::INTER_AREA);
//vtacc
matpic_roi160 = cv::Mat(matpic_org168,cv::Rect(4,4,160,160));
cv::Mat cvmask = pic_mask160->cvmat();
cv::Mat cvreal = pic_real160->cvmat();
matpic_roi160.copyTo(cvmask);
matpic_roi160.copyTo(cvreal);
//cv::rectangle(cvmask,cv::Rect(5,5,150,150),cv::Scalar(0,0,0),-1);//,cv::LineTypes::FILLED);
cv::rectangle(cvmask,cv::Rect(5,5,150,145),cv::Scalar(0,0,0),-1);//,cv::LineTypes::FILLED);
//cv::rectangle(cvmask,cv::Rect(4,4,152,152),cv::Scalar(0,0,0),-1);//,cv::LineTypes::FILLED);
//cv::imwrite("cvmask.bmp",cvmask);
//cv::waitKey(0);
pic_clone160 = pic_real160->refclone(0);
cv::resize(matpic_roisrc , matpic_orgsrcw, cv::Size(srcw, srcw), cv::INTER_AREA);
matpic_roiadjw = cv::Mat(matpic_orgsrcw,cv::Rect(edge,edge,adjw,adjw));
cv::Mat cvmask = pic_maskadjw->cvmat();
cv::Mat cvreal = pic_realadjw->cvmat();
//printf("===matpic %d %d\n",matpic_roiadjw.cols,matpic_roiadjw.rows);
//printf("===cvreal %d %d\n",cvreal.cols,cvreal.rows);
//getchar();
matpic_roiadjw.copyTo(cvreal);
matpic_roiadjw.copyTo(cvmask);
pic_cloneadjw = pic_realadjw->refclone(0);
cv::rectangle(cvmask,cv::Rect(mskx,msky,mskw,mskh),cv::Scalar(0,0,0),-1);//,cv::LineTypes::FILLED);
return 0;
}
int MWorkMat::finmunet(JMat* fgpic){
cv::Mat cvreal = pic_real160->cvmat();
cv::Mat cvreal = pic_realadjw->cvmat();
//for(int k=0;k<16;k++){
//cv::line(cvreal,cv::Point(0,k*10),cv::Point(160,k*10),cv::Scalar(0,255,0));
//cv::line(cvreal,cv::Point(0,k*10),cv::Point(adjw,k*10),cv::Scalar(0,255,0));
//}
//for(int k=0;k<16;k++){
//cv::line(cvreal,cv::Point(k*10,0),cv::Point(k*10,160),cv::Scalar(0,255,0));
//cv::line(cvreal,cv::Point(k*10,0),cv::Point(k*10,adjw),cv::Scalar(0,255,0));
//}
cvreal.copyTo(matpic_roi160);
//cv::imwrite("accpre.bmp",matpic_org168);
if(m_msk) vtacc((uint8_t*)matpic_org168.data,168*168);
//cv::imwrite("accend.bmp",matpic_org168);
if(fgpic&&(fgpic->width()==168)){
cvreal.copyTo(matpic_roiadjw);
//cv::imwrite("accpre.bmp",matpic_orgsrcw);
if(m_msk) vtacc((uint8_t*)matpic_orgsrcw.data,srcw*srcw);
//cv::imwrite("accend.bmp",matpic_orgsrcw);
if(fgpic&&(fgpic->width()==srcw)){
std::vector<cv::Mat> list;
cv::split(matpic_org168,list);
cv::split(matpic_orgsrcw,list);
matmsk_roisrc = cv::Mat(m_msk->cvmat(),cv::Rect(m_boxx,m_boxy,m_boxwidth,m_boxheight));
cv::resize(matmsk_roisrc , matmsk_org168, cv::Size(168, 168), cv::INTER_AREA);
cv::Mat rrr(168,168,CV_8UC1);
cv::cvtColor(matmsk_org168,rrr,cv::COLOR_RGB2GRAY);
cv::resize(matmsk_roisrc , matmsk_orgsrcw, cv::Size(srcw, srcw), cv::INTER_AREA);
cv::Mat rrr(srcw,srcw,CV_8UC1);
cv::cvtColor(matmsk_orgsrcw,rrr,cv::COLOR_RGB2GRAY);
list.push_back(rrr);
cv::merge(list,fgpic->cvmat());
}else{
cv::resize(matpic_org168, matpic_roirst, cv::Size(m_boxwidth, m_boxheight), cv::INTER_AREA);
cv::resize(matpic_orgsrcw, matpic_roirst, cv::Size(m_boxwidth, m_boxheight), cv::INTER_AREA);
if(fgpic){
matpic_roisrc = cv::Mat(fgpic->cvmat(),cv::Rect(m_boxx,m_boxy,m_boxwidth,m_boxheight));
matpic_roirst.copyTo(matpic_roisrc);
@@ -87,36 +107,28 @@ int MWorkMat::finmunet(JMat* fgpic){
}
int MWorkMat::alpha(JMat** preal,JMat** pimg,JMat** pmsk){
*preal = pic_clone160;
*pimg = pic_real160;
*pmsk = msk_real160;
*preal = pic_cloneadjw;
*pimg = pic_realadjw;
*pmsk = msk_realadjw;
return 0;
}
int MWorkMat::prealpha(){
printf("x %d y %d w %d h %d \n",m_boxx,m_boxy,m_boxwidth,m_boxheight);
//m_msk->show("cba");
//cv::waitKey(0);
matmsk_roisrc = cv::Mat(m_msk->cvmat(),cv::Rect(m_boxx,m_boxy,m_boxwidth,m_boxheight));
cv::resize(matmsk_roisrc , matmsk_org168, cv::Size(168, 168), cv::INTER_AREA);
cv::resize(matmsk_roisrc , matmsk_orgsrcw, cv::Size(srcw, srcw), cv::INTER_AREA);
matmsk_roi160 = cv::Mat(matmsk_org168,cv::Rect(4,4,160,160));
cv::Mat cvmask = msk_real160->cvmat();
cv::cvtColor(matmsk_roi160,cvmask,cv::COLOR_RGB2GRAY);
//BlendGramAlphaRev(pic_clone160->udata(),msk_real160->udata(),pic_crop160->udata(),160,160);
//pic_crop160->show("aaa");
//cv::waitKey(0);
//pic_crop160
//
matmsk_roiadjw = cv::Mat(matmsk_orgsrcw,cv::Rect(edge,edge,adjw,adjw));
cv::Mat cvmask = msk_realadjw->cvmat();
cv::cvtColor(matmsk_roiadjw,cvmask,cv::COLOR_RGB2GRAY);
return 0;
}
int MWorkMat::finalpha(){
cv::Mat cvmask = msk_real160->cvmat();
cv::cvtColor(cvmask,matmsk_roi160,cv::COLOR_GRAY2RGB);
cv::Mat cvmask = msk_realadjw->cvmat();
cv::cvtColor(cvmask,matmsk_roiadjw,cv::COLOR_GRAY2RGB);
//
cv::resize(matmsk_org168, matmsk_roirst, cv::Size(m_boxwidth, m_boxheight), cv::INTER_AREA);
cv::resize(matmsk_orgsrcw, matmsk_roirst, cv::Size(m_boxwidth, m_boxheight), cv::INTER_AREA);
matmsk_roirst.copyTo(matmsk_roisrc);
return 0;
}

70
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/duixsdka/dhunet/malpha.h
View File

@@ -7,44 +7,50 @@
#include <stdio.h>
class MWorkMat{
private:
int m_boxx;
int m_boxy;
int m_boxwidth;
int m_boxheight;
JMat* m_pic;
JMat* m_msk;
private:
int srcw = 168;
int edge = 4;
int adjw = 160;
JMat* pic_real160;//blendimg
JMat* pic_mask160;
int mskx = 5;
int msky = 5;
int mskw = 150;
int mskh = 145;
int m_boxx;
int m_boxy;
int m_boxwidth;
int m_boxheight;
JMat* m_pic;
JMat* m_msk;
cv::Mat matpic_roisrc;//box area
cv::Mat matpic_org168;
cv::Mat matpic_roi160;
JMat* pic_clone160;//blendimg
cv::Mat matpic_roirst;
JMat* pic_realadjw;//blendimg
JMat* pic_maskadjw;
//JMat* pic_crop160;
//
JMat* msk_real160;
//JMat* msk_mask160;
cv::Mat matpic_roisrc;//box area
cv::Mat matpic_orgsrcw;
cv::Mat matpic_roiadjw;
JMat* pic_cloneadjw;//blendimg
cv::Mat matpic_roirst;
cv::Mat matmsk_roisrc;//box area
cv::Mat matmsk_org168;
cv::Mat matmsk_roi160;
//
JMat* msk_realadjw;
cv::Mat matmsk_roirst;
cv::Mat matmsk_roisrc;//box area
cv::Mat matmsk_orgsrcw;
cv::Mat matmsk_roiadjw;
int vtacc(uint8_t* buf,int count);
public:
MWorkMat(JMat* pic,JMat* msk,const int* boxs);
int premunet();
int munet(JMat** ppic,JMat** pmsk);
int finmunet(JMat* fgpic=NULL);
int prealpha();
int alpha(JMat** preal,JMat** pimg,JMat** pmsk);
int finalpha();
cv::Mat matmsk_roirst;
virtual ~MWorkMat();
int vtacc(uint8_t* buf,int count);
public:
MWorkMat(JMat* pic,JMat* msk,const int* boxs,int kind=168);
int premunet();
int munet(JMat** ppic,JMat** pmsk);
int finmunet(JMat* fgpic=NULL);
int prealpha();
int alpha(JMat** preal,JMat** pimg,JMat** pmsk);
int finalpha();
virtual ~MWorkMat();
};

35
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/duixsdka/dhunet/munet.cpp
View File

@@ -23,11 +23,10 @@ Mobunet::Mobunet(const char* modeldir,const char* modelid,int rgb){
int Mobunet::initModel(const char* binfn,const char* paramfn,const char* mskfn){
unet.clear();
//ncnn::set_cpu_powersave(2);
unet.opt = ncnn::Option();
// ncnn::set_omp_num_threads(2);//ncnn::get_big_cpu_count());
//ncnn::set_omp_num_threads(2);//ncnn::get_big_cpu_count());
//unet.opt = ncnn::Option();
unet.opt.use_vulkan_compute = false;
unet.opt.num_threads = 1;
unet.opt.num_threads = ncnn::get_big_cpu_count(); // 1
//unet.load_param("model/mobileunet_v5_wenet_sim.param");
//unet.load_model("model/mobileunet_v5_wenet_sim.bin");
unet.load_param(paramfn);
@@ -37,8 +36,12 @@ int Mobunet::initModel(const char* binfn,const char* paramfn,const char* mskfn){
printf("===mskfn %s\n",mskfn);
mat_weights = new JMat(160,160,(uint8_t*)wbuf,1);
mat_weights->forceref(0);
//mat_weights->show("weight");
//cv::waitKey(0);
mat_weightmin = new JMat(128,128,1);
cv::Mat ma = mat_weights->cvmat();
cv::Mat mb;
cv::resize(ma,mb,cv::Size(128,128));
cv::Mat mc = mat_weightmin->cvmat();
mb.copyTo(mc);
return 0;
}
@@ -82,20 +85,16 @@ int Mobunet::domodelold(JMat* pic,JMat* msk,JMat* feat){
cvadj.convertTo(cvreal,CV_8UC3,scale);
cv::Mat cvmask;
cv::cvtColor(cvreal,cvmask,cv::COLOR_RGB2BGR);
cv::imshow("cvreal",cvreal);
cv::imshow("cvmask",cvmask);
//cv::waitKey(0);
//getchar();
BlendGramAlpha((uchar*)cvmask.data,(uchar*)mat_weights->data(),(uchar*)pic->data(),160,160);
return 0;
}
int Mobunet::domodel(JMat* pic,JMat* msk,JMat* feat){
int Mobunet::domodel(JMat* pic,JMat* msk,JMat* feat,int rect){
int width = pic->width();
int height = pic->height();
ncnn::Mat inmask = ncnn::Mat::from_pixels(msk->udata(), m_rgb?ncnn::Mat::PIXEL_RGB:ncnn::Mat::PIXEL_BGR2RGB, 160, 160);
ncnn::Mat inmask = ncnn::Mat::from_pixels(msk->udata(), m_rgb?ncnn::Mat::PIXEL_RGB:ncnn::Mat::PIXEL_BGR2RGB, rect, rect);
inmask.substract_mean_normalize(mean_vals, norm_vals);
ncnn::Mat inreal = ncnn::Mat::from_pixels(pic->udata(), m_rgb?ncnn::Mat::PIXEL_RGB:ncnn::Mat::PIXEL_BGR2RGB, 160, 160);
ncnn::Mat inreal = ncnn::Mat::from_pixels(pic->udata(), m_rgb?ncnn::Mat::PIXEL_RGB:ncnn::Mat::PIXEL_BGR2RGB, rect, rect);
inreal.substract_mean_normalize(mean_vals, norm_vals);
ncnn::Mat inpic(width,height,6);
float* buf = (float*)inpic.data;
@@ -113,13 +112,19 @@ int Mobunet::domodel(JMat* pic,JMat* msk,JMat* feat){
ncnn::Extractor ex = unet.create_extractor();
ex.input("face", inpic);
ex.input("audio", inwenet);
//printf("===debug ncnn\n");
ex.extract("output", outpic);
float outmean_vals[3] = {-1.0f, -1.0f, -1.0f};
float outnorm_vals[3] = { 127.5f, 127.5f, 127.5f};
outpic.substract_mean_normalize(outmean_vals, outnorm_vals);
cv::Mat cvout(width,height,CV_8UC3);
outpic.to_pixels(cvout.data,m_rgb?ncnn::Mat::PIXEL_RGB:ncnn::Mat::PIXEL_RGB2BGR);
BlendGramAlpha((uchar*)cvout.data,(uchar*)mat_weights->data(),(uchar*)pic->data(),width,height);
if(rect==160){
BlendGramAlpha((uchar*)cvout.data,(uchar*)mat_weights->data(),(uchar*)pic->data(),width,height);
}else{
BlendGramAlpha((uchar*)cvout.data,(uchar*)mat_weightmin->data(),(uchar*)pic->data(),width,height);
}
return 0;
}
@@ -198,10 +203,6 @@ int Mobunet::process2(JMat* pic,const int* boxs,JMat* feat){
roimask.copyTo(cvreal);
cv::rectangle(cvmask,cv::Rect(5,5,150,150),cv::Scalar(0,0,0),-1);//,cv::LineTypes::FILLED);
// cv::imshow("000",cvorig);
// cv::imshow("aaa",cvmask);
// cv::imshow("bbb",cvreal);
// cv::waitKey(0);
ncnn::Mat inmask = ncnn::Mat::from_pixels(picmask.udata(), ncnn::Mat::PIXEL_BGR2RGB, 160, 160);
inmask.substract_mean_normalize(mean_vals, norm_vals);

3
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/duixsdka/dhunet/munet.h
View File

@@ -16,9 +16,10 @@ class Mobunet{
float mean_vals[3] = {127.5f, 127.5f, 127.5f};
float norm_vals[3] = {1 / 127.5f, 1 / 127.5f, 1 / 127.5f};
JMat* mat_weights = nullptr;
JMat* mat_weightmin = nullptr;
int initModel(const char* binfn,const char* paramfn,const char* mskfn);
public:
int domodel(JMat* pic,JMat* msk,JMat* feat);
int domodel(JMat* pic,JMat* msk,JMat* feat,int rect = 160);
int domodelold(JMat* pic,JMat* msk,JMat* feat);
int preprocess(JMat* pic,JMat* feat);
int process(JMat* pic,const int* boxs,JMat* feat);

119
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/duixsdka/duix/gjsimp.cpp
View File

@@ -7,10 +7,13 @@
#include "munet.h"
#include "malpha.h"
#include "dhwenet.h"
#include <queue>
//#include "Log.h"
struct dhduix_s{
int kind;
int rect;
int width;
int height;
int mincalc;
@@ -24,7 +27,7 @@ struct dhduix_s{
WeAI* weai_first;
WeAI* weai_common;
PcmSession* cursess;
PcmSession* presess;
//PcmSession* presess;
volatile uint64_t sessid;
jmat_t *mat_feat;
@@ -32,6 +35,8 @@ struct dhduix_s{
pthread_t *calcthread;
pthread_mutex_t pushmutex;
pthread_mutex_t readmutex;
pthread_mutex_t freemutex;
std::queue<PcmSession*> *slist;
int rgb;
Mobunet *munet;
@@ -50,7 +55,16 @@ static void *calcworker(void *arg){
rst = sess->runcalc(mfcc->sessid,mfcc->weai_common,mfcc->mincalc);
}
if(rst!=1){
jtimer_mssleep(20);
if(!mfcc->slist->empty()){
pthread_mutex_lock(&mfcc->freemutex);
PcmSession* sess = mfcc->slist->front();
mfcc->slist->pop();
delete sess;
pthread_mutex_unlock(&mfcc->freemutex);
jtimer_mssleep(10);
}else{
jtimer_mssleep(20);
}
}else{
jtimer_mssleep(10);
}
@@ -67,6 +81,8 @@ int dhduix_alloc(dhduix_t** pdg,int mincalc,int width,int height){
duix->maxblock = STREAM_BASE_MAXBLOCK;
pthread_mutex_init(&duix->pushmutex,NULL);
pthread_mutex_init(&duix->readmutex,NULL);
pthread_mutex_init(&duix->freemutex,NULL);
duix->slist = new std::queue<PcmSession*>();
duix->calcthread = (pthread_t *)malloc(sizeof(pthread_t) );
duix->running = 1;
pthread_create(duix->calcthread, NULL, calcworker, (void*)duix);
@@ -78,6 +94,8 @@ int dhduix_alloc(dhduix_t** pdg,int mincalc,int width,int height){
duix->mat_pic = new JMat(width,height);
//duix->mat_feat = jmat_alloc(20,STREAM_BASE_BNF,1,0,4,NULL);
duix->mat_feat = jmat_alloc(STREAM_BASE_BNF,20,1,0,4,NULL);
duix->kind = 168;
duix->rect = 160;
*pdg = duix;
return 0;
}
@@ -152,10 +170,19 @@ int dhduix_initWenet(dhduix_t* dg,char* fnwenet){
uint64_t dhduix_newsession(dhduix_t* dg){
uint64_t sessid = ++dg->sessid;
PcmSession* sess = new PcmSession(sessid,dg->minoff,dg->minblock,dg->maxblock);
PcmSession* olds = dg->presess;
dg->presess = dg->cursess;
//PcmSession* olds = dg->presess;
//dg->presess = dg->cursess;
//dg->cursess = sess;
//if(olds)delete olds;
pthread_mutex_lock(&dg->pushmutex);
pthread_mutex_lock(&dg->readmutex);
PcmSession* olds = dg->cursess;
dg->cursess = sess;
if(olds)delete olds;
pthread_mutex_unlock(&dg->pushmutex);
pthread_mutex_unlock(&dg->readmutex);
pthread_mutex_lock(&dg->freemutex);
dg->slist->push(olds);
pthread_mutex_unlock(&dg->freemutex);
return sessid;
}
@@ -172,7 +199,7 @@ int dhduix_pushpcm(dhduix_t* dg,uint64_t sessid,char* buf,int size,int kind){
if(sess->first()){
sess->runfirst(sessid,dg->weai_first);
uint64_t tick = jtimer_msstamp();
// printf("====runfirst %ld %ld \n",sessid,tick);
printf("====runfirst %ld %ld \n",sessid,tick);
}
return 0;
}else{
@@ -211,6 +238,17 @@ int dhduix_finsession(dhduix_t* dg,uint64_t sessid){
int dhduix_free(dhduix_t* dg){
dg->running = 0;
pthread_join(*dg->calcthread, NULL);
if(dg->slist){
pthread_mutex_lock(&dg->freemutex);
while(!dg->slist->empty()){
PcmSession* sess = dg->slist->front();
dg->slist->pop();
delete sess;
}
pthread_mutex_unlock(&dg->freemutex);
delete dg->slist;
}
if(dg->weai_first){
delete dg->weai_first;
dg->weai_first = NULL;
@@ -223,10 +261,10 @@ int dhduix_free(dhduix_t* dg){
delete dg->cursess;
dg->cursess = NULL;
}
if(dg->presess){
delete dg->presess;
dg->presess = NULL;
}
//if(dg->presess){
//delete dg->presess;
//dg->presess = NULL;
//}
if(dg->munet){
delete dg->munet;
dg->munet = NULL;
@@ -245,6 +283,7 @@ int dhduix_free(dhduix_t* dg){
}
pthread_mutex_destroy(&dg->pushmutex);
pthread_mutex_destroy(&dg->readmutex);
pthread_mutex_destroy(&dg->freemutex);
free(dg->calcthread);
jmat_free(dg->mat_feat);
free(dg);
@@ -257,6 +296,22 @@ int dhduix_initMunet(dhduix_t* dg,char* fnparam,char* fnbin,char* fnmsk){
dg->munet = new Mobunet(fnbin,fnparam,fnmsk,20,dg->rgb);
dg->inited = 1;
printf("===init munet \n");
dg->kind = 168;
dg->rect = 160;
return 0;
}
int dhduix_initMunetex(dhduix_t* dg,char* fnparam,char* fnbin,char* fnmsk,int rect){
dg->munet = new Mobunet(fnbin,fnparam,fnmsk,20,dg->rgb);
dg->inited = 1;
if(rect==128){
dg->kind = 128;
dg->rect = 128;
}else{
dg->kind = 168;
dg->rect = 160;
}
printf("===init munet \n");
return 0;
}
@@ -284,11 +339,13 @@ int dhduix_readycnt(dhduix_t* dg,uint64_t sessid){
return sess->calcBlock();
}
#define AIRUN_FLAG 1
int dhduix_fileinx(dhduix_t* dg,uint64_t sessid,char* fnpic,int* box,char* fnmsk,char* fnfg,int bnfinx,char* bimg,int imgsize){
int dhduix_fileinx(dhduix_t* dg,uint64_t sessid,char* fnpic,int* box,char* fnmsk,char* fnfg,int bnfinx,char* bimg,char* mskbuf,int imgsize){
if(sessid!=dg->sessid)return -1;
if(!dg->running)return -2;
uint64_t ticka = jtimer_msstamp();
std::string sfnpic(fnpic);
std::string sfnmsk(fnmsk);
std::string sfnfg(fnfg);
@@ -302,6 +359,7 @@ int dhduix_fileinx(dhduix_t* dg,uint64_t sessid,char* fnpic,int* box,char* fnmsk
mat_msk = dg->mat_msk;
mat_msk->loadjpg(sfnmsk,1);
bmsk = (uint8_t*)mat_msk->data();
memcpy(mskbuf,bmsk,dg->width*dg->height*3);
}
JMat* mat_fg = NULL;
if(sfnfg.length()){
@@ -309,8 +367,23 @@ int dhduix_fileinx(dhduix_t* dg,uint64_t sessid,char* fnpic,int* box,char* fnmsk
mat_fg->loadjpg(sfnfg,1);
bfg = (uint8_t*)mat_fg->data();
}
int rst = dhduix_simpinx(dg,sessid, bpic,dg->width,dg->height, box, bmsk, bfg,bnfinx);
return 0;
uint64_t tickb = jtimer_msstamp();
uint64_t dist = tickb-ticka;
//LOGD("tooken","===loadjpg %ld\n",dist);
int rst = 0;
if(box){
rst = dhduix_simpinx(dg,sessid, bpic,dg->width,dg->height, box, bmsk, bfg,bnfinx);
}else{
rst = dhduix_simpblend(dg,sessid, bpic,dg->width,dg->height, bmsk, bfg);
}
int size = dg->width*dg->height*3;
if(bfg){
memcpy(bimg,bfg,size);
}else{
memcpy(bimg,bpic,size);
}
if(bmsk) memcpy(mskbuf,bmsk,size);
return rst;
}
int dhduix_simpinx(dhduix_t* dg,uint64_t sessid,uint8_t* bpic,int width,int height,int* box,uint8_t* bmsk,uint8_t* bfg,int inx){
@@ -319,17 +392,24 @@ int dhduix_simpinx(dhduix_t* dg,uint64_t sessid,uint8_t* bpic,int width,int heig
PcmSession* sess = dg->cursess;
if(!sess)return -3;
int rst = 0;
int w = width?width:dg->width;
int h = height?height:dg->height;
pthread_mutex_lock(&dg->readmutex);
rst = sess->readblock(sessid,dg->mat_feat,inx);
pthread_mutex_unlock(&dg->readmutex);
// printf("===readblock %d\n",rst);
//printf("===readblock %d\n",rst);
if(rst>0){
rst = dhduix_simprst(dg,sessid, bpic,width,height, box, bmsk, bfg,(uint8_t*)dg->mat_feat->data,STREAM_ALL_BNF);
rst = dhduix_simprst(dg,sessid, bpic,w,h, box, bmsk, bfg,(uint8_t*)dg->mat_feat->data,STREAM_ALL_BNF);
return 1;
}
return rst;
}
int dhduix_simpblend(dhduix_t* dg,uint64_t sessid,uint8_t* bpic,int width,int height,uint8_t* bmsk,uint8_t* bfg){
//
return 0;
}
int dhduix_simprst(dhduix_t* dg,uint64_t sessid,uint8_t* bpic,int width,int height,int* box,uint8_t* bmsk,uint8_t* bfg,uint8_t* bnfbuf,int bnflen){
//printf("simprst gogogo %d \n",dg->inited);
if(!dg->inited)return -1;
@@ -341,7 +421,8 @@ int dhduix_simprst(dhduix_t* dg,uint64_t sessid,uint8_t* bpic,int width,int heig
//read pcm
JMat* feat = new JMat(STREAM_CNT_BNF,STREAM_BASE_BNF,(float*)bnfbuf,1);
MWorkMat wmat(mat_pic,NULL,box);
// MWorkMat wmat(mat_pic,mat_msk,box);
MWorkMat wmat(mat_pic, NULL,box,dg->kind);
wmat.premunet();
JMat* mpic;
JMat* mmsk;
@@ -349,9 +430,10 @@ int dhduix_simprst(dhduix_t* dg,uint64_t sessid,uint8_t* bpic,int width,int heig
//tooken
#ifdef AIRUN_FLAG
uint64_t ticka = jtimer_msstamp();
rst = dg->munet->domodel(mpic, mmsk, feat);
rst = dg->munet->domodel(mpic, mmsk, feat,dg->rect);
uint64_t tickb = jtimer_msstamp();
uint64_t dist = tickb-ticka;
//LOGD("tooken","===domodel %ld\n",dist);
if(dist>40){
printf("===domodel %d dist %ld\n",rst,dist);
}
@@ -365,7 +447,6 @@ int dhduix_simprst(dhduix_t* dg,uint64_t sessid,uint8_t* bpic,int width,int heig
delete mat_pic;
if(mat_fg)delete mat_fg;
if(mat_msk)delete mat_msk;
return 0;
}

4
duix-ios/GJLocalDigitalDemo/GJLocalDigitalSDK/GJLocalDigitalSDK/duixsdka/duix/gjsimp.h
View File

@@ -13,6 +13,7 @@ int dhduix_alloc(dhduix_t** pdg,int mincalc,int width,int height);
int dhduix_initPcmex(dhduix_t* dg,int maxsize,int minoff ,int minblock ,int maxblock,int rgb);
int dhduix_initWenet(dhduix_t* dg,char* fnwenet);
int dhduix_initMunet(dhduix_t* dg,char* fnparam,char* fnbin,char* fnmsk);
int dhduix_initMunetex(dhduix_t* dg,char* fnparam,char* fnbin,char* fnmsk,int rect);
uint64_t dhduix_newsession(dhduix_t* dg);
@@ -23,7 +24,8 @@ int dhduix_simprst(dhduix_t* dg,uint64_t sessid,uint8_t* bpic,int width,int heig
int dhduix_allcnt(dhduix_t* dg,uint64_t sessid);
int dhduix_readycnt(dhduix_t* dg,uint64_t sessid);
int dhduix_simpinx(dhduix_t* dg,uint64_t sessid,uint8_t* bpic,int width,int height,int* box,uint8_t* bmsk,uint8_t* bfg,int bnfinx);
int dhduix_fileinx(dhduix_t* dg,uint64_t sessid,char* fnpic,int* box,char* fnmsk,char* fnfg,int bnfinx,char* bimg,int imgsize);
int dhduix_fileinx(dhduix_t* dg,uint64_t sessid,char* fnpic,int* box,char* fnmsk,char* fnfg,int bnfinx,char* bimg,char* mskbuf,int imgsize);
int dhduix_simpblend(dhduix_t* dg,uint64_t sessid,uint8_t* bpic,int width,int height,uint8_t* bmsk,uint8_t* bfg);
int dhduix_simppcm(dhduix_t* dg,char* buf,int size,char* pre,int presize,char* bnf,int bnfsize);

261
duix-ios/GJLocalDigitalDemo/README.md
View File

@@ -1,84 +1,81 @@
# Duix Mobile for iOS SDK ⽂档
# Duix Mobile for iOS SDK Documentation
简体中文 | [English](./README_en.md)
English | [中文](./README_zh.md)
## 一、产品概述
## 1. Product Overview
`Duix Mobile for iOS` 是一套轻量级的本地部署 2D 虚拟人解决方案，适用于 iOS 端，支持通过语音实时驱动虚拟人形象进行口型播报和动作响应。
`Duix Mobile for iOS` is a lightweight, locally deployed 2D digital human solution for iOS, supporting real-time avatar lip-sync and motion response driven by voice.
### 1.1 适用场景
### 1.1 Application Scenarios
- **部署成本低**：无服务端依赖，适用于政务终端、机场、展厅等场景。
- **弱网友好**：支持完全离线运行。
- **功能多样化**：支持导览播报、问答服务、数字迎宾等多种智能场景。
### 1.2 核心功能
- **数字人渲染与驱动**：支持本地渲染虚拟人形象，响应语音输入实时口型驱动；
- **语音播报控制**：支持音频播放、PCM 推流、动作与播报联动；
- **动作控制系统**：可自定义启动、停止、随机动作；
- **Low deployment cost**: No server dependency, suitable for government terminals, airports, exhibition halls, and more.
- **Offline friendly**: Fully offline operation supported.
- **Diverse functionality**: Supports guided broadcasting, Q&A services, digital reception, and other intelligent scenarios.
### 1.2 Core Features
- **Digital human rendering and driving**: Supports local avatar rendering, real-time lip-sync driven by voice input.
- **Voice playback control**: Supports audio playback, PCM streaming, and action-broadcast linkage.
- **Motion control system**: Customizable start, stop, and random actions.
---
## 二、术语说明
## 2. Terminology
| 术语 | 含义 |
|----------------|--------------------------------------------------------------|
| PCM | 原始音频流格式（16kHz，16bit，Mono） |
| WAV | 音频文件格式，适用于短语音播放，内部仍为 PCM 编码 |
| Session | 一次完整播报流程（推送→响应→结束） |
| DUIX-PRO | 本地渲染与驱动管理器，实现模型加载、渲染控制、播报驱动等功能 |
| GJLPCMManager | 提供的 PCM 管理类，用于处理音频文件和推送逻辑 |
| Term | Description |
|-----------------|-----------------------------------------------------------------------------|
| PCM | Raw audio stream format (16kHz, 16bit, Mono) |
| WAV | Audio file format suitable for short voice playback (internally PCM encoded)|
| Session | Complete broadcast process (Push → Response → End) |
| DUIX-PRO | Local rendering and driving manager handling model loading, render control, etc. |
| GJLPCMManager | PCM management class for handling audio files and streaming logic |
---
## 三、SDK 获取与集成
## 3. SDK Acquisition & Integration
### 3.1 Manual Integration (Recommended)
### 3.1 手动集成（推荐）
1. 将 `GJLocalDigitalSDK.framework` 拖入 Xcode 项目中，设置为：**Embed & Sign**。
2. 在 `Build Phases > Link Binary With Libraries` 中添加：`AVFoundation.framework`。
3. Info.plist 中添加麦克风权限：
1. Drag `GJLocalDigitalSDK.framework` into Xcode project → Set to **Embed & Sign**
2. Add `AVFoundation.framework` in `Build Phases > Link Binary With Libraries`
3. Add microphone permission in Info.plist:
```xml
<key>NSMicrophoneUsageDescription</key>
<string>App需要使用麦克风权限驱动数字人语音播报</string>
<string>App requires microphone access to drive digital human voice broadcast</string>
```
---
## 四、集成要求
## 4. Integration Requirements
| 项目 | 要求 |
|--------------|----------------------------------|
| 系统版本 | iOS 12.0 及以上 |
| 开发工具 | Xcode 12 及以上 |
| 支持设备 | iPhone 8 及以上 |
| 运行环境 | 支持离线，无需联网 |
| CPU 与内存 | 推荐 A12 芯片及以上，内存 ≥ 3GB |
| Item | Requirement |
|----------------|------------------------------------------|
| OS Version | iOS 12.0+ |
| Development Tools | Xcode 12+ |
| Supported Devices | iPhone 8+ |
| Runtime Environment | Offline operation (no network required) |
| CPU & Memory | Recommended A12+ chip, ≥3GB RAM |
---
## 五、使用流程概览
## 5. Workflow Overview
```mermaid
graph TD
A[检查配置与模型] --> B[构建 DUIX 实例]
B --> C[调用 init 初始化]
C --> D[展示形象 / 渲染]
D --> E[PCM 或 WAV 音频驱动]
E --> F[播放控制与动作触发]
F --> G[资源释放]
A[Check Configuration and Model] --> B[Build DUIX Instance]
B --> C[Call init to initialize]
C --> D[Display Image / Render]
D --> E[PCM or WAV Audio Driving]
E --> F[Playback Control and Action Trigger]
F --> G[Resource Release]
```
---
## 六、快速开始示例
```objc
## 6. Quick Start Example
```
NSInteger result = [[GJLDigitalManager manager] initBaseModel:basePath
digitalModel:digitalPath
showView:self.showView];
@@ -87,283 +84,273 @@ if (result == 1) {
if (isSuccess) {
[[GJLDigitalManager manager] toStartRuning];
} else {
NSLog(@"启动失败：%@", errorMsg);
NSLog(@"Start failed: %@", errorMsg);
}
}];
}
```
> 注意：basePath 为基础资源目录，digitalPath 为模型目录
```
> Note: basePath = base resource directory, digitalPath = model directory
---
## 七、关键接口与使用说明
## 7. Key Interfaces & Usage
### 7.1 初始化配置
### 7.1 Initialization Configuration
```
/**
* 初始化数字人服务
* @param basePath 基础模型路径（固定不变）
* @param digitalPath 数字人模型路径（替换数字人时更新此路径）
* @param showView 数字人渲染视图
* @return 状态码 1=成功, 0=未授权, -1=失败
* Initialize digital human service
* @param basePath Base model path (fixed)
* @param digitalPath Digital human model path (update when replacing digital human)
* @param showView Digital human rendering view
* @return Status code 1=success, 0=unauthorized, -1=failure
*/
-(NSInteger)initBaseModel:(NSString*)basePath digitalModel:(NSString*)digitalPath showView:(UIView*)showView;
```
### 7.2 渲染数字人控制
### 7.2 Digital Human Rendering Control
```
/*
*启动数字人渲染
* Start digital human rendering
*/
-(void)toStart:(void (^) (BOOL isSuccess, NSString *errorMsg))block;
```
```
/*
*停止渲染并释放资源
* Stop rendering and release resources
*/
-(void)toStop;
```
```
/*
*恢复播放（暂停后调用）
* Resume playback (call after pause)
*/
-(void)toPlay;
```
```
/*
*暂停数字人播放
* Pause digital human playback
*/
-(void)toPause;
```
### 7.3 背景管理
### 7.3 Background Management
```
/**
* 动态替换背景
* @param bbgPath JPG格式背景图路径
* Dynamically replace background
* @param bbgPath JPG format background image path
*/
-(void)toChangeBBGWithPath:(NSString*)bbgPath;
```
### 7.4 音频播报控制
### 7.4 Audio Control
```
/*
*audioData播放音频流 ，参考demo里面GJLPCMManager类里toSpeakWithPath 转换成pcm的代码
*驱动数字人播报(PCM流)
* Play audio stream (PCM format), refer to toSpeakWithPath in GJLPCMManager demo class for PCM conversion
* Drive digital human broadcast (PCM stream)
*/
-(void)toWavPcmData:(NSData*)audioData;
```
```
/*
* 开始音频流播放
* Start audio stream playback
*/
- (void)startPlaying;
```
```
/*
* 结束音频流播放
* Stop audio stream playback
*/
- (void)stopPlaying:(void (^)( BOOL isSuccess))success;
```
```
/*
*设置静音模式
* Set mute mode
*/
-(void)toMute:(BOOL)isMute;
```
```
/*
*清空音频缓冲区
* Clear audio buffer
*/
-(void)clearAudioBuffer;
```
```
/*
*暂停播放音频流
* Pause audio stream playback
*/
-(void)toPausePcm;
```
```
/*
*恢复播放音频流
* Resume audio stream playback
*/
-(void)toResumePcm;
```
```
/*
* 是否启用录音
* Enable/disable recording
*/
-(void)toEnableRecord:(BOOL)isEnable;
```
### 7.5 Streaming Session Management
### 7.5 流式会话管理
```
/*
*启动流式会话
* Start streaming session
*/
-(void)toStartRuning;
```
```
/*
*开始新会话（单句/段落）
* Start new session (single sentence/paragraph)
*/
-(void)newSession;
```
```
/*
*结束当前会话
* End current session
*/
-(void)finishSession;
```
```
/*
*继续会话（finish后调用）
* Continue session (call after finish)
*/
-(void)continueSession;
```
### 7.6 动作控制
### 7.6 Motion Control
```
/*
* 启用随机动作（建议在首段音频开始时调用）
* 返回：0=不支持, 1=成功
* Enable random motions (recommended at start of first audio segment)
* Return: 0=unsupported, 1=success
*/
-(NSInteger)toRandomMotion;
```
```
/*
* 启用开始动作（首段音频开始时调用）
* 返回：0=不支持, 1=成功
* Enable start motion (call at beginning of first audio segment)
* Return: 0=unsupported, 1=success
*/
-(NSInteger)toStartMotion;
```
```
/*
* 结束动作（末段音频结束时调用）
*isQuickly: YES=立即结束, NO=等待动作完成
*返回：0=不支持, 1=成功
* End motion (call at end of last audio segment)
* isQuickly: YES=end immediately, NO=wait for motion completion
* Return: 0=unsupported, 1=success
*/
-(NSInteger)toSopMotion:(BOOL)isQuickly;
```
### 7.7 状态查询
### 7.7 Status Queries
```
/*
*获取数字人模型尺寸（需初始化后调用）
* Get digital human model dimensions (call after initialization)
*/
-(CGSize)getDigitalSize;
```
```
/*
*检查授权状态（1=已授权）
* Check authorization status (1=authorized)
*/
-(NSInteger)isGetAuth;
```
---
## 八、回调定义
## 8. Callback Definitions
```
/*
*数字人渲染报错
*错误码说明：
* 0 = 未授权
* -1 = 未初始化
* 50009 = 资源超时/未配置
* Digital human rendering error
* Error codes:
* 0 = Unauthorized
* -1 = Uninitialized
* 50009 = Resource timeout/unconfigured
*/
@property (nonatomic, copy) void (^playFailed)(NSInteger code,NSString *errorMsg);
```
```
/*
*音频播放结束回调
* Audio playback ended callback
*/
@property (nonatomic, copy) void (^audioPlayEnd)(void);
```
```
/*
*音频播放进度回调
/
* Audio playback progress callback
*/
@property (nonatomic, copy) void (^audioPlayProgress)(float current,float total);
```
---
## 九、常见问题与排查指南
## 9. FAQ & Troubleshooting
| 问题现象 | 可能原因 | 建议处理方式 |
|--------------------|------------------------|--------------------------------------|
| 初始化返回 -1 | SDK 授权失败 | 检查 info.plist 是否包含授权字段 |
| 无法渲染画面 | showView 为空或未添加 | 确保 viewController 中已渲染视图挂载 |
| 播报无响应 | 音频格式错误或路径无效 | 确保 PCM 格式正确 / 路径有效 |
| 播放提前中断 | 会话未续接 / 缓冲区溢出 | 检查是否正确调用 `continueSession` |
| Symptom | Possible Cause | Recommended Solution |
|-----------------------|------------------------------|------------------------------------|
| Initialization returns -1 | SDK authorization failed | Check info.plist for auth fields |
| No rendered output | showView empty or not added | Ensure view is mounted in controller |
| No broadcast response | Invalid audio format/path | Verify PCM format/path validity |
| Premature playback stop| Session not continued/buffer overflow | Check `continueSession` usage |
---
## 10. Version History
## 十、版本更新记录
### v1.2.3
- Added support for 128 models
### v1.2.0
- 新增 PCM 推流支持
- Added PCM streaming support
### v1.0.3
- 支持透明背景
- 优化模型解压内存
- Supported transparent backgrounds
- Optimized model decompression memory
### v1.0.2
- 支持问答 / 语音识别 / 动作标注 / 合成播报
- Supported Q&A / speech recognition / motion tagging / synthesized broadcast
### v1.0.1
- 初始版本：授权 + 渲染 + 播报
- Initial version: authorization + rendering + broadcast
---
## 🔗 开源依赖
## 🔗 Open Source Dependencies
| 模块 | 描述 |
|-------------------------------------------|------------------------------|
| [ONNX](https://github.com/onnx/onnx) | 通用 AI 模型标准格式 |
| [ncnn](https://github.com/Tencent/ncnn) | 高性能神经网络推理框架（腾讯） |
---
如需更多集成帮助，请联系技术支持。
| Module | Description |
|------------------------------------------|----------------------------------|
| [ONNX](https://github.com/onnx/onnx) | Universal AI model format |
| [ncnn](https://github.com/Tencent/ncnn) | High-performance neural network inference framework (Tencent) |
For additional integration support, please contact technical support.

352
duix-ios/GJLocalDigitalDemo/README_en.md
View File

@@ -1,352 +0,0 @@
# Duix Mobile for iOS SDK Documentation
[简体中文](./README.md) | English
## 1. Product Overview
`Duix Mobile for iOS` is a lightweight, locally deployed 2D digital human solution for iOS, supporting real-time avatar lip-sync and motion response driven by voice.
### 1.1 Application Scenarios
- **Low deployment cost**: No server dependency, suitable for government terminals, airports, exhibition halls, and more.
- **Offline friendly**: Fully offline operation supported.
- **Diverse functionality**: Supports guided broadcasting, Q&A services, digital reception, and other intelligent scenarios.
### 1.2 Core Features
- **Digital human rendering and driving**: Supports local avatar rendering, real-time lip-sync driven by voice input.
- **Voice playback control**: Supports audio playback, PCM streaming, and action-broadcast linkage.
- **Motion control system**: Customizable start, stop, and random actions.
---
## 2. Terminology
| Term | Description |
|-----------------|-----------------------------------------------------------------------------|
| PCM | Raw audio stream format (16kHz, 16bit, Mono) |
| WAV | Audio file format suitable for short voice playback (internally PCM encoded)|
| Session | Complete broadcast process (Push → Response → End) |
| DUIX-PRO | Local rendering and driving manager handling model loading, render control, etc. |
| GJLPCMManager | PCM management class for handling audio files and streaming logic |
---
## 3. SDK Acquisition & Integration
### 3.1 Manual Integration (Recommended)
1. Drag `GJLocalDigitalSDK.framework` into Xcode project → Set to **Embed & Sign**
2. Add `AVFoundation.framework` in `Build Phases > Link Binary With Libraries`
3. Add microphone permission in Info.plist:
```xml
<key>NSMicrophoneUsageDescription</key>
<string>App requires microphone access to drive digital human voice broadcast</string>
```
---
## 4. Integration Requirements
| Item | Requirement |
|----------------|------------------------------------------|
| OS Version | iOS 12.0+ |
| Development Tools | Xcode 12+ |
| Supported Devices | iPhone 8+ |
| Runtime Environment | Offline operation (no network required) |
| CPU & Memory | Recommended A12+ chip, ≥3GB RAM |
---
## 5. Workflow Overview
```mermaid
graph TD
A[Check Configuration and Model] --> B[Build DUIX Instance]
B --> C[Call init to initialize]
C --> D[Display Image / Render]
D --> E[PCM or WAV Audio Driving]
E --> F[Playback Control and Action Trigger]
F --> G[Resource Release]
```
---
## 6. Quick Start Example
```
NSInteger result = [[GJLDigitalManager manager] initBaseModel:basePath
digitalModel:digitalPath
showView:self.showView];
if (result == 1) {
[[GJLDigitalManager manager] toStart:^(BOOL isSuccess, NSString *errorMsg) {
if (isSuccess) {
[[GJLDigitalManager manager] toStartRuning];
} else {
NSLog(@"Start failed: %@", errorMsg);
}
}];
}
```
> Note: basePath = base resource directory, digitalPath = model directory
---
## 7. Key Interfaces & Usage
### 7.1 Initialization Configuration
```
/**
* Initialize digital human service
* @param basePath Base model path (fixed)
* @param digitalPath Digital human model path (update when replacing digital human)
* @param showView Digital human rendering view
* @return Status code 1=success, 0=unauthorized, -1=failure
*/
-(NSInteger)initBaseModel:(NSString*)basePath digitalModel:(NSString*)digitalPath showView:(UIView*)showView;
```
### 7.2 Digital Human Rendering Control
```
/*
* Start digital human rendering
*/
-(void)toStart:(void (^) (BOOL isSuccess, NSString *errorMsg))block;
```
```
/*
* Stop rendering and release resources
*/
-(void)toStop;
```
```
/*
* Resume playback (call after pause)
*/
-(void)toPlay;
```
```
/*
* Pause digital human playback
*/
-(void)toPause;
```
### 7.3 Background Management
```
/**
* Dynamically replace background
* @param bbgPath JPG format background image path
*/
-(void)toChangeBBGWithPath:(NSString*)bbgPath;
```
### 7.4 Audio Control
```
/*
* Play audio stream (PCM format), refer to toSpeakWithPath in GJLPCMManager demo class for PCM conversion
* Drive digital human broadcast (PCM stream)
*/
-(void)toWavPcmData:(NSData*)audioData;
```
```
/*
* Start audio stream playback
*/
- (void)startPlaying;
```
```
/*
* Stop audio stream playback
*/
- (void)stopPlaying:(void (^)( BOOL isSuccess))success;
```
```
/*
* Set mute mode
*/
-(void)toMute:(BOOL)isMute;
```
```
/*
* Clear audio buffer
*/
-(void)clearAudioBuffer;
```
```
/*
* Pause audio stream playback
*/
-(void)toPausePcm;
```
```
/*
* Resume audio stream playback
*/
-(void)toResumePcm;
```
```
/*
* Enable/disable recording
*/
-(void)toEnableRecord:(BOOL)isEnable;
```
### 7.5 Streaming Session Management
```
/*
* Start streaming session
*/
-(void)toStartRuning;
```
```
/*
* Start new session (single sentence/paragraph)
*/
-(void)newSession;
```
```
/*
* End current session
*/
-(void)finishSession;
```
```
/*
* Continue session (call after finish)
*/
-(void)continueSession;
```
### 7.6 Motion Control
```
/*
* Enable random motions (recommended at start of first audio segment)
* Return: 0=unsupported, 1=success
*/
-(NSInteger)toRandomMotion;
```
```
/*
* Enable start motion (call at beginning of first audio segment)
* Return: 0=unsupported, 1=success
*/
-(NSInteger)toStartMotion;
```
```
/*
* End motion (call at end of last audio segment)
* isQuickly: YES=end immediately, NO=wait for motion completion
* Return: 0=unsupported, 1=success
*/
-(NSInteger)toSopMotion:(BOOL)isQuickly;
```
### 7.7 Status Queries
```
/*
* Get digital human model dimensions (call after initialization)
*/
-(CGSize)getDigitalSize;
```
```
/*
* Check authorization status (1=authorized)
*/
-(NSInteger)isGetAuth;
```
---
## 8. Callback Definitions
```
/*
* Digital human rendering error
* Error codes:
* 0 = Unauthorized
* -1 = Uninitialized
* 50009 = Resource timeout/unconfigured
*/
@property (nonatomic, copy) void (^playFailed)(NSInteger code,NSString *errorMsg);
```
```
/*
* Audio playback ended callback
*/
@property (nonatomic, copy) void (^audioPlayEnd)(void);
```
```
/*
* Audio playback progress callback
*/
@property (nonatomic, copy) void (^audioPlayProgress)(float current,float total);
```
---
## 9. FAQ & Troubleshooting
| Symptom | Possible Cause | Recommended Solution |
|-----------------------|------------------------------|------------------------------------|
| Initialization returns -1 | SDK authorization failed | Check info.plist for auth fields |
| No rendered output | showView empty or not added | Ensure view is mounted in controller |
| No broadcast response | Invalid audio format/path | Verify PCM format/path validity |
| Premature playback stop| Session not continued/buffer overflow | Check `continueSession` usage |
## 10. Version History
### v1.2.0
- Added PCM streaming support
### v1.0.3
- Supported transparent backgrounds
- Optimized model decompression memory
### v1.0.2
- Supported Q&A / speech recognition / motion tagging / synthesized broadcast
### v1.0.1
- Initial version: authorization + rendering + broadcast
---
## 🔗 Open Source Dependencies
| Module | Description |
|------------------------------------------|----------------------------------|
| [ONNX](https://github.com/onnx/onnx) | Universal AI model format |
| [ncnn](https://github.com/Tencent/ncnn) | High-performance neural network inference framework (Tencent) |
For additional integration support, please contact technical support.

373
duix-ios/GJLocalDigitalDemo/README_zh.md Normal file
View File

@@ -0,0 +1,373 @@
# Duix Mobile for iOS SDK ⽂档
中文 | [English](./README.md)
## 一、产品概述
`Duix Mobile for iOS` 是一套轻量级的本地部署 2D 虚拟人解决方案，适用于 iOS 端，支持通过语音实时驱动虚拟人形象进行口型播报和动作响应。
### 1.1 适用场景
- **部署成本低**：无服务端依赖，适用于政务终端、机场、展厅等场景。
- **弱网友好**：支持完全离线运行。
- **功能多样化**：支持导览播报、问答服务、数字迎宾等多种智能场景。
### 1.2 核心功能
- **数字人渲染与驱动**：支持本地渲染虚拟人形象，响应语音输入实时口型驱动；
- **语音播报控制**：支持音频播放、PCM 推流、动作与播报联动；
- **动作控制系统**：可自定义启动、停止、随机动作；
---
## 二、术语说明
| 术语 | 含义 |
|----------------|--------------------------------------------------------------|
| PCM | 原始音频流格式（16kHz，16bit，Mono） |
| WAV | 音频文件格式，适用于短语音播放，内部仍为 PCM 编码 |
| Session | 一次完整播报流程（推送→响应→结束） |
| DUIX-PRO | 本地渲染与驱动管理器，实现模型加载、渲染控制、播报驱动等功能 |
| GJLPCMManager | 提供的 PCM 管理类，用于处理音频文件和推送逻辑 |
---
## 三、SDK 获取与集成
### 3.1 手动集成（推荐）
1. 将 `GJLocalDigitalSDK.framework` 拖入 Xcode 项目中，设置为：**Embed & Sign**。
2. 在 `Build Phases > Link Binary With Libraries` 中添加：`AVFoundation.framework`。
3. Info.plist 中添加麦克风权限：
```xml
<key>NSMicrophoneUsageDescription</key>
<string>App需要使用麦克风权限驱动数字人语音播报</string>
```
---
## 四、集成要求
| 项目 | 要求 |
|--------------|----------------------------------|
| 系统版本 | iOS 12.0 及以上 |
| 开发工具 | Xcode 12 及以上 |
| 支持设备 | iPhone 8 及以上 |
| 运行环境 | 支持离线，无需联网 |
| CPU 与内存 | 推荐 A12 芯片及以上，内存 ≥ 3GB |
---
## 五、使用流程概览
```mermaid
graph TD
A[检查配置与模型] --> B[构建 DUIX 实例]
B --> C[调用 init 初始化]
C --> D[展示形象 / 渲染]
D --> E[PCM 或 WAV 音频驱动]
E --> F[播放控制与动作触发]
F --> G[资源释放]
```
---
## 六、快速开始示例
```objc
NSInteger result = [[GJLDigitalManager manager] initBaseModel:basePath
digitalModel:digitalPath
showView:self.showView];
if (result == 1) {
[[GJLDigitalManager manager] toStart:^(BOOL isSuccess, NSString *errorMsg) {
if (isSuccess) {
[[GJLDigitalManager manager] toStartRuning];
} else {
NSLog(@"启动失败：%@", errorMsg);
}
}];
}
```
> 注意：basePath 为基础资源目录，digitalPath 为模型目录
---
## 七、关键接口与使用说明
### 7.1 初始化配置
```
/**
* 初始化数字人服务
* @param basePath 基础模型路径（固定不变）
* @param digitalPath 数字人模型路径（替换数字人时更新此路径）
* @param showView 数字人渲染视图
* @return 状态码 1=成功, 0=未授权, -1=失败
*/
-(NSInteger)initBaseModel:(NSString*)basePath digitalModel:(NSString*)digitalPath showView:(UIView*)showView;
```
### 7.2 渲染数字人控制
```
/*
*启动数字人渲染
*/
-(void)toStart:(void (^) (BOOL isSuccess, NSString *errorMsg))block;
```
```
/*
*停止渲染并释放资源
*/
-(void)toStop;
```
```
/*
*恢复播放（暂停后调用）
*/
-(void)toPlay;
```
```
/*
*暂停数字人播放
*/
-(void)toPause;
```
### 7.3 背景管理
```
/**
* 动态替换背景
* @param bbgPath JPG格式背景图路径
*/
-(void)toChangeBBGWithPath:(NSString*)bbgPath;
```
### 7.4 音频播报控制
```
/*
*audioData播放音频流 ，参考demo里面GJLPCMManager类里toSpeakWithPath 转换成pcm的代码
*驱动数字人播报(PCM流)
*/
-(void)toWavPcmData:(NSData*)audioData;
```
```
/*
* 开始音频流播放
*/
- (void)startPlaying;
```
```
/*
* 结束音频流播放
*/
- (void)stopPlaying:(void (^)( BOOL isSuccess))success;
```
```
/*
*设置静音模式
*/
-(void)toMute:(BOOL)isMute;
```
```
/*
*清空音频缓冲区
*/
-(void)clearAudioBuffer;
```
```
/*
*暂停播放音频流
*/
-(void)toPausePcm;
```
```
/*
*恢复播放音频流
*/
-(void)toResumePcm;
```
```
/*
* 是否启用录音
*/
-(void)toEnableRecord:(BOOL)isEnable;
```
### 7.5 流式会话管理
```
/*
*启动流式会话
*/
-(void)toStartRuning;
```
```
/*
*开始新会话（单句/段落）
*/
-(void)newSession;
```
```
/*
*结束当前会话
*/
-(void)finishSession;
```
```
/*
*继续会话（finish后调用）
*/
-(void)continueSession;
```
### 7.6 动作控制
```
/*
* 启用随机动作（建议在首段音频开始时调用）
* 返回：0=不支持, 1=成功
*/
-(NSInteger)toRandomMotion;
```
```
/*
* 启用开始动作（首段音频开始时调用）
* 返回：0=不支持, 1=成功
*/
-(NSInteger)toStartMotion;
```
```
/*
* 结束动作（末段音频结束时调用）
*isQuickly: YES=立即结束, NO=等待动作完成
*返回：0=不支持, 1=成功
*/
-(NSInteger)toSopMotion:(BOOL)isQuickly;
```
### 7.7 状态查询
```
/*
*获取数字人模型尺寸（需初始化后调用）
*/
-(CGSize)getDigitalSize;
```
```
/*
*检查授权状态（1=已授权）
*/
-(NSInteger)isGetAuth;
```
---
## 八、回调定义
```
/*
*数字人渲染报错
*错误码说明：
* 0 = 未授权
* -1 = 未初始化
* 50009 = 资源超时/未配置
*/
@property (nonatomic, copy) void (^playFailed)(NSInteger code,NSString *errorMsg);
/*
*音频播放结束回调
*/
@property (nonatomic, copy) void (^audioPlayEnd)(void);
/*
*音频播放进度回调
/
@property (nonatomic, copy) void (^audioPlayProgress)(float current,float total);
```
---
## 九、常见问题与排查指南
| 问题现象 | 可能原因 | 建议处理方式 |
|--------------------|------------------------|--------------------------------------|
| 初始化返回 -1 | SDK 授权失败 | 检查 info.plist 是否包含授权字段 |
| 无法渲染画面 | showView 为空或未添加 | 确保 viewController 中已渲染视图挂载 |
| 播报无响应 | 音频格式错误或路径无效 | 确保 PCM 格式正确 / 路径有效 |
| 播放提前中断 | 会话未续接 / 缓冲区溢出 | 检查是否正确调用 `continueSession` |
---
## 十、版本更新记录
### v1.2.3
- 新增 支持 128模型
### v1.2.0
- 新增 PCM 推流支持
### v1.0.3
- 支持透明背景
- 优化模型解压内存
### v1.0.2
- 支持问答 / 语音识别 / 动作标注 / 合成播报
### v1.0.1
- 初始版本：授权 + 渲染 + 播报
---
## 🔗 开源依赖
| 模块 | 描述 |
|-------------------------------------------|------------------------------|
| [ONNX](https://github.com/onnx/onnx) | 通用 AI 模型标准格式 |
| [ncnn](https://github.com/Tencent/ncnn) | 高性能神经网络推理框架（腾讯） |
---
如需更多集成帮助，请联系技术支持。

32
duix-ios/LICENSE
View File

@@ -1,32 +0,0 @@
Silicon Intelligence COMMUNITY LICENSE AGREEMENT
“Agreement” means the terms and conditions for use, reproduction, distribution and modification of this product forth herein.
“Documentation” means the specifications, manuals and documentation by Silicon Intelligence.
“Licensee” or “you” means you, or your employer or any other person or entity (if you are entering into this Agreement on such person or entity’s behalf), of the age required under applicable laws, rules or regulations to provide legal consent and that has legal authority to bind your employer or such other person or entity if you are entering in this Agreement on their behalf.
“Silicon Intelligence Materials” means, collectively, Silicon Intelligence’s proprietary code and Documentation (and any portion thereof) made available under this Agreement.
By clicking “I Accept” below or by using or distributing any portion or element of the Silicon Intelligence Materials, you agree to be bound by this Agreement.
1. License Rights and Redistribution.
a. Grant of Rights. You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under ’s intellectual property or other rights owned by Silicon Intelligence embodied in the SILICON INTELLIGENCE Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the Silicon Intelligence Materials.
b. Redistribution and Use.
i. If you distribute or make available the Silicon Intelligence Materials (or any derivative works thereof), or a product or service that uses any of them, you shall (A) provide a copy of this Agreement with any such Silicon Intelligence Materials; and (B) prominently display “Built with Silicon Intelligence” on a related website, user interface, blogpost, about page, or product documentation. If you use the Silicon Intelligence Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Silicon Intelligence” at the beginning of any such AI model name.
ii. If you receive Silicon Intelligence Materials, or any derivative works thereof, from a Licensee as part of an integrated end user product, then Section 2 of this Agreement will not apply to you.
iii. You must retain in all copies of the Silicon Intelligence Materials that you distribute the following attribution notice within a “Notice” text file distributed as a part of such copies: “Silicon Intelligence is licensed under the Silicon Intelligence Community License, Copyright © Silicon Intelligence Platforms, Inc. All Rights Reserved.”
iv. Your use of the Silicon Intelligence Materials must comply with applicable laws and regulations (including trade compliance laws and regulations) .
2. Additional Commercial Terms. If, on the Silicon Intelligence duix.ai version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 1 thousand monthly active users in the preceding calendar month, or your product based Silicon Intelligence material your active users greater 1 thousand, you must request a license from Silicon Intelligence, which Silicon Intelligence may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Silicon Intelligence otherwise expressly grants you such rights.
3. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE SILICON INTELLIGENCE MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND SILICON INTELLIGENCE DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE SILICON INTELLIGENCE MATERIALS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE SILICON INTELLIGENCE MATERIALS AND ANY OUTPUT AND RESULTS.
4. Limitation of Liability. IN NO EVENT WILL SILICON INTELLIGENCE OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF SILICON INTELLIGENCE OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
5. Intellectual Property.
a. No trademark licenses are granted under this Agreement, and in connection with the Silicon Intelligence Materials, neither Silicon Intelligence nor Licensee may use any name or mark owned by or associated with the other or any of its affiliates, except as required for reasonable and customary use in describing and redistributing the Silicon Intelligence Materials or as set forth in this Section 5(a). Silicon Intelligence hereby grants you a license to use “Silicon Intelligence” solely as required to comply with the last sentence of Section 1.b.i. You will comply with Silicon Intelligence’s brand guidelines . All goodwill arising out of your use of the Mark will inure to the benefit of Silicon Intelligence.
b. If you institute litigation or other proceedings against Silicon Intelligenceor any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Silicon Intelligence Materials or outputs or results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or claim is filed or instituted. You will indemnify and hold harmless Silicon Intelligence from and against any claim by any third party arising out of or related to your use or distribution of the Silicon Intelligence Materials.
6. Term and Termination. The term of this Agreement will commence upon your acceptance of this Agreement or access to the Silicon Intelligence Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Silicon Intelligence may terminate this Agreement if you are in breach of any term or condition of this Agreement. Upon termination of this Agreement, you shall delete and cease use of the Silicon Intelligence Materials. Sections 3, 4 shall survive the termination of this Agreement.

BIN
res/270p/caishen/caishen.mp4
View File

Binary file not shown.

BIN
res/270p/guanyin/guanyin.mp4
View File

Binary file not shown.

BIN
res/270p/nvyou/nvyou.mp4
View File

Binary file not shown.

BIN
res/270p/sushi/sushi.mp4
View File

Binary file not shown.

BIN
res/540p/laoshi/laoshi.mp4
View File

Binary file not shown.

BIN
res/540p/lvshi/lvshi.mp4
View File

Binary file not shown.

BIN
res/540p/peiban/peiban.mp4
View File

Binary file not shown.

BIN
res/540p/yisheng/yisheng.mp4
View File

Binary file not shown.

BIN
res/avatar/Emma.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 91 KiB

BIN
res/avatar/Kai.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 79 KiB

BIN
res/avatar/Leo.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 104 KiB

BIN
res/avatar/Lily.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 144 KiB

BIN
res/avatar/Oliver.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 102 KiB

BIN
res/avatar/Sofia.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 97 KiB

BIN
res/main_video_thumbnail.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 435 KiB

BIN
res/main_video_thumbnail.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 39 KiB