feat: implement advanced state API for Go and Python

.gitignore

@@ -22,6 +22,8 @@ examples/python-processor/build/**
 examples/python-processor/dependencies/**
 python/functionstream-client/src/fs_client/_proto/
 python/functionstream-api/build
+python/functionstream-api-advanced/build
+
 
 
 # python Runtime - Build artifacts and intermediate files

Makefile

@@ -82,7 +82,7 @@ build-lite: .check-env
 .build-wasm:
 	$(call log,WASM,Building Python Runtime using $(PYTHON_EXEC))
 	@cd $(PYTHON_ROOT)/functionstream-runtime && \
-		PYTHONPATH=../functionstream-api ../../$(PYTHON_EXEC) build.py > /dev/null
+		PYTHONPATH=../functionstream-api:../functionstream-api-advanced ../../$(PYTHON_EXEC) build.py > /dev/null
 	@[ -f "$(WASM_SOURCE)" ] || (printf "$(C_R)[X] WASM Build Failed$(C_0)\n" && exit 1)
 
 dist: build

README-zh.md

@@ -206,7 +206,8 @@ function-stream-<version>/
 | [Function 任务配置规范](docs/function-configuration-zh.md) | 任务定义规范        |
 | [SQL CLI 交互式管理指南](docs/sql-cli-guide-zh.md)          | 交互式管理指南       |
 | [Function 管理与开发指南](docs/function-development-zh.md)  | 管理与开发指南       |
-| [Python SDK 开发与交互指南](docs/python-sdk-guide-zh.md)    | Python SDK 指南 |
+| [Go SDK 开发与交互指南](docs/Go-SDK/go-sdk-guide-zh.md)     | Go SDK 指南        |
+| [Python SDK 开发与交互指南](docs/Python-SDK/python-sdk-guide-zh.md) | Python SDK 指南 |
 
 ## 配置
 

README.md

@@ -205,7 +205,8 @@ We provide a robust shell script to manage the server process, capable of handli
 | [Function Configuration](docs/function-configuration.md) | Task Definition Specification     |
 | [SQL CLI Guide](docs/sql-cli-guide.md)                   | Interactive Management Guide      |
 | [Function Development](docs/function-development.md)     | Management & Development Guide    |
-| [Python SDK Guide](docs/python-sdk-guide.md)             | Python SDK Guide                  |
+| [Go SDK Guide](docs/Go-SDK/go-sdk-guide.md)              | Go SDK Guide                      |
+| [Python SDK Guide](docs/Python-SDK/python-sdk-guide.md)   | Python SDK Guide                  |
 
 ## Configuration
 

docs/go-sdk-guide-zh.md → docs/Go-SDK/go-sdk-guide-zh.md

@@ -263,7 +263,7 @@ create function with (
 );
 ```
 
-config.yaml 中需配置 `name`、`type: processor`、`input-groups`、`outputs`（如 Kafka）。详见 [Function 配置](function-configuration-zh.md) 与 [examples/go-processor/README.md](../examples/go-processor/README.md)。
+config.yaml 中需配置 `name`、`type: processor`、`input-groups`、`outputs`（如 Kafka）。详见 [Function 配置](../function-configuration-zh.md) 与 [examples/go-processor/README.md](../../examples/go-processor/README.md)。
 
 ---
 
@@ -301,7 +301,18 @@ if err != nil {
 
 ---
 
-## 七、目录结构参考
+## 七、高级状态 API（进阶文档）
+
+本指南仅覆盖**低阶 go-sdk**（Driver、Context、Store、目录结构）。**高级状态 API**（Codec、ValueState、ListState、MapState、PriorityQueueState、AggregatingState、ReducingState、Keyed\* 工厂与用法）由独立库 **go-sdk-advanced** 提供，完整说明、Codec 约定、构造函数表与示例均在进阶文档中：
+
+- **[Go SDK — 高级状态 API](go-sdk-advanced-state-api-zh.md)**（中文）
+- [Go SDK — Advanced State API](go-sdk-advanced-state-api.md)（英文）
+
+---
+
+## 八、目录结构参考
+
+**低阶库 go-sdk**：
 
 ```text
 go-sdk/
@@ -317,8 +328,20 @@ go-sdk/
 │   ├── runtime.go
 │   ├── context.go
 │   └── store.go
+├── state/
+│   └── common/       # 公共辅助（Store 类型别名、DupBytes）
 ├── wit/              # processor.wit 及依赖（可由 make wit 生成）
 └── bindings/         # wit-bindgen-go 生成的 Go 代码（make bindings）
 ```
 
-更多示例与 SQL 操作见 [examples/go-processor/README.md](../examples/go-processor/README.md)、[SQL CLI 指南](sql-cli-guide-zh.md)。
+**高阶库 go-sdk-advanced**（依赖 go-sdk，含 Codec 与全部状态类型）：
+
+```text
+go-sdk-advanced/
+├── go.mod            # require go-sdk
+├── codec/            # Codec[T]、DefaultCodecFor、内置与 JSON codec
+├── structures/       # ValueState、ListState、MapState、PriorityQueue、Aggregating、Reducing
+└── keyed/            # Keyed 状态工厂（value、list、map、PQ、aggregating、reducing）
+```
+
+更多示例与 SQL 操作见 [examples/go-processor/README.md](../../examples/go-processor/README.md)、[SQL CLI 指南](../sql-cli-guide-zh.md)。

docs/go-sdk-guide.md → docs/Go-SDK/go-sdk-guide.md

@@ -263,7 +263,7 @@ create function with (
 );
 ```
 
-Configure `name`, `type: processor`, `input-groups`, and `outputs` (e.g. Kafka) in config.yaml. See [Function Configuration](function-configuration.md) and [examples/go-processor/README.md](../examples/go-processor/README.md).
+Configure `name`, `type: processor`, `input-groups`, and `outputs` (e.g. Kafka) in config.yaml. See [Function Configuration](../function-configuration.md) and [examples/go-processor/README.md](../../examples/go-processor/README.md).
 
 ---
 
@@ -301,7 +301,18 @@ if err != nil {
 
 ---
 
-## 7. Directory Layout
+## 7. Advanced State API (see advanced doc)
+
+This guide covers only the **low-level go-sdk** (Driver, Context, Store, directory layout). The **advanced state API** (Codec, ValueState, ListState, MapState, PriorityQueueState, AggregatingState, ReducingState, Keyed\* factories and usage) is provided by a separate library **go-sdk-advanced**. Full reference, codec contract, constructor tables, and examples are in the advanced document:
+
+- **[Go SDK — Advanced State API](go-sdk-advanced-state-api.md)** (English)
+- [Go SDK — 高级状态 API](go-sdk-advanced-state-api-zh.md) (中文)
+
+---
+
+## 8. Directory Layout
+
+**Low-level library (go-sdk):**
 
 ```text
 go-sdk/
@@ -317,8 +328,10 @@ go-sdk/
 │   ├── runtime.go
 │   ├── context.go
 │   └── store.go
+├── state/
+│   └── common/       # Shared helpers (Store type alias, DupBytes)
 ├── wit/              # processor.wit and deps (make wit)
 └── bindings/         # Generated by wit-bindgen-go (make bindings)
 ```
 
-For more examples and SQL operations, see [examples/go-processor/README.md](../examples/go-processor/README.md) and the [SQL CLI Guide](sql-cli-guide.md).
+For more examples and SQL operations, see [examples/go-processor/README.md](../../examples/go-processor/README.md) and the [SQL CLI Guide](../sql-cli-guide.md).

docs/Python-SDK/python-sdk-advanced-state-api-zh.md

@@ -0,0 +1,154 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# Python SDK — 高级状态 API
+
+本文档介绍 Python SDK 的**高级状态 API**：基于底层 KvStore 的带类型状态抽象（ValueState、ListState、MapState 等），通过 **codec** 序列化，并支持按主键的 **keyed state**。
+
+**两个独立库：** 高级状态 API 由 **functionstream-api-advanced** 提供，依赖低阶 **functionstream-api**。安装：`pip install functionstream-api functionstream-api-advanced`。使用时从 `fs_api_advanced` 导入 Codec、ValueState、ListState、MapState 等。
+
+| 库 | 包名 | 内容 |
+|----|------|------|
+| **functionstream-api**（低阶） | `fs_api` | Context（仅 getOrCreateKVStore、getConfig、emit）、KvStore、KvIterator、ComplexKey、错误类。 |
+| **functionstream-api-advanced**（高阶） | `fs_api_advanced` | Codec、ValueState、ListState、MapState、PriorityQueueState、AggregatingState、ReducingState、Keyed\* 工厂与状态类型。 |
+
+---
+
+## 1. 概述
+
+当需要结构化状态（单值、列表、Map、优先队列、聚合、归约）而不想手写字节编码或 key 布局时，可使用高级状态 API。创建方式有两种：通过**运行时的 Context**（如使用 functionstream-runtime 时 `ctx.getOrCreateValueState(...)`）或通过状态类型上的**类型级构造方法**（推荐，便于复用）。
+
+---
+
+## 2. 创建状态的两种方式
+
+### 2.1 通过 Context（getOrCreate\*）
+
+使用 **functionstream-api-advanced** 时，运行时的 Context 实现（如 functionstream-runtime 的 WitContext）会提供 `getOrCreateValueState(store_name, codec)`、`getOrCreateValueStateAutoCodec(store_name)` 以及 ListState、MapState、PriorityQueueState、AggregatingState、ReducingState 与所有 Keyed\* 工厂的对应方法，内部委托给下面所述的类型级 `from_context` / `from_context_auto_codec`。
+
+### 2.2 通过状态类型（推荐）
+
+每种状态类型和 keyed 工厂提供：
+
+- **带 codec：** `XxxState.from_context(ctx, store_name, codec, ...)`
+- **AutoCodec：** `XxxState.from_context_auto_codec(ctx, store_name)` 或带可选类型参数，由 SDK 使用默认 codec（如 PickleCodec，或 Map key / PQ 元素所需的有序 codec）。
+
+状态实例是轻量的；可在每次 `process` 中创建，或在 driver 中（如 `init`）缓存。同一 store 名称对应同一底层 store。
+
+---
+
+## 3. 非 Keyed 状态 — 构造方法一览
+
+| 状态类型               | 带 codec                                                                                                                                 | AutoCodec                                                             |
+|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------|
+| ValueState         | `ValueState.from_context(ctx, store_name, codec)`                                                                                       | `ValueState.from_context_auto_codec(ctx, store_name)`                 |
+| ListState          | `ListState.from_context(ctx, store_name, codec)`                                                                                        | `ListState.from_context_auto_codec(ctx, store_name)`                  |
+| MapState           | `MapState.from_context(ctx, store_name, key_codec, value_codec)` 或 `MapState.from_context_auto_key_codec(ctx, store_name, value_codec)` | —                                                                     |
+| PriorityQueueState | `PriorityQueueState.from_context(ctx, store_name, codec)`                                                                               | `PriorityQueueState.from_context_auto_codec(ctx, store_name)`         |
+| AggregatingState   | `AggregatingState.from_context(ctx, store_name, acc_codec, agg_func)`                                                                   | `AggregatingState.from_context_auto_codec(ctx, store_name, agg_func)` |
+| ReducingState      | `ReducingState.from_context(ctx, store_name, value_codec, reduce_func)`                                                                 | `ReducingState.from_context_auto_codec(ctx, store_name, reduce_func)` |
+
+以上均可通过 Context 的 `ctx.getOrCreate*` 方法获得（如 `ctx.getOrCreateValueState(store_name, codec)`），其内部会委托给上述构造方法。
+
+---
+
+## 4. Keyed 状态 — 工厂与 key_group / key / namespace
+
+**Keyed 状态面向 keyed 算子。** 流按 key 分区（如 keyBy）时，每个 key 拥有独立状态。可先获取一次**工厂**（通过 context、store 名称、**namespace** 和 **key_group**），再按**主键**（当前记录的流 key）创建状态。
+
+### 4.1 key_group、key（主键）与 namespace
+
+| 概念            | API 参数                                   | 含义                                                      |
+|---------------|------------------------------------------|---------------------------------------------------------|
+| **key_group** | 创建工厂时的 `key_group`                       | **keyed 组**：标识该状态所属分区/组（如一组 “counters”，另一组 “sessions”）。 |
+| **key**       | 工厂方法参数（如 `new_keyed_value(primary_key)`） | 当前记录的**流 key 的值**（如用户 ID、分区 key）。不同 key 对应不同状态。         |
+| **namespace** | 创建工厂时的 `namespace`（bytes）                | **有窗口时**为**窗口标识的 bytes**；**无窗口时**传**空 bytes**（如 `b""`）。 |
+
+### 4.2 Keyed 工厂构造方法一览
+
+| 工厂                             | 带 codec                                                                                                   | AutoCodec                                                                                                                |
+|--------------------------------|-----------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------|
+| KeyedValueStateFactory         | `KeyedValueStateFactory.from_context(ctx, store_name, namespace, key_group, value_codec)`                 | `KeyedValueStateFactory.from_context_auto_codec(ctx, store_name, namespace, key_group, value_type=None)`                 |
+| KeyedListStateFactory          | `KeyedListStateFactory.from_context(ctx, store_name, namespace, key_group, value_codec)`                  | `KeyedListStateFactory.from_context_auto_codec(ctx, store_name, namespace, key_group, value_type=None)`                  |
+| KeyedMapStateFactory           | `KeyedMapStateFactory.from_context(ctx, store_name, namespace, key_group, key_codec, value_codec)`        | `KeyedMapStateFactory.from_context_auto_codec(ctx, store_name, namespace, key_group, value_codec)`                       |
+| KeyedPriorityQueueStateFactory | `KeyedPriorityQueueStateFactory.from_context(ctx, store_name, namespace, key_group, item_codec)`          | `KeyedPriorityQueueStateFactory.from_context_auto_codec(ctx, store_name, namespace, key_group, item_type=None)`          |
+| KeyedAggregatingStateFactory   | `KeyedAggregatingStateFactory.from_context(ctx, store_name, namespace, key_group, acc_codec, agg_func)`   | `KeyedAggregatingStateFactory.from_context_auto_codec(ctx, store_name, namespace, key_group, agg_func, acc_type=None)`   |
+| KeyedReducingStateFactory      | `KeyedReducingStateFactory.from_context(ctx, store_name, namespace, key_group, value_codec, reduce_func)` | `KeyedReducingStateFactory.from_context_auto_codec(ctx, store_name, namespace, key_group, reduce_func, value_type=None)` |
+
+也可使用 Context 的 `ctx.getOrCreateKeyed*Factory(...)` 方法，其内部会委托给上述构造方法。
+
+### 4.3 KeyedValueState
+
+KeyedValueState 只需 **value codec**，不要求有序。工厂创建状态：`factory.new_keyed_value(primary_key, state_name="")`，得到 `KeyedValueState[V]`。状态方法：`update(value)`、`value()`（返回 `Optional[V]`）、`clear()`。主键由创建时传入的 `primary_key`（bytes）固定。
+
+---
+
+## 5. 示例
+
+### 5.1 ValueState（from_context_auto_codec）
+
+从 **fs_api_advanced** 导入 ValueState（Codec、ListState、MapState 等同此包）：
+
+```python
+from fs_api import FSProcessorDriver, Context
+from fs_api_advanced import ValueState
+
+class CounterProcessor(FSProcessorDriver):
+    def process(self, ctx: Context, source_id: int, data: bytes):
+        state = ValueState.from_context_auto_codec(ctx, "my-store")
+        cur = state.value()
+        if cur is None:
+            cur = 0
+        state.update(cur + 1)
+        ctx.emit(str(cur + 1).encode(), 0)
+```
+
+### 5.2 KeyedValueState（keyed 算子）
+
+流按 key 分区时，在 `init` 中创建工厂，在 `process` 中按当前记录的 `primary_key` 取状态，再 `update(value)` / `value()` / `clear()`：
+
+```python
+from fs_api import FSProcessorDriver, Context
+from fs_api_advanced import KeyedValueStateFactory
+
+class KeyedCounterProcessor(FSProcessorDriver):
+    def init(self, ctx: Context, config: dict):
+        self._factory = KeyedValueStateFactory.from_context_auto_codec(
+            ctx, "counters", b"", b"by_key", value_type=int
+        )
+
+    def process(self, ctx: Context, source_id: int, data: bytes):
+        primary_key = data[:8]
+        state = self._factory.new_keyed_value(primary_key, "count")
+        cur = state.value()
+        if cur is None:
+            cur = 0
+        state.update(cur + 1)
+        ctx.emit(str(cur + 1).encode(), 0)
+```
+
+其他状态类型按上表使用 `XxxState.from_context(ctx, store_name, ...)` 或 `XxxState.from_context_auto_codec(ctx, store_name)`。
+
+---
+
+## 6. 参见
+
+- [Python SDK 指南](python-sdk-guide-zh.md) — fs_api、fs_client 及 Context/KvStore 基础用法。