Knowledge Graph 驱动的结构化上下文处理:Context-Engineering 从图表示到多策略推理的完整实践

原创2026-10-07 13:20:501,660 阅读
文章标签:文档教程人工智能提示工程

Knowledge Graph 驱动的结构化上下文处理:Context-Engineering 从图表示到多策略推理的完整实践

结构化上下文处理(Structured Context Processing)是 Context Engineering 课程中"上下文处理"模块(Module 02.4)的核心内容之一,也是从线性文本处理走向关系感知智能的关键一跃。本篇文章以 04_structured_context.md 为主体骨架,结合仓库内配套实验室 structured_data_lab.py 与图增强 RAG 模块 03_graph_enhanced_rag.md 的源码级细节,系统讲解知识图谱如何进入上下文组装:从实体-关系建模、层次信息编码、多跳路径推理的数学基础,到"Prompt 推理模板"与"Programming 上下文引擎"两套 Software 3.0 落地范式。读完本文,你将能够亲手搭建一个具备图遍历、相关性子图提取、演绎/归纳/溯因/类比四类推理策略的完整知识图谱上下文引擎,并理解它与 GraphRAG、Schema 校验、图神经网络嵌入等生产级能力的衔接方式。


模块定位与学习目标

在 00_overview.md 所描绘的上下文处理流水线中,输入侧包含"Structured / Relational"(结构化与关系型)信息源,而结构化上下文处理正是负责"整合关系数据、知识图谱与层次化信息,同时保持结构语义不丢失"的关键环节。本模块的学习目标可以概括为四点:

  • 基于图(Graph)的上下文表示:把复杂的关联关系建模为相互连通的知识结构,而非孤立的文本片段;
  • 关系推理系统(Relational Reasoning):理解实体与关系如何共同创造语义;
  • 知识图谱集成(Knowledge Graph Integration):将结构化知识融入上下文组装过程;
  • 层次化信息组织(Hierarchical Organization):用嵌套与递归的数据结构管理上下文,以获得最优的信息密度。

换言之,本模块回答的核心问题是:当上下文不再是一串 token,而是一张由实体与关系构成的网络时,如何围绕一个查询(query)把这网络中最有价值的部分"裁剪"出来、组装成可供大模型消费的结构化上下文?


概念演进:从线性文本到网络智能的五阶段进阶

文档用一个贯穿始终的隐喻展开:理解结构化上下文,就像区分"按字母顺序阅读词典"(线性)与"理解一个活的生态系统"(网络化、关系化、相互依存)之间的差异。整个能力演化被划分为五个递进阶段。

Stage 1:线性信息处理(Linear Information Processing)

线性处理把信息视为一串前后相接的事实,例如:

Text: "Alice works at Google. Google is a tech company. Tech companies develop software."

Processing: Alice → works_at → Google → is_a → tech_company → develops → software

Understanding: Sequential, limited connections

就像逐条阅读教科书中的事实,你获得了信息,却丢失了构成深层理解的关系之网。这一阶段的局限包括:

  • 信息被孤立地处理;
  • 关系没有被显式建模;
  • 难以对连接进行推理;
  • 缺乏层次化组织。

Stage 2:简单实体-关系识别(Entity-Relationship Recognition)

第二阶段开始显式提取实体与关系:

Entities: [Alice, Google, tech_company, software]
Relationships: [works_at(Alice, Google), is_a(Google, tech_company), develops(tech_company, software)]

Basic Graph:
Alice --works_at--> Google --is_a--> tech_company --develops--> software

这相当于画出一张组织架构图或家族树:能看见直接连接,也能回答简单的单跳关系查询,但复杂模式仍然隐藏。遗留问题包括:关系结构扁平、没有推理能力、上下文传播受限。

Stage 3:知识图谱集成(Knowledge Graph Integration)

第三阶段引入富知识图谱——实体携带属性,关系可以多跳延伸,并支持分类继承:

Rich Knowledge Graph:

    Alice (Person)
      ├─ works_at → Google (Company)
      ├─ skills → [Programming, AI]
      └─ location → Mountain_View

    Google (Company)
      ├─ is_a → Tech_Company
      ├─ founded → 1998
      ├─ headquarters → Mountain_View
      ├─ develops → [Search, Android, AI]
      ├─ employees → 150000
      └─ competes_with → [Apple, Microsoft]

    Tech_Company (Category)
      ├─ characteristics → [Innovation, Software, Digital]
      └─ examples → [Google, Apple, Microsoft]

这一阶段具备的能力:

  • 跨关系的多跳推理(multi-hop reasoning);
  • 层次分类与属性继承;
  • 通过图遍历实现上下文富集;
  • 支持复杂查询与推理。

Stage 4:动态层次化上下文网络(Dynamic Hierarchical Context Networks)

第四阶段把不同抽象层级组织成一张层次化上下文网络,例如从"行业域 → 公司层 → 个人层 → 任务层"逐级嵌套,并在层级之间建立交叉连接:

┌─────────────────────────────────────────────────────────────────┐
│                HIERARCHICAL CONTEXT NETWORK                     │
│                                                                 │
│  Domain Level: Technology Industry                              │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │  Company Level: Google                                  │   │
│  │  ├─ Business Model: Advertising, Cloud, Hardware       │   │
│  │  ├─ Core Technologies: AI, Search, Mobile              │   │
│  │  └─ Market Position: Leader in Search, Growing in AI   │   │
│  │    Individual Level: Alice                              │   │
│  │    ├─ Role Context: AI Researcher                      │   │
│  │    ├─ Skill Context: Machine Learning, Python          │   │
│  │    └─ Project Context: Large Language Models           │   │
│  │      Task Level: Current Assignment                     │   │
│  │      ├─ Objective: Improve Model Safety               │   │
│  │      ├─ Methods: Constitutional AI, RLHF               │   │
│  │      └─ Timeline: Q3-Q4 2024                          │   │
│  └─────────────────────────────────────────────────────────┘   │
│                                                                 │
│  Cross-Level Connections:                                       │
│  • Industry trends influence company strategy                   │
│  • Company resources enable individual projects               │
│  • Individual expertise shapes project approaches             │
│  • Project outcomes affect company positioning                │
└─────────────────────────────────────────────────────────────────┘

其意义相当于拥有一位"总策略师":同时理解个人行为如何连接团队动态、组织目标与行业趋势。跨层连接(cross-level connections)正是层次化上下文相对扁平图谱的核心增值点。

Stage 5:自适应图智能与涌现结构发现(Adaptive Graph Intelligence)

第五阶段是自组织、自改进的图智能系统,包含五个子系统:

  • 模式识别引擎(Pattern Recognition Engine):发现数据中的隐式关系、识别重复出现的结构模式、学习最优的图组织策略;
  • 涌现结构形成(Emergent Structure Formation):创造原始数据中不存在的新关系类型、在关系模式之间形成元关系、自动发展层次化抽象;
  • 动态上下文适配(Dynamic Context Adaptation):根据查询模式重构图谱、为不同推理类型优化信息路径、依据使用与反馈演化表示;
  • 实时推理(Real-time Inference and Reasoning):跨复杂关系链的多跳推理、相似图模式之间的类比推理、从结构关系进行因果推断、关于关系演化的时序推理;
  • 自改进机制(Self-Improvement Mechanisms):学习更好的图构建策略、改进关系提取与分类、依据结果增强推理算法、为计算效率优化结构。

这五个阶段构成了完整的"线性 → 关系感知 → 图谱化 → 层次化 → 自适应"演进脉络,也为后文的数学建模与工程实现提供了问题域。


数学基础:为图上下文建立可计算的骨架

图表示与上下文组装(Graph-Based Context Representation)

知识图谱可以形式化为三元组集合:

Knowledge Graph: G = (E, R, T)
Where:
- E = set of entities {e₁, e₂, ..., eₙ}
- R = set of relation types {r₁, r₂, ..., rₖ}
- T = set of triples {(eᵢ, rⱼ, eₖ)} representing facts

Context Assembly from Graph:
C(q, G) = TraversePath(q, G, depth=d, strategy=s)

Where:
- q = query or information need
- G = knowledge graph
- d = maximum traversal depth
- s = traversal strategy (BFS, DFS, relevance-guided)

直觉解释:知识图谱就像一张信息地图,实体是地点、关系是路径,上下文组装则变成一个导航问题——在知识网络中寻找从查询到答案的最相关路径。这里 d 与 s 两个参数直接对应后文代码中的 max_depth 与各种推理/遍历策略(BFS、DFS 或相关性引导)。

层次化信息编码(Hierarchical Information Encoding)

层次化上下文树可用节点、父子关系与交叉链接描述,并通过传播公式让信息在层级间流动:

Hierarchical Context Tree: H = (N, P, C)
Where:
- N = set of nodes representing information units
- P = parent-child relationships (taxonomic structure)
- C = cross-links (associative relationships)

Information Propagation:
I(n) = Local(n) + α·∑ᵢ Parent(i)·w(i→n) + β·∑ⱼ Child(j)·w(n→j) + γ·∑ₖ CrossLink(k)·w(n↔k)

Where:
- Local(n) = information directly at node n
- α, β, γ = propagation weights for different relationship types
- w(·) = relationship strength weights

直觉解释:层次结构中的信息不只存在于单个节点——概念从父级(所属类别)、子级(具体实例)与交叉链接(相关概念)继承含义。就像理解"dog"需要"animal"(父)、"golden retriever"(子)与"companion"(交叉链接)共同参与。α、β、γ 三个传播权重即工程实现中"相关性打分"的数学原型。

关系推理优化(Relational Reasoning Optimization)

多跳路径推理被建模为在图中寻找最可能路径的概率问题:

Multi-Hop Path Reasoning:
P(answer | query, graph) = ∑ paths π P(answer | π) · P(π | query, graph)

Where a path π = (e₀, r₁, e₁, r₂, e₂, ..., rₙ, eₙ)

Path Probability:
P(π | query, graph) = ∏ᵢ P(rᵢ₊₁ | eᵢ, query) · P(eᵢ₊₁ | eᵢ, rᵢ₊₁, query)

Optimized Traversal:
π* = argmax_π P(π | query, graph) subject to |π| ≤ max_hops

直觉解释:在图谱上推理时,从问题到答案存在许多可能路径,我们希望找到连接查询与相关信息的最可能路径——同时考虑每条关系的似然度与整条路径的连贯性。这条公式正是后文 KnowledgeGraph.find_paths 中"按路径得分排序、返回 Top 10 路径"的数学依据(04_structured_context.md 的 find_paths 实现部分)。


Software 3.0 范式一:Prompts——知识图谱推理模板

文档把"提示词"视为 Software 3.0 的第一个落地范式:通过结构化的推理模板,引导模型进行系统的图遍历与关系分析。以下是完整模板(占位符 {...} 需在运行时填充):

# Knowledge Graph Reasoning Framework

## Graph Context Analysis
You are reasoning through structured information represented as a knowledge graph. Use systematic traversal and relationship analysis to build comprehensive understanding.

## Graph Structure Assessment
**Available Entities**: {entities_in_current_graph}
**Relationship Types**: {relation_types_and_their_meanings}
**Graph Depth**: {maximum_relationship_chain_length}
**Query Context**: {specific_question_or_reasoning_goal}

### Entity Analysis
For each relevant entity in the reasoning path:

**Entity**: {entity_name}
- **Type/Category**: {entity_classification}
- **Direct Properties**: {attributes_directly_associated_with_entity}
- **Outgoing Relationships**: {relationships_where_entity_is_subject}
- **Incoming Relationships**: {relationships_where_entity_is_object}
- **Hierarchical Context**: {parent_and_child_entities_in_taxonomy}

### Relationship Chain Construction

#### Single-Hop Reasoning
**Direct Connections**: {entity1} --{relationship}--> {entity2}
- **Relationship Strength**: {confidence_or_weight_of_relationship}
- **Context Relevance**: {how_relevant_to_current_query}
- **Information Content**: {what_this_relationship_tells_us}

#### Multi-Hop Reasoning Paths
**Path 1**: {entity1} --{rel1}--> {entity2} --{rel2}--> {entity3} --{rel3}--> {target}
- **Path Coherence**: {how_logically_consistent_is_this_chain}
- **Cumulative Evidence**: {strength_of_evidence_along_path}
- **Alternative Interpretations**: {other_ways_to_understand_this_path}

**Path 2**: {alternative_reasoning_path}
**Path 3**: {additional_reasoning_path_if_relevant}

### Reasoning Strategy Selection

#### Bottom-Up Reasoning (From Specific to General)

IF query_requires_generalization: START WITH specific_instances IDENTIFY common_patterns_and_properties TRAVERSE upward_through_hierarchical_relationships SYNTHESIZE general_principles_or_categories


#### Top-Down Reasoning (From General to Specific)

IF query_requires_specific_information: START WITH general_categories_or_principles TRAVERSE downward_through_specialization_relationships IDENTIFY relevant_specific_instances EXTRACT detailed_information_about_instances


#### Lateral Reasoning (Across Same Level)

IF query_requires_comparison_or_analogy: IDENTIFY entities_at_similar_hierarchical_levels TRAVERSE cross_links_and_associative_relationships COMPARE properties_and_relationship_patterns IDENTIFY similarities_and_differences


### Hierarchical Context Integration

#### Local Context (Immediate Neighborhood)
- **Direct Properties**: {properties_of_focus_entity}
- **Immediate Relations**: {one_hop_relationships}
- **Local Constraints**: {rules_or_constraints_in_immediate_context}

#### Intermediate Context (2-3 Hops)
- **Extended Relationships**: {multi_hop_connections}
- **Pattern Recognition**: {recurring_structures_in_extended_neighborhood}
- **Contextual Modifiers**: {how_intermediate_context_affects_interpretation}

#### Global Context (Full Graph Perspective)
- **Domain-Level Patterns**: {large_scale_structures_and_patterns}
- **Cross-Domain Connections**: {relationships_spanning_different_knowledge_areas}
- **System-Level Constraints**: {global_rules_or_principles}

### Inference and Reasoning Execution

#### Deductive Reasoning
**Given Facts**: {explicit_relationships_and_properties_in_graph}
**Logical Rules**: {if_then_rules_that_can_be_applied}
**Conclusions**: {what_can_be_logically_derived}

Example:

IF Alice works_at Google AND Google is_a Tech_Company THEN Alice works_at a Tech_Company (transitivity of employment and classification)


#### Inductive Reasoning
**Observed Patterns**: {recurring_structures_or_relationships_in_graph}
**Generalized Rules**: {patterns_that_might_apply_more_broadly}
**Confidence Levels**: {how_certain_are_we_about_these_generalizations}

#### Abductive Reasoning (Best Explanation)
**Observed Evidence**: {facts_that_need_explanation}
**Candidate Explanations**: {possible_reasons_for_observed_evidence}
**Best Explanation**: {most_likely_explanation_given_graph_structure}

### Context Assembly Strategy

#### Query-Driven Assembly
1. **Parse Query**: Identify key entities and relationships mentioned
2. **Seed Selection**: Choose starting points in the graph
3. **Expansion Strategy**: Decide how to grow context from seeds
4. **Relevance Filtering**: Keep most relevant information, prune irrelevant
5. **Coherence Verification**: Ensure assembled context forms coherent narrative

#### Structure-Driven Assembly
1. **Identify Key Structures**: Find important subgraphs or patterns
2. **Extract Hierarchies**: Build taxonomic and part-whole relationships
3. **Map Cross-Links**: Include important associative relationships
4. **Context Layering**: Organize information by levels of abstraction
5. **Integration Synthesis**: Combine different structural views

### Quality Assessment

#### Completeness Check
- **Required Information Coverage**: {percentage_of_necessary_information_included}
- **Key Relationship Coverage**: {important_relationships_represented}
- **Hierarchical Completeness**: {coverage_across_different_abstraction_levels}

#### Coherence Verification
- **Logical Consistency**: {absence_of_contradictions_in_assembled_context}
- **Relationship Validity**: {all_relationships_are_meaningful_and_correct}
- **Narrative Flow**: {information_flows_logically_from_premise_to_conclusion}

#### Relevance Optimization
- **Query Alignment**: {how_well_context_addresses_original_query}
- **Information Density**: {ratio_of_useful_to_total_information}
- **Focus Appropriateness**: {correct_level_of_detail_for_query_type}

## Structured Context Output

**Primary Reasoning Path**: {most_confident_reasoning_chain}
**Supporting Evidence**: {additional_relationships_that_support_conclusion}
**Alternative Interpretations**: {other_possible_ways_to_understand_the_information}
**Uncertainty Factors**: {areas_where_reasoning_confidence_is_lower}

**Hierarchical Summary**:
- **High-Level Concepts**: {general_categories_and_principles}
- **Mid-Level Relationships**: {specific_connections_and_patterns}
- **Detailed Facts**: {specific_properties_and_instances}

**Cross-References**: {related_information_that_provides_additional_context}

通俗解读(Ground-up Explanation):这个模板像一位通过线索网络查案的侦探——不仅看单个证据,还绘制证据间的连接、逐条构建推理链,并在得出结论前考虑多种可能的解释。模板中三个要点值得特别注意:

  1. 实体分析的双向视角:同时枚举出边(outgoing)与入边(incoming)关系,避免只沿着单一方向理解实体;
  2. 推理策略选择是条件化的:自底向上(归纳泛化)、自顶向下(细化特例)、横向(比较类比)由查询类型决定——这与后文代码中 _analyze_query 对查询类型的判定逻辑一一对应;
  3. 质量评估三维度:完整性(Completeness)、连贯性(Coherence)、相关性(Relevance),构成了组装后上下文的验收标准。

模板中的"Query-Driven Assembly"五步(解析查询 → 种子选择 → 扩展策略 → 相关性过滤 → 连贯性验证)直接对应后文编程范式里 StructuredContextAssembler.assemble_context 的执行管线。模板还可在 GraphRAG 场景复用:仓库的图增强 RAG 模块 03_graph_enhanced_rag.md 中提供了 GRAPH_QUERY_TEMPLATE、MULTI_HOP_REASONING_TEMPLATE、SEMANTIC_GRAPH_INTELLIGENCE_TEMPLATE 等渐进式复杂度的模板族,可作为本模板在生产检索链路中的延伸。


Software 3.0 范式二:Programming——知识图谱上下文引擎

如果说 Prompt 范式解决"如何让模型按图结构思考",Programming 范式则解决"如何把图本身建出来并完成推理"。原文档给出了一个完整、可运行的 Python 实现(依赖 numpy、networkx、dataclasses、abc、collections、enum、json),下面按模块逐层拆解并给出运行方式。

核心数据结构:关系类型、实体、关系、推理路径

class RelationType(Enum):
    """Types of relationships in knowledge graph"""
    IS_A = "is_a"
    PART_OF = "part_of"
    RELATED_TO = "related_to"
    INSTANCE_OF = "instance_of"
    HAS_PROPERTY = "has_property"
    WORKS_AT = "works_at"
    LOCATED_IN = "located_in"
    CAUSES = "causes"
    ENABLES = "enables"
    SIMILAR_TO = "similar_to"

@dataclass
class Entity:
    """Knowledge graph entity with properties"""
    id: str
    name: str
    entity_type: str
    properties: Dict[str, Any] = field(default_factory=dict)
    embeddings: Optional[np.ndarray] = None
    confidence: float = 1.0

@dataclass
class Relationship:
    """Knowledge graph relationship"""
    subject: str
    predicate: RelationType
    object: str
    weight: float = 1.0
    confidence: float = 1.0
    metadata: Dict[str, Any] = field(default_factory=dict)

@dataclass
class ReasoningPath:
    """Path through knowledge graph for reasoning"""
    entities: List[str]
    relationships: List[Relationship]
    path_score: float
    reasoning_type: str
    evidence_strength: float

设计要点:RelationType 枚举集中管理十种关系语义;Entity.confidence 与 Relationship.confidence 双置信度贯穿全链路——实体与关系都携带置信度,使得路径得分可以沿边累积(见下文 _convert_to_reasoning_path 中 path_score *= rel.confidence)。

KnowledgeGraph:图谱的表示与基础操作

class KnowledgeGraph:
    """Core knowledge graph representation and operations"""

    def __init__(self):
        self.entities: Dict[str, Entity] = {}
        self.relationships: List[Relationship] = []
        self.graph = nx.MultiDiGraph()
        self.entity_types: Dict[str, Set[str]] = defaultdict(set)
        self.relation_index: Dict[RelationType, List[Relationship]] = defaultdict(list)

    def add_entity(self, entity: Entity):
        """Add entity to knowledge graph"""
        self.entities[entity.id] = entity
        self.graph.add_node(entity.id, **entity.properties)
        self.entity_types[entity.entity_type].add(entity.id)

    def add_relationship(self, relationship: Relationship):
        """Add relationship to knowledge graph"""
        self.relationships.append(relationship)
        self.graph.add_edge(
            relationship.subject,
            relationship.object,
            predicate=relationship.predicate,
            weight=relationship.weight,
            confidence=relationship.confidence
        )
        self.relation_index[relationship.predicate].append(relationship)

底层选择 networkx.MultiDiGraph(多重有向图),支持同节点对之间存在多条带不同谓词的边。同时维护三个辅助索引:entity_types(类型 → 实体集合)、relation_index(谓词 → 关系列表),分别服务于类型过滤与按关系类型检索。

邻居获取(get_neighbors)支持方向参数 direction ∈ {"outgoing", "incoming", "both"},并可选按 relation_type 过滤;实现上遍历 graph.successors / graph.predecessors 及其多重边,动态重建 Relationship 对象。

路径查找(find_paths)是推理的核心入口:

    def find_paths(self, start_entity: str, end_entity: str,
                   max_depth: int = 3) -> List[ReasoningPath]:
        """Find reasoning paths between two entities"""
        paths = []

        try:
            # Find all simple paths up to max_depth
            nx_paths = nx.all_simple_paths(self.graph, start_entity, end_entity, cutoff=max_depth)

            for path in nx_paths:
                reasoning_path = self._convert_to_reasoning_path(path)
                if reasoning_path:
                    paths.append(reasoning_path)

        except nx.NetworkXNoPath:
            pass  # No path exists

        # Sort by path score
        paths.sort(key=lambda p: p.path_score, reverse=True)
        return paths[:10]  # Return top 10 paths

    def _convert_to_reasoning_path(self, node_path: List[str]) -> Optional[ReasoningPath]:
        """Convert networkx path to reasoning path"""
        if len(node_path) < 2:
            return None

        relationships = []
        path_score = 1.0

        for i in range(len(node_path) - 1):
            source, target = node_path[i], node_path[i + 1]

            # Find the relationship between these nodes
            edges = self.graph[source][target]
            if not edges:
                return None

            # Take the edge with highest confidence
            best_edge = max(edges.values(), key=lambda e: e['confidence'])

            rel = Relationship(
                subject=source,
                predicate=best_edge['predicate'],
                object=target,
                weight=best_edge['weight'],
                confidence=best_edge['confidence']
            )
            relationships.append(rel)

            # Update path score based on relationship confidence
            path_score *= rel.confidence

        return ReasoningPath(
            entities=node_path,
            relationships=relationships,
            path_score=path_score,
            reasoning_type="multi_hop",
            evidence_strength=path_score
        )

要点:用 nx.all_simple_paths 枚举给定 max_depth(默认 3)内的所有简单路径;多重边存在时取置信度最高的那条;路径得分是各边置信度的乘积,这与前述数学基础中 P(π | query, graph) = ∏ᵢ ... 的连乘语义完全吻合;最终按得分降序返回 Top 10。

实体上下文提取(get_entity_context)返回四部分信息,直接对应用户提问时需要的"三层上下文":

    def get_entity_context(self, entity_id: str, depth: int = 2) -> Dict[str, Any]:
        """Get rich context for an entity including neighbors at specified depth"""
        if entity_id not in self.entities:
            return {}

        context = {
            'entity': self.entities[entity_id],
            'immediate_neighbors': {},
            'extended_context': {},
            'hierarchical_context': {}
        }

        # Get immediate neighbors (depth 1)
        immediate = self.get_neighbors(entity_id, direction="both")
        context['immediate_neighbors'] = {
            'outgoing': [(target, rel) for target, rel in immediate if rel.subject == entity_id],
            'incoming': [(source, rel) for source, rel in immediate if rel.object == entity_id]
        }

        # Get extended context (depth 2+)
        if depth > 1:
            extended_entities = set()
            queue = deque([(entity_id, 0)])
            visited = {entity_id}

            while queue:
                current_entity, current_depth = queue.popleft()

                if current_depth >= depth:
                    continue

                neighbors = self.get_neighbors(current_entity, direction="both")
                for neighbor_id, rel in neighbors:
                    if neighbor_id not in visited:
                        extended_entities.add(neighbor_id)
                        visited.add(neighbor_id)
                        queue.append((neighbor_id, current_depth + 1))

            context['extended_context'] = {
                eid: self.entities[eid] for eid in extended_entities if eid in self.entities
            }

        # Get hierarchical context (is_a relationships)
        hierarchical = self._get_hierarchical_context(entity_id)
        context['hierarchical_context'] = hierarchical

        return context

    def _get_hierarchical_context(self, entity_id: str) -> Dict[str, List[str]]:
        """Get hierarchical context (parents and children in taxonomy)"""
        parents = []
        children = []

        # Find parents (things this entity is_a instance of)
        parent_rels = self.get_neighbors(entity_id, RelationType.IS_A, "outgoing")
        parents.extend([target for target, _ in parent_rels])

        instance_rels = self.get_neighbors(entity_id, RelationType.INSTANCE_OF, "outgoing")
        parents.extend([target for target, _ in instance_rels])

        # Find children (things that are instances of this entity)
        child_rels = self.get_neighbors(entity_id, RelationType.IS_A, "incoming")
        children.extend([source for source, _ in child_rels])

        instance_child_rels = self.get_neighbors(entity_id, RelationType.INSTANCE_OF, "incoming")
        children.extend([source for source, _ in instance_child_rels])

        return {
            'parents': parents,
            'children': children
        }

其中 immediate_neighbors(一跳邻居)与 extended_context(BFS 到指定深度)共同构成"局部 + 扩展"上下文;_get_hierarchical_context 通过 IS_A / INSTANCE_OF 两个谓词的出入方向确定实体在分类学中的父/子位置,正是文档"Hierarchical Context Integration"三层的代码实现。

StructuredContextAssembler:查询驱动的上下文组装与多策略推理

组装器是整套系统的"大脑",注册了四种推理策略,并围绕查询分析、子图提取、相关性打分/过滤、结果整合组织主流程:

class StructuredContextAssembler:
    """Assembles context from structured knowledge representations"""

    def __init__(self, knowledge_graph: KnowledgeGraph):
        self.kg = knowledge_graph
        self.reasoning_strategies = {
            'deductive': self._deductive_reasoning,
            'inductive': self._inductive_reasoning,
            'abductive': self._abductive_reasoning,
            'analogical': self._analogical_reasoning
        }

    def assemble_context(self, query: str, entities: List[str],
                        max_context_size: int = 2000,
                        reasoning_strategy: str = "deductive") -> Dict[str, Any]:
        """Main context assembly process"""

        print(f"Assembling structured context for query: {query}")
        print(f"Starting entities: {entities}")

        # Extract key information from query
        query_analysis = self._analyze_query(query)

        # Collect relevant subgraphs around seed entities
        relevant_subgraphs = []
        for entity_id in entities:
            if entity_id in self.kg.entities:
                subgraph = self._extract_relevant_subgraph(entity_id, query_analysis, depth=3)
                relevant_subgraphs.append(subgraph)

        # Apply reasoning strategy
        reasoning_results = self.reasoning_strategiesreasoning_strategy

        # Assemble final context
        assembled_context = self._integrate_reasoning_results(
            query, query_analysis, reasoning_results, max_context_size
        )

        return assembled_context

assemble_context 的四个可调参数分别是:query(用户问题)、entities(种子实体列表)、max_context_size(最终上下文最大字符数,默认 2000)、reasoning_strategy(推理策略,默认 deductive,可选 inductive / abductive / analogical)。

查询分析(_analyze_query)通过关键词启发式判定查询类型,并据此推断需要的关系类型:

    def _analyze_query(self, query: str) -> Dict[str, Any]:
        """Analyze query to understand information needs"""
        query_lower = query.lower()

        analysis = {
            'query_text': query,
            'query_type': 'factual',  # Default
            'entities_mentioned': [],
            'relationships_implied': [],
            'reasoning_depth': 'shallow',
            'answer_type': 'descriptive'
        }

        # Determine query type
        if any(word in query_lower for word in ['why', 'because', 'cause', 'reason']):
            analysis['query_type'] = 'causal'
            analysis['reasoning_depth'] = 'deep'
        elif any(word in query_lower for word in ['how', 'process', 'method', 'way']):
            analysis['query_type'] = 'procedural'
        elif any(word in query_lower for word in ['compare', 'difference', 'similar', 'versus']):
            analysis['query_type'] = 'comparative'
            analysis['reasoning_depth'] = 'medium'
        elif any(word in query_lower for word in ['what is', 'define', 'definition']):
            analysis['query_type'] = 'definitional'

        # Extract mentioned entities (simplified)
        for entity_id, entity in self.kg.entities.items():
            if entity.name.lower() in query_lower:
                analysis['entities_mentioned'].append(entity_id)

        # Infer required relationships
        if analysis['query_type'] == 'causal':
            analysis['relationships_implied'].append(RelationType.CAUSES)
        elif analysis['query_type'] == 'comparative':
            analysis['relationships_implied'].append(RelationType.SIMILAR_TO)

        return analysis

关键词表:why/because/cause/reason → causal(深层推理);how/process/method/way → procedural;compare/difference/similar/versus → comparative(中等深度);what is/define/definition → definitional。实体提取采用简化的名称匹配,因果类查询隐含 CAUSES 关系、比较类查询隐含 SIMILAR_TO 关系。

相关性子图提取(_extract_relevant_subgraph)是"把图裁剪成上下文"的关键:

    def _extract_relevant_subgraph(self, start_entity: str, query_analysis: Dict,
                                 depth: int = 3) -> Dict[str, Any]:
        """Extract relevant subgraph around an entity"""

        # Start with entity context
        entity_context = self.kg.get_entity_context(start_entity, depth=depth)

        # Score relevance of different parts
        relevance_scores = self._score_context_relevance(entity_context, query_analysis)

        # Filter based on relevance
        filtered_context = self._filter_by_relevance(entity_context, relevance_scores, threshold=0.3)

        return {
            'root_entity': start_entity,
            'context': filtered_context,
            'relevance_scores': relevance_scores,
            'subgraph_summary': self._summarize_subgraph(filtered_context)
        }

相关性打分(_score_context_relevance)的规则如下:

  • 一跳邻居:基础分 0.5;若关系类型被查询隐含则 +0.3;若目标实体被查询提及则 +0.4;
  • 扩展上下文实体:基础分 0.3;查询提及 +0.4;实体类型命中相关类型 +0.2;
  • 层次化上下文:父节点固定 0.4、子节点固定 0.3。

过滤(_filter_by_relevance)默认阈值 threshold=0.3,即得分 ≥ 0.3 的邻居/扩展实体/层次节点保留,其余剪枝。子图摘要(_summarize_subgraph)则输出形如 Entity: Alice (Person); Direct connections: 3; Extended network: 4 entities; Hierarchical: 1 parents, 0 children 的文本概括。

演绎推理(_deductive_reasoning)通过传递性规则链找逻辑结论:

    def _find_inference_chains(self, context: Dict, query_analysis: Dict) -> List[Dict]:
        """Find logical inference chains in context"""
        chains = []

        # Simple transitivity chains
        entity = context['entity']

        # For each outgoing relationship, see if we can chain it
        for target_id, rel1 in context['immediate_neighbors']['outgoing']:
            if target_id in context['extended_context']:
                # Look for relationships from this target
                target_context = self.kg.get_entity_context(target_id, depth=1)

                for final_target, rel2 in target_context['immediate_neighbors']['outgoing']:
                    # Check if this creates a meaningful chain
                    if self._is_valid_inference_chain(rel1, rel2):
                        chains.append({
                            'premises': [f"{entity.name} {rel1.predicate.value} {target_id}",
                                       f"{target_id} {rel2.predicate.value} {final_target}"],
                            'conclusion': f"{entity.name} (transitively) {rel2.predicate.value} {final_target}",
                            'confidence': rel1.confidence * rel2.confidence,
                            'chain_length': 2
                        })

        return chains

    def _is_valid_inference_chain(self, rel1: Relationship, rel2: Relationship) -> bool:
        """Check if two relationships can form valid inference chain"""
        # Valid transitivity patterns
        valid_patterns = [
            (RelationType.IS_A, RelationType.IS_A),
            (RelationType.PART_OF, RelationType.PART_OF),
            (RelationType.LOCATED_IN, RelationType.LOCATED_IN),
            (RelationType.WORKS_AT, RelationType.LOCATED_IN)
        ]

        return (rel1.predicate, rel2.predicate) in valid_patterns

演绎推理的"规则表"是显式编码的传递性模式:IS_A→IS_A、PART_OF→PART_OF、LOCATED_IN→LOCATED_IN、WORKS_AT→LOCATED_IN。这正是 Prompt 模板中 IF Alice works_at Google AND Google is_a Tech_Company THEN Alice works_at a Tech_Company 这类演绎示例的工程化表达。注意文档中的演绎示例文本与规则表并不完全一致(示例写的是 works_at + is_a,规则表收录的是 works_at + located_in),从源码结构看,规则表是有意收窄的"保守传递性白名单",实际使用时可以按业务扩展 valid_patterns。

归纳推理(_inductive_reasoning)跨多个子图统计重复出现的"实体类型—关系—目标类型"模式,至少出现 2 次才允许泛化,置信度按 min(1.0, instances/5) 计算(04_structured_context.md 的 _generalize_patterns 实现)。

溯因推理(_abductive_reasoning)先识别"现象"——例如某实体某类关系数量超过 3 条(high_relationship_count),再依据实体类型生成候选解释并打分:大型公司拥有多样属性(plausibility 0.8)、个人可能有多段工作经历(plausibility 0.6)等,最后取最优解释。

类比推理(_analogical_reasoning)对子图两两计算"出边关系类型集合"的 Jaccard 相似度,相似度 > 0.5 即视为结构类比,输出"两者关系模式相似,可能属于同一类别或承担相似角色"的洞察。

结果整合(_integrate_reasoning_results)把推理结论按策略格式化为可读文本(Deductive reasoning conclusions: / Identified patterns: / Best explanation: / Analogical insights:),并受 max_context_size 约束做截断——这正是"上下文组装"环节,输出包含 query、reasoning_type、context、confidence、reasoning_details、query_analysis 六个字段。

示例数据与演示入口

文档附带的示例知识图谱(create_sample_knowledge_graph)包含 7 个实体与 8 条关系:Alice(Person,属性 age=30、location=San Francisco)→ WORKS_AT Google、IS_A ai_researcher、LOCATED_IN san_francisco;Google(Company,founded=1998、employees=150000)→ IS_A tech_company、LOCATED_IN san_francisco;ai_researcher → RELATED_TO machine_learning;machine_learning → ENABLES python;tech_company → HAS_PROPERTY machine_learning。每条关系带 weight 与 confidence(0.6~1.0 不等)。

演示函数 demonstrate_structured_context 运行三类测试查询:

  • "What can you tell me about Alice?",种子 ["alice"]
  • "How is Google related to technology?",种子 ["google", "tech_company"]
  • "What is the connection between Alice and machine learning?",种子 ["alice", "machine_learning"]

对每个查询依次尝试 deductive、inductive、abductive、analogical 四种策略并打印上下文与置信度;随后演示 kg.find_paths("alice", "machine_learning", max_depth=3) 的路径查找与 kg.get_entity_context("alice", depth=2) 的实体上下文。整套代码位于文档正文(04_structured_context.md 的 Programming 章节),以 if __name__ == "__main__": kg, assembler = demonstrate_structured_context() 收尾,复制后安装 numpy、networkx 即可运行。


仓库配套实现:从教学代码到生产级实验室

文档给出的引擎是教学级的骨架;仓库配套的 structured_data_lab.py(约 1248 行)则把它扩展为"生产就绪"版本,覆盖六大能力:知识图谱构建与查询、GNN 嵌入、GraphRAG、Schema 处理与校验、图推理引擎、性能基准与可视化。以下是与本模块主题直接相关的五处关键扩展。

1. Schema 驱动的图构建与校验

实验室版新增 EntityType 枚举(PERSON / ORGANIZATION / LOCATION / EVENT / CONCEPT / DOCUMENT / PRODUCT)与 Schema 数据类:

@dataclass
class Schema:
    """Data schema definition with validation rules."""
    name: str
    entity_types: Set[EntityType]
    relation_types: Set[RelationType]
    constraints: Dict[str, Any] = field(default_factory=dict)

    def validate_entity(self, entity: Entity) -> bool:
        """Validate entity against schema."""
        return entity.entity_type in self.entity_types

    def validate_relation(self, relation: Relation) -> bool:
        """Validate relation against schema."""
        return (relation.relation_type in self.relation_types and
                self.validate_entity(relation.source) and
                self.validate_entity(relation.target))

KnowledgeGraph.__init__(d_model=256, schema=None) 接受可选 Schema:add_entity / add_relation 先做 Schema 校验,非法实体或关系直接拒绝写入。StructuredDataProcessor 进一步支持约束校验(required_fields、field_types)与从结构化数据批量提取实体——文档"Structured Context"关心的"保持结构语义"在此落地为可执行的校验规则。

2. GNN 嵌入与相似度检索

实验室版为图谱引入了嵌入层:

  • entity_embeddings(预分配 1000×d_model 的数组)与 relation_embeddings(每个 RelationType 一个 d_model 维向量);
  • compute_embeddings_gnn(num_iterations=3):迭代式消息传递——邻居嵌入经 relation_transform 变换并叠加关系嵌入×权重,聚合(均值)后与自身嵌入拼接,经 message_aggregation 与 tanh 更新,无邻居的节点退化为 entity_transform 变换;
  • similarity_search(query_embedding, top_k=5):余弦相似度排序取 Top-k;
  • _generate_entity_embedding:由实体类型基础嵌入 + 属性哈希特征叠加并归一化。

这让文档引擎"只按拓扑推理"的能力升级为"可嵌入检索 + 图结构融合",是文档 Entity.embeddings 字段的完整实现。

3. GraphRAG:图与文本的统一检索

class GraphRAG:
    """Graph-enhanced Retrieval-Augmented Generation system."""

    def __init__(self, d_model: int = 256):
        self.d_model = d_model
        self.knowledge_graph = KnowledgeGraph(d_model)
        self.document_embeddings = {}
        self.entity_document_map = defaultdict(set)  # entity_id -> document_ids
        ...
    def retrieve_context(self, query: str, query_embedding: np.ndarray,
                        top_k_entities: int = 5, top_k_documents: int = 3) -> Dict[str, Any]:
        ...

检索五步:查询嵌入变换 → 图谱相似度搜索找相关实体 → 对每个实体取半径 1 子图与邻居 → 经 entity_document_map 反查候选文档并按其嵌入相似度排序 → 拼接查询、实体、文档三路表示经 context_fusion(d_model*3 → d_model)融合为统一上下文。这与图增强 RAG 模块 03_graph_enhanced_rag.md 中 BasicGraphRAG / MultiHopGraphRAG / SemanticGraphIntelligence 三层架构的"实体链接 → 图遍历 → 文本检索 → 融合合成"管线同源,后者还给出了 /graph.rag.basic、/graph.rag.multi.hop、/graph.intelligence.semantic 三份协议级编排示例,可视为本模块编程范式的 RAG 化延伸。

4. GraphReasoner:规则驱动的显式推理

实验室版提供独立的推理引擎 GraphReasoner:

  • add_inference_rule(rule_name, premise_pattern, conclusion_pattern):注册形如 [("?x", RelationType.WORKS_FOR, "?y"), ("?y", RelationType.LOCATED_IN, "?z")] → ("?x", RelationType.LOCATED_IN, "?z") 的规则;
  • infer_new_relations():模式匹配后生成新关系,推断关系统一 weight=0.8 并打上 {'inferred': True, 'rule': ...} 元数据,且去重后返回;
  • answer_query(query, query_entities):按关键词路由到连通性(connected)、相似性(similar)、路径(path)、通用(其他)四类查询处理器。

这套"规则白名单 + 低置信度推断 + 可追溯元数据"的设计,与文档 _is_valid_inference_chain 的保守传递性策略互补:文档侧重推理策略选择,实验室侧重规则可扩展性与结果可解释性。

5. 性能基准与可视化

benchmark_graph_operations 对 100 / 500 / 1000 / 2000 个实体的图分别测量实体创建、关系创建、GNN 嵌入、路径查找(随机 100 对源-目标)、相似度搜索耗时,并统计平均度;visualize_graph_performance 输出六联图(构建性能、嵌入计算、路径查找、相似度搜索、连通性、吞吐量)。运行方式:

cd gh_mirrors/co/Context-Engineering
python 00_COURSE/02_context_processing/labs/structured_data_lab.py

依赖 numpy、networkx、matplotlib。这套基准给出的是相对耗时数据而非绝对值,用于横向比较图规模与各操作开销的缩放关系,适合作为文档演示代码的工程化验收补充。


研究关联与未来方向

与 Context Engineering Survey 的连接

原文档声明本模块直接实现并扩展了 2025 年发表的《A Systematic Analysis of Over 1400 Research Papers — A Survey of Context Engineering for Large Language Models》(00_COURSE/README.md 中引用)中的关键概念:

  • 知识图谱集成:实现了 StructGPT、GraphFormers 等结构化数据处理思路,并把 KG 集成扩展到完整的上下文组装;
  • 上下文处理创新(§4.2):把上下文处理原则应用到图结构化信息,将自改进(self-refinement)概念扩展到图谱优化;
  • 新颖研究贡献:① 多策略推理——演绎、归纳、溯因、类比四种策略的系统整合;② 层次化上下文网络——跨抽象层级动态组织信息;③ 自适应图智能——自改进的知识表示优化系统。

未来研究方向

原文档提出四条前沿路线:

  • 时序知识图谱(Temporal Knowledge Graphs):把静态图谱扩展为可捕获实体与关系随时间演化的结构,支持时序推理与预测;
  • 概率图推理(Probabilistic Graph Reasoning):把不确定性与概率推断引入图谱推理,使上下文组装更稳健;
  • 多模态知识图谱(Multi-Modal Knowledge Graphs):把上一模块(多模态上下文处理,见 03_multimodal_context.md)的跨模态能力与结构化表示结合,形成更丰富的上下文;
  • 涌现关系发现(Emergent Relationship Discovery):系统自动发现未被显式编程的新关系类型与模式,突破当前图谱的局限。

图增强 RAG 模块 03_graph_enhanced_rag.md 的未来章节还补充了超图 RAG、时序图 RAG、概率图 RAG、神经-符号图 RAG、跨模态图 RAG,以及 GNN 与传统图算法结合、联邦图智能、量子图算法等研究前沿,可作为本模块方向的外围地图。


小结与下一步

核心概念清单:

  • 基于图的上下文表示与遍历算法(BFS / DFS / 相关性引导);
  • 多策略推理系统(演绎、归纳、溯因、类比);
  • 层次化信息组织与传播(parent / child / cross-link 三层权重);
  • 面向上下文组装的知识图谱集成。

Software 3.0 三种范式的落点:

  • Prompts:结构化推理模板,驱动系统性图遍历(本文"知识图谱推理模板");
  • Programming:具备多策略推理能力的知识图谱引擎(本文 KnowledgeGraph + StructuredContextAssembler);
  • Protocols:自适应图智能系统,优化自身推理(对应 Stage 5 与实验室版 GraphReasoner 的规则扩展能力)。

掌握技能:知识图谱的构建与管理、多跳推理与路径查找算法、带相关性过滤的结构化上下文组装、四类推理策略的完整实现。

衔接路径:本模块(结构化上下文处理)之后,课程进入"Long Context Processing Lab"——通过交互式编码练习动手实现注意力机制、记忆系统与层次化处理架构;向上游看,结构化上下文可继续沉淀到 04_retrieval_augmented_generation/03_graph_enhanced_rag.md 的图增强 RAG 系统中形成检索-推理闭环。本模块演示的正是从线性信息处理走向网络化智能的演进——即 Software 3.0 原则下"系统不仅存储与检索信息,更能理解并推理那些创造意义、产生洞察的复杂关系"。


本文基于 00_COURSE/02_context_processing/04_structured_context.md(Context Engineering 课程 Module 02.4)整理撰写,源码级细节与运行示例分别取自 00_COURSE/02_context_processing/labs/structured_data_lab.py 与 00_COURSE/04_retrieval_augmented_generation/03_graph_enhanced_rag.md。

登录后查看全文
Context-Engineering