Map Your Entity Network failled

[Snakdoc]

Hello,

I have an issue during the training of my second me during the Synthesize Your Life Narrative - Map Your Entity Network. I'm using the Integrated Setup (Non-Docker) on a Mac M4 - 36 Go RAM.
Whatever the model I use, i have the same issue
This is the logs that I have

100%|██████████| 3/3 [00:00<00:00, 13888.42it/s]
2025-08-09 16:26:14 [INFO] data.py:461 - Removed existing file: /Users/snakdoc/Desktop/Mehdi/Projet/Second-Me/resources/L1/processed_data/objective/note_remade.json
2025-08-09 16:26:14 [INFO] data.py:465 - Cleared all existing files in /Users/snakdoc/Desktop/Mehdi/Projet/Second-Me/resources/L1/processed_data/objective
0it [00:00, ?it/s]
2025-08-09 16:26:14 [INFO] data.py:103 - Data refinement completed, preparing to extract entities and relations
2025-08-09 16:26:14 [INFO] data.py:546 - Input base_dir has been updated to /Users/snakdoc/Desktop/Mehdi/Projet/Second-Me/resources/L1/processed_data/subjective and saved.
2025-08-09 16:26:14 [INFO] data.py:547 - Output base_dir has been updated to /Users/snakdoc/Desktop/Mehdi/Projet/Second-Me/resources/L1/graphrag_indexing_output/subjective and saved.
2025-08-09 16:26:14 [INFO] data.py:548 - Report base_dir has been updated to /Users/snakdoc/Desktop/Mehdi/Projet/Second-Me/resources/L1/graphrag_indexing_output/subjective/report and saved.
2025-08-09 16:26:14 [INFO] _internal.py:97 - 127.0.0.1 - - [09/Aug/2025 16:26:14] "GET /api/trainprocess/progress/Qwen2.5-0.5B-Instruct HTTP/1.1" 200 -
2025-08-09 16:26:15 [ERROR] trainprocess_service.py:312 - Map entity network failed: Command '['bash', '/Users/snakdoc/Desktop/Mehdi/Projet/Second-Me/lpm_kernel/L2/data_pipeline/data_prep/scripts/graphrag_indexing.sh']' returned non-zero exit status 1.
2025-08-09 16:26:15 [INFO] trainprocess_service.py:390 - Cleaning up resources to prevent memory leaks
2025-08-09 16:26:15 [INFO] trainprocess_service.py:404 - Memory usage after cleanup: 554.84 MB
2025-08-09 16:26:15 [ERROR] trainprocess_service.py:1112 - Step map_your_entity_network failed
2025-08-09 16:26:15 [INFO] trainprocess_service.py:1113 - Marking step as failed: stage=map_your_entity_network, step=map_your_entity_network
2025-08-09 16:26:17 [INFO] _internal.py:97 - 127.0.0.1 - - [09/Aug/2025 16:26:17] "GET /api/trainprocess/progress/Qwen2.5-0.5B-Instruct HTTP/1.1" 200 -

Any help would be much appreciated.

Thank You

[slhad]

I was looking at the bash script failing as it's also mentioned when I get the same error

( I use ollama on another computer more powerfull, while second-me exec on docker on an older repurposed computer as linux server)

If I run the command manually with --verbose and default logger I get :

root@6e5a1c80d9ca:/app# graphrag index --config lpm_kernel/L2/data_pipeline/graphrag_indexing/settings.yaml --root lpm_kernel/L2/data_pipeline/graphrag_indexing --method standard --verbose

Logging enabled at /app/resources/L1/graphrag_indexing_output/report/indexing-engine.log
Starting pipeline run. dry_run=False
Using default configuration: {
    "root_dir": "/app/lpm_kernel/L2/data_pipeline/graphrag_indexing",
    "models": {
        "default_chat_model": {
            "api_key": "==== REDACTED ====",
            "auth_type": "api_key",
            "type": "openai_chat",
            "model": "qwen2.5:0.5b",
            "encoding_model": "cl100k_base",
            "max_tokens": 4000,
            "temperature": 0,
            "top_p": 1,
            "n": 1,
            "frequency_penalty": 0.0,
            "presence_penalty": 0.0,
            "request_timeout": 180.0,
            "api_base": "http://192.168.0.118:11434/v1",
            "api_version": null,
            "deployment_name": null,
            "proxy": null,
            "audience": null,
            "model_supports_json": true,
            "tokens_per_minute": 0,
            "requests_per_minute": 0,
            "retry_strategy": "native",
            "max_retries": -1,
            "max_retry_wait": 10.0,
            "concurrent_requests": 2,
            "responses": null,
            "async_mode": "threaded"
        },
        "default_embedding_model": {
            "api_key": "==== REDACTED ====",
            "auth_type": "api_key",
            "type": "openai_embedding",
            "model": "snowflake-arctic-embed:110m",
            "encoding_model": "cl100k_base",
            "max_tokens": 4000,
            "temperature": 0,
            "top_p": 1,
            "n": 1,
            "frequency_penalty": 0.0,
            "presence_penalty": 0.0,
            "request_timeout": 180.0,
            "api_base": "http://192.168.0.118:11434/v1",
            "api_version": null,
            "deployment_name": null,
            "proxy": null,
            "audience": null,
            "model_supports_json": true,
            "tokens_per_minute": 0,
            "requests_per_minute": 0,
            "retry_strategy": "native",
            "max_retries": -1,
            "max_retry_wait": 10.0,
            "concurrent_requests": 2,
            "responses": null,
            "async_mode": "threaded"
        }
    },
    "reporting": {
        "type": "file",
        "base_dir": "/app/resources/L1/graphrag_indexing_output/report",
        "storage_account_blob_url": null
    },
    "output": {
        "type": "file",
        "base_dir": "/app/resources/L1/graphrag_indexing_output/subjective",
        "storage_account_blob_url": null,
        "cosmosdb_account_url": null
    },
    "outputs": null,
    "update_index_output": {
        "type": "file",
        "base_dir": "/app/lpm_kernel/L2/data_pipeline/graphrag_indexing/update_output",
        "storage_account_blob_url": null,
        "cosmosdb_account_url": null
    },
    "cache": {
        "type": "file",
        "base_dir": "cache",
        "storage_account_blob_url": null,
        "cosmosdb_account_url": null
    },
    "input": {
        "type": "file",
        "file_type": "text",
        "base_dir": "/app/resources/L1/processed_data/subjective",
        "storage_account_blob_url": null,
        "encoding": "utf-8",
        "file_pattern": ".*\\.txt$",
        "file_filter": null,
        "text_column": "text",
        "title_column": null,
        "metadata": null
    },
    "embed_graph": {
        "enabled": false,
        "dimensions": 1536,
        "num_walks": 10,
        "walk_length": 40,
        "window_size": 2,
        "iterations": 3,
        "random_seed": 597832,
        "use_lcc": true
    },
    "embed_text": {
        "batch_size": 16,
        "batch_max_tokens": 8191,
        "target": "required",
        "names": [],
        "strategy": null,
        "model_id": "default_embedding_model",
        "vector_store_id": "default_vector_store"
    },
    "chunks": {
        "size": 500,
        "overlap": 100,
        "group_by_columns": [
            "id"
        ],
        "strategy": "tokens",
        "encoding_model": "cl100k_base",
        "prepend_metadata": false,
        "chunk_size_includes_metadata": false
    },
    "snapshots": {
        "embeddings": false,
        "graphml": false
    },
    "extract_graph": {
        "prompt": "prompts/extract_graph.txt",
        "entity_types": [
            "organization",
            "person",
            "geo",
            "event",
            "specific object",
            "abstract object"
        ],
        "max_gleanings": 1,
        "strategy": null,
        "encoding_model": null,
        "model_id": "default_chat_model"
    },
    "extract_graph_nlp": {
        "normalize_edge_weights": true,
        "text_analyzer": {
            "extractor_type": "regex_english",
            "model_name": "en_core_web_md",
            "max_word_length": 15,
            "word_delimiter": " ",
            "include_named_entities": true,
            "exclude_nouns": null,
            "exclude_entity_tags": [
                "DATE"
            ],
            "exclude_pos_tags": [
                "DET",
                "PRON",
                "INTJ",
                "X"
            ],
            "noun_phrase_tags": [
                "PROPN",
                "NOUNS"
            ],
            "noun_phrase_grammars": {
                "PROPN,PROPN": "PROPN",
                "NOUN,NOUN": "NOUNS",
                "NOUNS,NOUN": "NOUNS",
                "ADJ,ADJ": "ADJ",
                "ADJ,NOUN": "NOUNS"
            }
        },
        "concurrent_requests": 25
    },
    "summarize_descriptions": {
        "prompt": "prompts/summarize_descriptions.txt",
        "max_length": 500,
        "strategy": null,
        "model_id": "default_chat_model"
    },
    "community_reports": {
        "graph_prompt": null,
        "text_prompt": null,
        "max_length": 2000,
        "max_input_length": 8000,
        "strategy": null,
        "model_id": "default_chat_model"
    },
    "extract_claims": {
        "enabled": false,
        "prompt": null,
        "description": "Any claims or facts that could be relevant to information discovery.",
        "max_gleanings": 1,
        "strategy": null,
        "encoding_model": null,
        "model_id": "default_chat_model"
    },
    "prune_graph": {
        "min_node_freq": 2,
        "max_node_freq_std": null,
        "min_node_degree": 1,
        "max_node_degree_std": null,
        "min_edge_weight_pct": 40,
        "remove_ego_nodes": false,
        "lcc_only": false
    },
    "cluster_graph": {
        "max_cluster_size": 10,
        "use_lcc": true,
        "seed": 3735928559
    },
    "umap": {
        "enabled": false
    },
    "local_search": {
        "prompt": null,
        "text_unit_prop": 0.5,
        "community_prop": 0.15,
        "conversation_history_max_turns": 5,
        "top_k_entities": 10,
        "top_k_relationships": 10,
        "temperature": 0,
        "top_p": 1,
        "n": 1,
        "max_tokens": 12000,
        "llm_max_tokens": 2000
    },
    "global_search": {
        "map_prompt": null,
        "reduce_prompt": null,
        "knowledge_prompt": null,
        "temperature": 0,
        "top_p": 1,
        "n": 1,
        "max_tokens": 12000,
        "data_max_tokens": 12000,
        "map_max_tokens": 1000,
        "reduce_max_tokens": 2000,
        "concurrency": 32,
        "dynamic_search_llm": "gpt-4o-mini",
        "dynamic_search_threshold": 1,
        "dynamic_search_keep_parent": false,
        "dynamic_search_num_repeats": 1,
        "dynamic_search_use_summary": false,
        "dynamic_search_concurrent_coroutines": 16,
        "dynamic_search_max_level": 2
    },
    "drift_search": {
        "prompt": null,
        "reduce_prompt": null,
        "temperature": 0,
        "top_p": 1,
        "n": 1,
        "max_tokens": 12000,
        "data_max_tokens": 12000,
        "reduce_max_tokens": 2000,
        "reduce_temperature": 0,
        "concurrency": 32,
        "drift_k_followups": 20,
        "primer_folds": 5,
        "primer_llm_max_tokens": 12000,
        "n_depth": 3,
        "local_search_text_unit_prop": 0.9,
        "local_search_community_prop": 0.1,
        "local_search_top_k_mapped_entities": 10,
        "local_search_top_k_relationships": 10,
        "local_search_max_data_tokens": 12000,
        "local_search_temperature": 0,
        "local_search_top_p": 1,
        "local_search_n": 1,
        "local_search_llm_max_gen_tokens": 2000
    },
    "basic_search": {
        "prompt": null,
        "text_unit_prop": 0.5,
        "conversation_history_max_turns": 5,
        "temperature": 0,
        "top_p": 1,
        "n": 1,
        "max_tokens": 12000,
        "llm_max_tokens": 2000
    },
    "vector_store": {
        "default_vector_store": {
            "type": "lancedb",
            "db_uri": "/app/lpm_kernel/L2/data_pipeline/graphrag_indexing/output/lancedb",
            "url": null,
            "audience": null,
            "container_name": "==== REDACTED ====",
            "database_name": null,
            "overwrite": true
        }
    },
    "workflows": [
        "create_base_text_units",
        "create_final_documents",
        "extract_graph",
        "finalize_graph"
    ]
}
Running standard indexing.
🚀 create_base_text_units
                                                  id                                               text                                       document_ids  n_tokens
0  da5694791d5531142ef6215ba77d4a720e7bb8328569af...  Here is the data recorded by Slash:I have prob...  [541b2974c75d9f8df5b1970fc8afd3c81d88a850c5cd4...       137
1  fdd00c9f68faf06fc3e1afd87c63d9781304375b1b9900...  Here is the data recorded by Slash:I love anim...  [590736012fdaa1b81cb4102ce104bb1ccca2553601d13...        57
2  8b5b8083fed2691ce3db98efdcb5da27ce49e7e614783b...  The following is data recorded by Slash:I have...  [6de9ceec04a01465c160a8702cdc4737193a790ddb43a...        75
🚀 create_final_documents
                                                  id  human_readable_id       title                                               text                                      text_unit_ids              creation_date
0  541b2974c75d9f8df5b1970fc8afd3c81d88a850c5cd4e...                  1  note_1.txt  Here is the data recorded by Slash:I have prob...  [da5694791d5531142ef6215ba77d4a720e7bb8328569a...  2025-08-12 21:17:53 +0000
1  6de9ceec04a01465c160a8702cdc4737193a790ddb43af...                  2  note_2.txt  The following is data recorded by Slash:I have...  [8b5b8083fed2691ce3db98efdcb5da27ce49e7e614783...  2025-08-12 21:17:53 +0000
2  590736012fdaa1b81cb4102ce104bb1ccca2553601d130...                  3  note_0.txt  Here is the data recorded by Slash:I love anim...  [fdd00c9f68faf06fc3e1afd87c63d9781304375b1b990...  2025-08-12 21:17:53 +0000
❌ extract_graph
None
⠸ GraphRAG Indexer
├── Loading Input (InputFileType.text) - 3 files loaded (0 filtered) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:00
├── create_base_text_units ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:00
├── create_final_documents ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 0:00:00
❌ Errors occurred during the pipeline run, see logs for more details.

[nicocarv]

Hello,

I have the same issue: Map Your Entity Network as FAILED

MAC M1 32GB.
I'm using the docker version.

2025-08-30 14:33:39 [INFO] trainprocess_service.py:486 - extract_notes_from_documents len: 12
2025-08-30 14:33:40 [INFO] trainprocess_service.py:508 - Loading user information...
2025-08-30 14:33:40 [INFO] data.py:340 - Subjective notes: 12, Objective notes: 0
2025-08-30 14:33:40 [INFO] data.py:385 - Refined subjective notes: 12
2025-08-30 14:33:40 [INFO] data.py:429 - Refined objective notes: 0
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_7.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_6.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_remade.json
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_4.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_5.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_1.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_0.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_2.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_3.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_10.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_11.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_8.txt
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/subjective/note_9.txt
2025-08-30 14:33:40 [INFO] data.py:465 - Cleared all existing files in /app/resources/L1/processed_data/subjective
2025-08-30 14:33:40 [INFO] data.py:461 - Removed existing file: /app/resources/L1/processed_data/objective/note_remade.json
2025-08-30 14:33:40 [INFO] data.py:465 - Cleared all existing files in /app/resources/L1/processed_data/objective
2025-08-30 14:33:40 [INFO] data.py:103 - Data refinement completed, preparing to extract entities and relations
2025-08-30 14:33:40 [INFO] data.py:546 - Input base_dir has been updated to /app/resources/L1/processed_data/subjective and saved.
2025-08-30 14:33:40 [INFO] data.py:547 - Output base_dir has been updated to /app/resources/L1/graphrag_indexing_output/subjective and saved.
2025-08-30 14:33:40 [INFO] data.py:548 - Report base_dir has been updated to /app/resources/L1/graphrag_indexing_output/subjective/report and saved.
2025-08-30 14:33:41 [ERROR] trainprocess_service.py:312 - Map entity network failed: Command '['bash', '/app/lpm_kernel/L2/data_pipeline/data_prep/scripts/graphrag_indexing.sh']' returned non-zero exit status 1.
2025-08-30 14:33:41 [INFO] trainprocess_service.py:390 - Cleaning up resources to prevent memory leaks
2025-08-30 14:33:41 [INFO] trainprocess_service.py:404 - Memory usage after cleanup: 596.38 MB
2025-08-30 14:33:41 [ERROR] trainprocess_service.py:1112 - Step map_your_entity_network failed
2025-08-30 14:33:41 [INFO] trainprocess_service.py:1113 - Marking step as failed: stage=map_your_entity_network, step=map_your_entity_network