Global biography generation failure

[Heterohabilis]

Operating System

• macOS
• Linux
• Windows

Deployment Method

• Docker
• non-Docker

CUDA Usage

• Yes
• No

Training Process Details (if applicable)

N/A

Second-Me version

latest master

Describe the bug

2025-06-10 22:59:15 [ERROR] shade_generator.py:711 - Traceback (most recent call last):
File "/app/lpm_kernel/L1/shade_generator.py", line 696, in merge_shades
new_cluster_embedd = self._calculate_merged_shades_center_embed(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/lpm_kernel/L1/shade_generator.py", line 570, in _calculate_merged_shades_center_embed
len(shades[0].cluster_info["centerEmbedding"])
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

Current Behavior

From the code, we can know that the stes BEFORE the creation of global_bio are:

1. create shades
2. convert shades to shades info
3. merge shades
   Also, we can know that when shades are firstly created, it doesn't have an ID (id = None), so when we are merging the shades, it will regard the shades as empty as there is no id.

                    if not shades:
                        logger.info(
                            f"No valid shades found for shadeIds: {shade_ids}. Skipping this group."
                        )
                        continue

However, I tried to temporarily add id to the shade info class by:

        shades_merge_infos = convert_from_shades_to_merge_info(shades)
        for i in range(len(shades_merge_infos)):
            shades_merge_infos[i].id = str(uuid.uuid4())

I found the first part of the merge shade function would work; but the second half would raise an exception

2025-06-10 22:59:15 [ERROR] shade_generator.py:711 - Traceback (most recent call last):
  File "/app/lpm_kernel/L1/shade_generator.py", line 696, in merge_shades
    new_cluster_embedd = self._calculate_merged_shades_center_embed(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lpm_kernel/L1/shade_generator.py", line 570, in _calculate_merged_shades_center_embed
    len(shades[0].cluster_info["centerEmbedding"])
        ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

I checked the convert_from_shades_to_merge_info function, I found that:

def convert_from_shades_to_merge_info(shades: List[ShadeInfo]) -> List[ShadeMergeInfo]:
    return [ShadeMergeInfo(
        id=shade.id,
        name=shade.name,
        aspect=shade.aspect,
        icon=shade.icon,
        desc_third_view=shade.desc_third_view,
        content_third_view=shade.content_third_view,
        desc_second_view=shade.desc_second_view,
        content_second_view=shade.content_second_view,
        cluster_info=None
    ) for shade in shades]

It directly set cluster_info as None. Thus, in _calculate_merged_shades_center_embed, when we are trying to access cluster_info field, a exception will be raised.

Expected Behavior

The convert_from_shades_to_merge_info should properly initialize the cluster_info field to load cooresponding cluster.

Reproduction Steps

After

        # 3.3 Generate features for each cluster and merge them
        shades = generate_shades(clusters, l1_generator, notes_list)
        shades_merge_infos = convert_from_shades_to_merge_info(shades)

, add this to give each shade_merge_info a temp id.

        for i in range(len(shades_merge_infos)):
            shades_merge_infos[i].id = str(uuid.uuid4())

Possible Workaround

No response

Additional Information

No response

Link to related Github discussion or issue

No response

[seokwoosong]

I'm experiencing the same error (shade ID keeps getting assigned as None).

• I can't even find where the shade_id value is being assigned or initialized.

How can I fix this?

[Heterohabilis]

I'm experiencing the same error (shade ID keeps getting assigned as None).

• I can't even find where the shade_id value is being assigned or initialized.

How can I fix this?

I don’t know, so I posted this issue.

1. There is a function called save_shade that will return an object called “model”, in which the id field has been set. According to my knowledge, before being inserted to the db, the shade will be None forever. The id is possibly being set automatically in the insertion to db (most likely be a trigger and a automatic increase counter)
2. Even if you generate a random id for the shade, the code still doesn’t work. You will observe that before calling merge_shades function, the program will convert shades from shadesInfo class to another class, in which there is a field called cluster_info (hope this is the name). However, during this conversion, the cluster_info field will be set to None as default (I suspect that the authors haven’t implemented this part). Then, because of this None, after you enter the merge_shades function, there will be a step that will access a field of cluster_info. Exception.

Without the direct answer from the authors, I do believe that it’s quite hard to correct this place.
P.S., normally, this part is very hard to be found, as the global bio generator will not raise any exceptions, but just return something like “too limited info provided”. This won’t impact the general effect of the program [if you turn on both l1 and l0] when you interact with it. However, without l1 and l0, if you ask questions about yourself, it will be extremely silly. This is because self-QA train set (L2) is mainly based on global bio.

[seokwoosong]

The training completed successfully despite this error, but the trained model fails to comprehend the perspective of the training data. I suspect this error might be the cause.

So, I'm trying to fix this errors. We might need author's help.

I can't locate where shade.cluster_info is assigned, but the code accesses cluster_info["centerEmbedding"] and cluster_info["clusterSize"].

from the LATEST CODE :

• def generate_l1_from_l0()
  ...
  shades = generates_shade()
  shades_merge_infos = convert_from_shades_to_merge_info(shades) # RETURN "shade.cluster_info =NONE"
  merged_shades = l1_generator.merge_shades(shades_merge_infos) # Use shade.cluster_info["centerEmbedding"] and Null Exception / cluster_info["clusterSize"] also is called after this

• def convert_from_shades_to_merge_info():
  return [ShadeMergeInfo(
  ...
  cluster_info=None
  ) for shade in shades]

• def merge_shades():
  ...
  new_cluster_embedd = self._calculate_merged_shades_center_embed(shades)
  ...

• def _calculate_merged_shades_center_embed( )
  ....
  total_embedding = np.zeros(
  **len(shades[0].cluster_info["centerEmbedding"]) # ERROR still NULL **
  ) # Assuming center_embedding is a fixed-length vector
  ...

[Heterohabilis]

The training completed successfully despite this error, but the trained model fails to comprehend the perspective of the training data. I suspect this error might be the cause.

So, I'm trying to fix this errors. We might need author's help.

I can't locate where shade.cluster_info is assigned, but the code accesses cluster_info["centerEmbedding"] and cluster_info["clusterSize"].

from the LATEST CODE :

• def generate_l1_from_l0()
  ...
  shades = generates_shade()
  shades_merge_infos = convert_from_shades_to_merge_info(shades) # RETURN "shade.cluster_info =NONE"
  merged_shades = l1_generator.merge_shades(shades_merge_infos) # Use shade.cluster_info["centerEmbedding"] and Null Exception / cluster_info["clusterSize"] also is called after this
• def convert_from_shades_to_merge_info():
  return [ShadeMergeInfo(
  ...
  cluster_info=None
  ) for shade in shades]
• def merge_shades():
  ...
  new_cluster_embedd = self._calculate_merged_shades_center_embed(shades)
  ...
• def _calculate_merged_shades_center_embed( )
  ....
  total_embedding = np.zeros(
  **len(shades[0].cluster_info["centerEmbedding"]) # ERROR still NULL **
  ) # Assuming center_embedding is a fixed-length vector
  ...

To be honest, the model still works fine if u turn on L0 and L1 retrieval. But it definitely affects the performance negatively.
Try to use curl, or python requests.

[seokwoosong]

Your suggestion for shader_id is good, but I'm not sure if it is perfectly problem-free.
I hope the authors can update the code to address the following two errors:

1. The shade ID is always 'None'.
2. shade.cluster_info is initialized as None and will be accessed without assigning new data.

[seokwoosong]

I think this problems might come from few memory data.

• With only 5 memories (each containing 5-10 sentences), I encountered a NoneType error when accessing shades.cluster_info["clusterSize"] descirbed in this issue.
• The error resolved (without code change) when using 20 memories (each containing 5-10 sentences)
• Additionaly, data synthesis requires a sufficiently large LLM (>200B parameters or public API), based on my experiment.

[seokwoosong]

I think this problems might come from few memory data.

• With only 5 memories (each containing 5-10 sentences), I encountered a NoneType error when accessing shades.cluster_info["clusterSize"] descirbed in this issue.
• The error resolved (without code change) when using 20 memories (each containing 5-10 sentences)
• Additionaly, data synthesis requires a sufficiently large LLM (>200B parameters or public API), based on my experiment.

Sorry I missed something.
I still have the same errors (NoneType for cluster_info) regardless of memory data length.

[seokwoosong]

I found the solution of NoneType error for cluster_info.
This is because of unknown embedding size.

Please refer :
#390

[Heterohabilis]

I found the solution of NoneType error for cluster_info. This is because of unknown embedding size.

Please refer : #390

Try to give them cluster size, and you will find more mistakes in old bio creating

[doubleBlack2]

Hi, there are some bugs in the merge shapes section that can cause the shapes to be merged into 0 when there is only one. We have resolved this issue in the new version, which will soon be launched and adapted to the latest Qwen3 model. The data synthesis section has also been updated.