Quesiton about history image in evaluation

[syorami]

Hi I'm wondering if there is anything wrong with the hsitory image in evaluation. The code is as below:

        img = data['image']
        question = f"Picture 1: <img>{img}</img>\nI'm looking for guidance on how to {data['question']}"
        answer = data['answer']
        
        history_action = eval(data['history_action'])[-self.his_len:]
        if IMAGE_HISTORY:
            if len(history_action) > 0:
                his_img = f'\nPrevious screenshots: <img>image-history: {img}</img>'
                his_str = '\nPrevious Actions: '
                for idx, hi in enumerate(history_action):
                    his_str += f"{idx+1}. {hi}\n"
                
                question = f"{question}{his_img}{his_str}"

And the prompt output is:

Picture 1: <img>/heyuan79/lingyu/GUI-Odyssey/screenshots/6518639607424300_5.png</img>
I'm looking for guidance on how to Using Simple Calculator, compute today's total expense by adding 100, 23, and 2. Once you have the sum, document the total in Wallet: Budget Money Manager.
Previous screenshots: <img>image-history: /heyuan79/lingyu/GUI-Odyssey/screenshots/6518639607424300_5.png</img>
Previous Actions: 1. CLICK: (471, 472)
2. CLICK: (479, 750)
3. CLICK: (638, 917)
4. CLICK: (974, 782)

It seems that history screenshots are not actually used and the previous screenshot here is actually the current image. Could you help clarify on this?

[Lqf-HFNJU]

<img>image-history: xxx</img> is essentially an index. During processing, it first looks up his_index.json and uses that index to retrieve the preceding screenshot. For details on how this is implemented, see lines 639–646 and 710–714 of modeling_qwen.py. The reason for retrieving previous screenshots in this manner is that qwenvl can only store 256 characters for image paths, so listing every image path might exceed that limit :)