Swift Code Extraction Issue

[yckota]

In extract_swift_code function, a pattern is written in a way to remove any function starting with check. https://github.com/MCEVAL/McEval/blob/main/eval/extract/extract_swift_code.py#L24

    pattern = r"func\s+check\w*\s*\(.*?\)\s*\{.*\}"
    code = re.sub(pattern, "", code, flags=re.DOTALL)

A task in the Swift dataset has function name starting with checkLikes which removes the code for this task causing failures and incorrectly marked as Failed.

https://github.com/MCEVAL/McEval/blob/main/data/Swift.jsonl#L28C62-L28C72

{
    "task_id": "Swift/28",
    "prompt": "import Foundation\n\nfunc checkLikes(_ x: Int) -> (Int, Int, Int, Int) {\n/**\nChecks if the given integer is liked by the four individuals:\n- Person A likes the integer if it is both even and between 4 and 12 (inclusive).\n- Person U likes the integer if it is either even or between 4 and 12 (inclusive).\n- Person H likes the integer if it is exactly one of the two properties (even or between 4 and 12).\n- Person G likes the integer if it does not have both of the properties (even and between 4 and 12).\n\n>>> checkLikes(12)\n(0, 1, 1, 0)\n*/",
    "canonical_solution": "let isEven = x % 2 == 0\nlet isBetween4And12 = x > 4 && x < 12\nlet isExactlyOneProperty = isEven != isBetween4And12\nlet isNeitherProperty = !isEven && !isBetween4And12\n\nlet a = isEven && isBetween4And12 ? 1 : 0\nlet u = isEven || isBetween4And12 ? 1 : 0\nlet h = isExactlyOneProperty ? 1 : 0\nlet g = isNeitherProperty ? 1 : 0\n\nreturn (a, u, h, g)\n}",
    "test": "func testCheckLikes(_ checkLikes: (Int) -> (Int, Int, Int, Int)) {\nassert(checkLikes(12) == (0, 1, 1, 0))\nassert(checkLikes(5) == (0, 1, 1, 0))\nassert(checkLikes(6) == (1, 1, 0, 0))\nassert(checkLikes(3) == (0, 0, 0, 1))\nassert(checkLikes(13) == (0, 0, 0, 1))\nassert(checkLikes(4) == (0, 1, 1, 0))\nassert(checkLikes(11) == (0, 1, 1, 0))\nassert(checkLikes(7) == (0, 1, 1, 0))\nassert(checkLikes(2) == (0, 1, 1, 0))\n}\n\ntestCheckLikes(checkLikes)",
    "entry_point": "checkLikes",
    "signature": "func checkLikes(_ x: Int) -> (Int, Int, Int, Int) {",
    "docstring": "\nChecks if the given integer is liked by the four individuals:\n- Person A likes the integer if it is both even and between 4 and 12 (inclusive).\n- Person U likes the integer if it is either even or between 4 and 12 (inclusive).\n- Person H likes the integer if it is exactly one of the two properties (even or between 4 and 12).\n- Person G likes the integer if it does not have both of the properties (even and between 4 and 12).\n\n>>> checkLikes(12)\n(0, 1, 1, 0)\n",
    "instruction": "Write a Swift function `func checkLikes(_ x: Int) -> (Int, Int, Int, Int) {` to solve the following problem:\n\nChecks if the given integer is liked by the four individuals:\n- Person A likes the integer if it is both even and between 4 and 12 (inclusive).\n- Person U likes the integer if it is either even or between 4 and 12 (inclusive).\n- Person H likes the integer if it is exactly one of the two properties (even or between 4 and 12).\n- Person G likes the integer if it does not have both of the properties (even and between 4 and 12).\n\n>>> checkLikes(12)\n(0, 1, 1, 0)\n",
    "level": "hard"
}

[Challenging6]

Sorry for the late reply! Thank you so much for pointing out the error! This issue was introduced because we were adapting to certain model outputs that included a check function at the time. We have now commented out that part of the code. Once again, we sincerely appreciate your detailed reminder!