"Shell patterns" broken beyond repair

[mc-butler]

Important

This issue was migrated from Trac:

Origin	https://midnight-commander.org/ticket/2309
Reporter	wjaguar (@wjaguar)

Function mc_search__glob_translate_to_regex() in lib/search/glob.c is horribly broken, and mishandles a number of trivial patterns such as:
- an inverted character class: "[^abc]" or "[!abc]"
- a freestanding comma: "abc,def"
- a literal brace: "{" or "}"
- an incorrectly formed brace expansion: "{abc}"
- a sequence expression: "{5..10}" or "{a..f}"

Also, a nested brace expansion: "a{b,c{d,e}}f" will produce capturing parentheses for every pair of braces, instead of only the outermost pair.

All this because current code is far too simple-minded to correctly convert Bash patterns into PCRE regexes, and needs a complete rewrite.

Note

Original attachments:

• glob-to-regex.patch (raw) by boris on Dec 20, 2014 at 22:41 UTC
• glob4827.patch (raw) by wjaguar (@wjaguar) on Feb 26, 2025 at 22:39 UTC
• glob4833.patch (raw) by wjaguar (@wjaguar) on Feb 26, 2025 at 22:40 UTC
• glob_tests.c (raw) by wjaguar (@wjaguar) on Feb 26, 2025 at 22:41 UTC

[mc-butler]

Changed by andrew_b (@aborodin) on Nov 16, 2011 at 5:54 UTC (comment 1)

• Milestone changed from 4.7 to Future Releases
• Branch state set to no branch

[mc-butler]

Changed by boris on Dec 20, 2014 at 22:41 UTC

• Attachment glob-to-regex.patch (raw) added

[mc-butler]

Changed by boris on Dec 20, 2014 at 22:42 UTC (comment 2)

Please, look at the proposed patch. It does not intend to fix all Bash patterns (I believe mc never claimed to support all kinds of them), but it fixes some issues.
Namely, backslash-escaped metacharacter like {}*? will remain in the pattern (with the current code it is just stripped). Second, comma will be transformed to | only inside a group.

[mc-butler]

Changed by andrew_b (@aborodin) on Feb 17, 2015 at 11:18 UTC

• Blocking set to #3350

[mc-butler]

Changed by andrew_b (@aborodin) on Feb 17, 2015 at 11:18 UTC

• Blocking changed from #3350 to #2577, #3350

[mc-butler]

Changed by slavazanko (@slavaz) on Feb 23, 2015 at 10:10 UTC (comment 5)

• Status changed from new to accepted
• Owner set to slavazanko

[mc-butler]

Changed by slavazanko (@slavaz) on Feb 23, 2015 at 14:10 UTC (comment 6)

• Branch state changed from no branch to on review

Created branch 2309_shell_patterns
Initial [0eca32a]

Review please.

[mc-butler]

Changed by andrew_b (@aborodin) on Feb 23, 2015 at 15:04 UTC (comment 7)

• Milestone changed from Future Releases to 4.8.14
• Branch state changed from on review to approved
• Votes set to andrew_b

Approved with simple fixup.

[mc-butler]

Changed by slavazanko (@slavaz) on Feb 24, 2015 at 12:02 UTC (comment 8)

• Blocking #2577, #3350 deleted
• Status changed from accepted to testing
• Resolution set to fixed
• Votes changed from andrew_b to committed-master
• Branch state changed from approved to merged

Merged to master [e5ec837]:

git log --pretty=oneline f4faf60..6ca737d

[mc-butler]

Changed by slavazanko (@slavaz) on Feb 24, 2015 at 12:04 UTC (comment 9)

• Status changed from testing to closed

[mc-butler]

Changed by mooffie (@mooffie) on Dec 4, 2016 at 19:03 UTC (comment 10)

Ticket #2577 has been marked as a duplicate of this ticket.

[mc-butler]

Changed by zaytsev (@zyv) on Feb 21, 2025 at 7:40 UTC (comment 11)

Ticket #2379 has been marked as a duplicate of this ticket.

[mc-butler]

Changed by zaytsev (@zyv) on Feb 21, 2025 at 7:44 UTC (comment 12)

Ticket #3350 has been marked as a duplicate of this ticket.

[mc-butler]

Changed by wjaguar (@wjaguar) on Feb 22, 2025 at 2:51 UTC (comment 13)

• Status changed from closed to reopened
• Resolution fixed deleted

This STILL REMAINS broken. Yes, after all these years. The fix has done nothing for inverted character classes, they still do NOT work, instead being treated as regular ones.

Naturally, all uses of MC_SEARCH_T_GLOB are affected. A couple examples:

• searching for "[^g]" or "[!g]" finds a letter "g";
• selecting "[^g]*" or "[!g]*" in mc's lib/search dir select "glob.c" instead of everything but.

All the listed problems with brace expansion also remain:

• unpaired literal braces are misinterpreted and break the RE:
  you cannot select, say, "{" or "*.{" despite files matching the pattern being there;
• a text in braces w/o a comma is misinterpreted as without braces:
  trying to select "{abc}*" selects "abc";
• a sequence expression is misinterpreted as literal text w/o braces:
  trying to select "{a..h}*" selects "a..h" and does not select "glob.c" or "hex.c";
• nested braces now just plain do not work:
  "{{a,b},c}*" selects "a,c" and "b,c", but neither "abc" nor "ccc".