Skip to content

Commit a50a86e

Browse files
committed
feat: support inline multi-form plurals and refined language rules
- Added support for inline multi-form plural syntax: {{plural:count|one:item|other:{{count}} items}} - Refined plural rules for French (0 and 1 are 'one') - Added more comprehensive language coverage in internal/plural - Updated README, docs, tests, and examples with new pluralization features
1 parent a8081ec commit a50a86e

7 files changed

Lines changed: 132 additions & 17 deletions

File tree

README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,8 @@ set:
5252
items.count:
5353
short: You have {{count}} {{plural:count|item|items}}
5454
long: Total: {{num:amount}} generated at {{date:when}}
55+
person.cats:
56+
short: "{{plural:count|zero:No cats|one:One cat|other:{{count}} cats}}"
5557
```
5658
5759
Example `es.yaml`:
@@ -146,6 +148,7 @@ All fields of `msgcat.Config`:
146148
- **Template tokens (named parameters)**
147149
- `{{name}}` — simple substitution.
148150
- `{{plural:count|singular|plural}}` — binary plural by named count parameter.
151+
- `{{plural:count|one:item|few:items|many:items|other:items}}` — multi-form plural by named count parameter using CLDR rules (supports 0, 1, 2, few, many, other depending on language).
149152
- **CLDR plural forms** — optional `short_forms` / `long_forms` per entry (keys: `zero`, `one`, `two`, `few`, `many`, `other`) for full locale rules; see [CLDR and messages in Go](docs/CLDR_AND_GO_MESSAGES_PLAN.md).
150153
- `{{num:amount}}` — localized number for named parameter.
151154
- `{{date:when}}` — localized date for named parameter (`time.Time` or `*time.Time`).

docs/CLDR_AND_GO_MESSAGES_PLAN.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,20 @@
22

33
Plan to add **CLDR-style plural forms** and **messages defined in Go with extract** to msgcat, closing the gap with go-i18n.
44

5-
**Implementation status:** Done. Library has `ShortForms`/`LongForms`/`PluralParam` on `RawMessage`, `internal/plural` for form selection, `MessageDef` type, CLI extract finds MessageDef literals and merges into YAML, merge copies plural fields. See README, examples/cldr_plural, examples/msgdef.
5+
**Implementation status:** Done. Library has `ShortForms`/`LongForms`/`PluralParam` on `RawMessage`, **inline multi-form plural syntax** `{{plural:count|one:...|other:...}}`, `internal/plural` for form selection, `MessageDef` type, CLI extract finds MessageDef literals and merges into YAML, merge copies plural fields. See README, examples/cldr_plural, examples/msgdef.
66

77
---
88

99
## 1. CLDR plurals
1010

1111
### 1.1 Goal
1212

13-
Support the full set of CLDR plural categories (**zero**, **one**, **two**, **few**, **many**, **other**) so that languages with more than two forms (e.g. Arabic, Russian, Welsh) can have correct pluralization. Current msgcat only has binary `{{plural:count|singular|plural}}`.
13+
Support the full set of CLDR plural categories (**zero**, **one**, **two**, **few**, **many**, **other**) so that languages with more than two forms (e.g. Arabic, Russian, Welsh) can have correct pluralization. Support both optional entry-level maps and **inline multi-form tokens**.
1414

1515
### 1.2 Backward compatibility
1616

17-
- Keep existing **short** / **long** strings and **`{{plural:count|singular|plural}}`** unchanged. No breaking change.
17+
- Keep existing **short** / **long** strings and **`{{plural:count|singular|plural}}`** (binary) unchanged. No breaking change.
18+
- Add **inline multi-form support**: `{{plural:count|one:item|other:items}}` uses the language rule to pick a form.
1819
- Add **optional** plural form maps. When present, they are used instead of the binary plural token for that entry.
1920

2021
### 1.3 YAML format (optional plural forms)
@@ -162,7 +163,7 @@ type MessageDef struct {
162163

163164
## 4. Out of scope (for this plan)
164165

165-
- **Changing existing binary plural token:** `{{plural:count|singular|plural}}` stays as-is when CLDR forms are not used.
166+
- **Changing existing binary plural token:** `{{plural:count|singular|plural}}` stays as-is (backward compatible). Added **multi-form plural token** `{{plural:count|one:singular|other:plural|few:...}}` for inline CLDR rules.
166167
- **Runtime default message from Go:** Passing a MessageDef into GetMessageWithCtx as fallback when key is missing is a possible later extension.
167168
- **Hash/change detection** for merged translations (like goi18n) remains out of scope.
168169

examples/cldr_plural/main.go

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@ set:
3131
one: "{{name}} has one cat."
3232
other: "{{name}} has {{count}} cats."
3333
plural_param: count
34+
person.dogs:
35+
short: "{{name}} has {{plural:count|zero:no dogs|one:one dog|other:{{count}} dogs}}"
3436
`)
3537
if err := os.WriteFile(filepath.Join(dir, "en.yaml"), en, 0o600); err != nil {
3638
panic(err)
@@ -45,6 +47,26 @@ set:
4547

4648
for _, count := range []int{0, 1, 2, 5} {
4749
msg := catalog.GetMessageWithCtx(ctx, "person.cats", msgcat.Params{"name": "Nick", "count": count})
48-
fmt.Printf("count=%d: %s\n", count, msg.ShortText)
50+
fmt.Printf("cats count=%d: %s\n", count, msg.ShortText)
51+
msgDog := catalog.GetMessageWithCtx(ctx, "person.dogs", msgcat.Params{"name": "Nick", "count": count})
52+
fmt.Printf("dogs count=%d: %s\n", count, msgDog.ShortText)
53+
}
54+
55+
ar := []byte(`default:
56+
short: خطأ
57+
long: خطأ
58+
set:
59+
person.dogs:
60+
short: "{{name}} لديه {{plural:count|zero:لا كلاب|one:كلب واحد|two:كلبان|few:{{count}} كلاب|many:{{count}} كلباً|other:{{count}} كلب}}"
61+
`)
62+
if err := os.WriteFile(filepath.Join(dir, "ar.yaml"), ar, 0o600); err != nil {
63+
panic(err)
64+
}
65+
msgcat.Reload(catalog)
66+
67+
ctxAR := context.WithValue(context.Background(), "language", "ar")
68+
for _, count := range []int{0, 1, 2, 5, 11, 100} {
69+
msgDog := catalog.GetMessageWithCtx(ctxAR, "person.dogs", msgcat.Params{"name": "Nick", "count": count})
70+
fmt.Printf("AR dogs count=%d: %s\n", count, msgDog.ShortText)
4971
}
5072
}

internal/plural/plural.go

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,11 +25,13 @@ func Form(lang string, count int) string {
2525
return formRussian(n)
2626
case "pl":
2727
return formPolish(n)
28+
case "fr", "pt", "oc", "it":
29+
return formFrench(n)
2830
case "cy", "br", "ga", "gd", "gv", "kw", "mt", "sm", "ak":
2931
return formWelsh(n)
3032
case "he", "iw":
3133
return formHebrew(n)
32-
case "en", "es", "fr", "de", "it", "pt", "nl", "no", "sv", "da", "fi", "tr", "el", "ja", "ko", "zh", "th", "vi", "id", "hi":
34+
case "en", "es", "de", "nl", "no", "sv", "da", "fi", "tr", "el", "ja", "ko", "zh", "th", "vi", "id", "hi":
3335
return formOneOther(n)
3436
default:
3537
return "other"
@@ -43,6 +45,13 @@ func formOneOther(n int) string {
4345
return "other"
4446
}
4547

48+
func formFrench(n int) string {
49+
if n == 0 || n == 1 {
50+
return "one"
51+
}
52+
return "other"
53+
}
54+
4655
func formArabic(n int) string {
4756
if n == 0 {
4857
return "zero"

internal/plural/plural_test.go

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,9 @@ func TestForm(t *testing.T) {
1414
{"en-US", 1, "one"},
1515
{"es", 1, "one"},
1616
{"es", 5, "other"},
17+
{"fr", 0, "one"},
18+
{"fr", 1, "one"},
19+
{"fr", 2, "other"},
1720
{"ar", 0, "zero"},
1821
{"ar", 1, "one"},
1922
{"ar", 2, "two"},

msgcat.go

Lines changed: 41 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ var messageKeyRegex = regexp.MustCompile(`^[a-zA-Z0-9_.-]+$`)
3131

3232
var (
3333
simplePlaceholderRegex = regexp.MustCompile(`\{\{([a-zA-Z_][a-zA-Z0-9_.]*)\}\}`)
34-
pluralPlaceholderRegex = regexp.MustCompile(`\{\{plural:([a-zA-Z_][a-zA-Z0-9_.]*)\|([^|}]*)\|([^}]*)\}\}`)
34+
pluralPlaceholderRegex = regexp.MustCompile(`\{\{plural:([a-zA-Z_][a-zA-Z0-9_.]*)\|((?:[^{}]|\{\{.*?\}\})*)\}\}`)
3535
numberPlaceholderRegex = regexp.MustCompile(`\{\{num:([a-zA-Z_][a-zA-Z0-9_.]*)\}\}`)
3636
datePlaceholderRegex = regexp.MustCompile(`\{\{date:([a-zA-Z_][a-zA-Z0-9_.]*)\}\}`)
3737
)
@@ -388,13 +388,13 @@ func selectCLDRForm(forms map[string]string, lang string, count int, defaultTpl
388388
return defaultTpl
389389
}
390390

391-
// parsePluralTokenNamed extracts param name, singular and plural from {{plural:name|singular|plural}}.
392-
func parsePluralTokenNamed(token string) (paramName string, singular string, plural string, ok bool) {
391+
// parsePluralToken extracts param name and content from {{plural:name|content}}.
392+
func parsePluralToken(token string) (paramName string, content string, ok bool) {
393393
matches := pluralPlaceholderRegex.FindStringSubmatch(token)
394-
if len(matches) != 4 {
395-
return "", "", "", false
394+
if len(matches) != 3 {
395+
return "", "", false
396396
}
397-
return matches[1], matches[2], matches[3], true
397+
return matches[1], matches[2], true
398398
}
399399

400400
func toString(value interface{}) string {
@@ -722,23 +722,53 @@ func (dmc *DefaultMessageCatalog) renderTemplate(lang string, msgKey string, tem
722722
}
723723

724724
rendered = pluralPlaceholderRegex.ReplaceAllStringFunc(rendered, func(token string) string {
725-
paramName, singular, plural, ok := parsePluralTokenNamed(token)
725+
paramName, content, ok := parsePluralToken(token)
726726
if !ok {
727727
return token
728728
}
729729
val, ok := getParam(paramName)
730730
if !ok {
731731
return replaceMissing("plural_missing_param_"+paramName, token, paramName)
732732
}
733-
isOne, ok := isPluralOne(val)
733+
count, ok := pluralCountFromParam(val)
734734
if !ok {
735735
dmc.onTemplateIssue(lang, msgKey, "plural_invalid_param_"+paramName)
736736
return token
737737
}
738-
if isOne {
739-
return singular
738+
739+
parts := strings.Split(content, "|")
740+
// Case 1: Legacy binary plural: {{plural:count|singular|plural}}
741+
if len(parts) == 2 && !strings.Contains(parts[0], ":") {
742+
isOne, _ := isPluralOne(val)
743+
if isOne {
744+
return parts[0]
745+
}
746+
return parts[1]
747+
}
748+
749+
// Case 2: Multi-form plural: {{plural:count|one:singular|few:items|many:items|other:items}}
750+
forms := make(map[string]string)
751+
for _, part := range parts {
752+
if idx := strings.Index(part, ":"); idx > 0 {
753+
formName := strings.TrimSpace(part[:idx])
754+
formValue := part[idx+1:]
755+
forms[formName] = formValue
756+
}
740757
}
741-
return plural
758+
759+
if len(forms) > 0 {
760+
form := plural.Form(lang, count)
761+
if tpl, ok := forms[form]; ok {
762+
return tpl
763+
}
764+
if tpl, ok := forms["other"]; ok {
765+
return tpl
766+
}
767+
// fallback to first form if no match
768+
return parts[0]
769+
}
770+
771+
return token
742772
})
743773

744774
rendered = numberPlaceholderRegex.ReplaceAllStringFunc(rendered, func(token string) string {

test/suites/msgcat/msgcat_test.go

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -260,6 +260,53 @@ var _ = Describe("Message Catalog", func() {
260260
Expect(msgES.LongText).To(Equal("Total: 12.345,5 generado el 03/01/2026"))
261261
})
262262

263+
It("should support multi-form plurals and advanced language rules", func() {
264+
// Test English: 1 -> one, 2 -> other
265+
err := messageCatalog.LoadMessages("en", []msgcat.RawMessage{{
266+
Key: "sys.multi",
267+
ShortTpl: "{{plural:count|one:one item|other:{{count}} items}}",
268+
}})
269+
Expect(err).NotTo(HaveOccurred())
270+
271+
msgEN1 := messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 1})
272+
Expect(msgEN1.ShortText).To(Equal("one item"))
273+
274+
msgEN2 := messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 2})
275+
Expect(msgEN2.ShortText).To(Equal("2 items"))
276+
277+
// Test French: 0 -> one, 1 -> one, 2 -> other
278+
err = messageCatalog.LoadMessages("fr", []msgcat.RawMessage{{
279+
Key: "sys.multi",
280+
ShortTpl: "{{plural:count|one:un item|other:{{count}} items}}",
281+
}})
282+
Expect(err).NotTo(HaveOccurred())
283+
284+
ctx.SetValue("language", "fr")
285+
msgFR0 := messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 0})
286+
Expect(msgFR0.ShortText).To(Equal("un item"))
287+
288+
msgFR1 := messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 1})
289+
Expect(msgFR1.ShortText).To(Equal("un item"))
290+
291+
msgFR2 := messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 2})
292+
Expect(msgFR2.ShortText).To(Equal("2 items"))
293+
294+
// Test Arabic: 0 -> zero, 1 -> one, 2 -> two, 3 -> few, 11 -> many, 100 -> other
295+
err = messageCatalog.LoadMessages("ar", []msgcat.RawMessage{{
296+
Key: "sys.multi",
297+
ShortTpl: "{{plural:count|zero:zero|one:one|two:two|few:few|many:many|other:other}}",
298+
}})
299+
Expect(err).NotTo(HaveOccurred())
300+
301+
ctx.SetValue("language", "ar")
302+
Expect(messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 0}).ShortText).To(Equal("zero"))
303+
Expect(messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 1}).ShortText).To(Equal("one"))
304+
Expect(messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 2}).ShortText).To(Equal("two"))
305+
Expect(messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 5}).ShortText).To(Equal("few"))
306+
Expect(messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 11}).ShortText).To(Equal("many"))
307+
Expect(messageCatalog.GetMessageWithCtx(ctx.Ctx, "sys.multi", msgcat.Params{"count": 100}).ShortText).To(Equal("other"))
308+
})
309+
263310
It("should support strict template checks and report template issues", func() {
264311
observer := &mockObserver{}
265312
strictCatalog, err := msgcat.NewMessageCatalog(msgcat.Config{

0 commit comments

Comments
 (0)