-
Notifications
You must be signed in to change notification settings - Fork 28
CRE-1601: Ring OCR plugin for shard orchestration #1742
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
51 commits
Select commit
Hold shift + click to select a range
9c4741b
cre-1601: shard orchestrator plugin for delegate
mchain0 de0d9f0
cre-1601: consistent hashing and plugin test
mchain0 045f5ec
cre-1601: tidy
mchain0 6ae609e
cre-1601: pb generate
mchain0 adaf093
cre-1601: review improvement
mchain0 93a8734
cre-1601: review improvement
mchain0 38db9f4
cre-1601: review improvement
mchain0 e3e2e02
cre-1601: remove previous outcome, use outctx.SeqNr instead
mchain0 4b0684f
cre-1601: transition state machine
mchain0 36252b0
cre-1601: removed TransmissionScheduleOverride
mchain0 0553e74
cre-1601: comments
mchain0 cec9228
cre-1601: rename plugin to ring
mchain0 6d1b057
cre-1601: renames and cleanup
mchain0 d848886
cre-1601: delegate integrations adjustments
mchain0 65f2e1d
cre-1601: proto comments; import fix
mchain0 a0601eb
cre-1601: snake_case consistent for proto fields
mchain0 b9e70a7
cre-1601: remove unused field
mchain0 790bc5d
cre-1601: more proto comments
mchain0 7be5eb2
cre-1601: proto cleanup, orphans removed
mchain0 ffe484b
cre-1601: log overrides, log plugin config
mchain0 26f8c8c
cre-1601: better comment
mchain0 cf0bb98
cre-1601: deterministic time; f check for round; improved time median…
mchain0 2f2e45d
cre-1601: log improvement
mchain0 d7db767
cre-1601: shard count health refactor
mchain0 effb57a
cre-1601: hash ring pure function refactor for both storage and obser…
mchain0 18161c1
cre-1601: Transmitter notifies Arbiter
mchain0 f455bc0
cre-1601: store in two states, steady and transition; enque for alloc…
mchain0 9198b9c
cre-1601: comments improved
mchain0 457af17
cre-1601: more comments improvements
mchain0 9599327
cre-1601: test extension to validate workflows to shards eassignments
mchain0 1c53fef
cre-1601: test improvement - distribution check by percents
mchain0 15465d1
cre-1601: using maps improvement
mchain0 5022467
cre-1601: remove number of shards limits (ref. review)
mchain0 287bf71
cre-1601: initial state fix; intial transition state until OCR round;…
mchain0 c8cd5fe
Merge branch 'main' into cre-1601-shard-orchestrator-plugin
mchain0 df7e4e4
cre-1601: more tests; boosting test coverage
mchain0 600d035
cre-1601: better comments
mchain0 0bd0aaf
cre-1601: better comments
mchain0 a1e8307
cre-1601: improved test for plugin outcome
mchain0 1e3f43b
cre-1601: comments improved
mchain0 57f0423
cre-1601: comment improvement
mchain0 e042bb2
cre-1601: more tests; more coverage
mchain0 26cbc3b
cre-1601: refactor of state; state verification tests; related changes;
mchain0 29b4072
cre-1601: bool wrapper ShardStatus to extend with weights later
mchain0 693eb34
cre-1601: ArbiterScaler.Status() wantShards being part of the observa…
mchain0 3198932
cre-1601: improved comments; improved naming; simplified logic; obser…
mchain0 16f7555
cre-1601: on 1st round make prior outcome wantShards equal to the cur…
mchain0 dbc586d
cre-1601: removed unnecessary check (code review suggestions)
mchain0 5b1c2a1
Merge branch 'main' into cre-1601-shard-orchestrator-plugin
mchain0 eaec50d
Fail Observation() on Arbiter error
bolekk c6c46c9
Merge branch 'main' into cre-1601-shard-orchestrator-plugin
mchain0 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| package ring | ||
|
|
||
| import ( | ||
| "context" | ||
| "errors" | ||
|
|
||
| "github.com/smartcontractkit/chainlink-common/pkg/logger" | ||
| "github.com/smartcontractkit/chainlink-common/pkg/services" | ||
| "github.com/smartcontractkit/chainlink-common/pkg/types/core" | ||
| "github.com/smartcontractkit/chainlink-common/pkg/workflows/ring/pb" | ||
| "github.com/smartcontractkit/libocr/offchainreporting2plus/ocr3types" | ||
| ) | ||
|
|
||
| const ( | ||
| defaultMaxPhaseOutputBytes = 1000000 // 1 MB | ||
| defaultMaxReportCount = 1 | ||
| defaultBatchSize = 100 | ||
| ) | ||
|
|
||
| var _ core.OCR3ReportingPluginFactory = &Factory{} | ||
|
|
||
| type Factory struct { | ||
| store *Store | ||
| arbiterScaler pb.ArbiterScalerClient | ||
| config *ConsensusConfig | ||
| lggr logger.Logger | ||
|
|
||
| services.StateMachine | ||
| } | ||
|
|
||
| func NewFactory(s *Store, arbiterScaler pb.ArbiterScalerClient, lggr logger.Logger, cfg *ConsensusConfig) (*Factory, error) { | ||
| if arbiterScaler == nil { | ||
| return nil, errors.New("arbiterScaler is required") | ||
| } | ||
| if cfg == nil { | ||
| cfg = &ConsensusConfig{ | ||
| BatchSize: defaultBatchSize, | ||
| } | ||
| } | ||
| return &Factory{ | ||
| store: s, | ||
| arbiterScaler: arbiterScaler, | ||
| config: cfg, | ||
| lggr: logger.Named(lggr, "RingPluginFactory"), | ||
| }, nil | ||
| } | ||
|
|
||
| func (o *Factory) NewReportingPlugin(_ context.Context, config ocr3types.ReportingPluginConfig) (ocr3types.ReportingPlugin[[]byte], ocr3types.ReportingPluginInfo, error) { | ||
| plugin, err := NewPlugin(o.store, o.arbiterScaler, config, o.lggr, o.config) | ||
| pluginInfo := ocr3types.ReportingPluginInfo{ | ||
| Name: "RingPlugin", | ||
| Limits: ocr3types.ReportingPluginLimits{ | ||
| MaxQueryLength: defaultMaxPhaseOutputBytes, | ||
| MaxObservationLength: defaultMaxPhaseOutputBytes, | ||
| MaxOutcomeLength: defaultMaxPhaseOutputBytes, | ||
| MaxReportLength: defaultMaxPhaseOutputBytes, | ||
| MaxReportCount: defaultMaxReportCount, | ||
| }, | ||
| } | ||
| return plugin, pluginInfo, err | ||
| } | ||
|
|
||
| func (o *Factory) Start(ctx context.Context) error { | ||
| return o.StartOnce("RingPlugin", func() error { | ||
| return nil | ||
| }) | ||
| } | ||
|
|
||
| func (o *Factory) Close() error { | ||
| return o.StopOnce("RingPlugin", func() error { | ||
| return nil | ||
| }) | ||
| } | ||
|
|
||
| func (o *Factory) Name() string { return o.lggr.Name() } | ||
|
|
||
| func (o *Factory) HealthReport() map[string]error { | ||
| return map[string]error{o.Name(): o.Healthy()} | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| package ring | ||
|
|
||
| import ( | ||
| "context" | ||
| "testing" | ||
|
|
||
| "github.com/smartcontractkit/chainlink-common/pkg/logger" | ||
| "github.com/smartcontractkit/libocr/offchainreporting2plus/ocr3types" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| func TestFactory_NewFactory(t *testing.T) { | ||
| lggr := logger.Test(t) | ||
| store := NewStore() | ||
| arbiter := &mockArbiter{} | ||
|
|
||
| t.Run("with_nil_config", func(t *testing.T) { | ||
| f, err := NewFactory(store, arbiter, lggr, nil) | ||
| require.NoError(t, err) | ||
| require.NotNil(t, f) | ||
| }) | ||
|
|
||
| t.Run("with_custom_config", func(t *testing.T) { | ||
| cfg := &ConsensusConfig{BatchSize: 50} | ||
| f, err := NewFactory(store, arbiter, lggr, cfg) | ||
| require.NoError(t, err) | ||
| require.NotNil(t, f) | ||
| }) | ||
|
|
||
| t.Run("nil_arbiter_returns_error", func(t *testing.T) { | ||
| _, err := NewFactory(store, nil, lggr, nil) | ||
| require.Error(t, err) | ||
| require.Contains(t, err.Error(), "arbiterScaler is required") | ||
| }) | ||
| } | ||
|
|
||
| func TestFactory_NewReportingPlugin(t *testing.T) { | ||
| lggr := logger.Test(t) | ||
| store := NewStore() | ||
| f, err := NewFactory(store, &mockArbiter{}, lggr, nil) | ||
| require.NoError(t, err) | ||
|
|
||
| config := ocr3types.ReportingPluginConfig{N: 4, F: 1} | ||
| plugin, info, err := f.NewReportingPlugin(context.Background(), config) | ||
| require.NoError(t, err) | ||
| require.NotNil(t, plugin) | ||
| require.NotEmpty(t, info.Name) | ||
| require.Equal(t, "RingPlugin", info.Name) | ||
| require.Equal(t, defaultMaxReportCount, info.Limits.MaxReportCount) | ||
| } | ||
|
|
||
| func TestFactory_Lifecycle(t *testing.T) { | ||
| lggr := logger.Test(t) | ||
| store := NewStore() | ||
| f, err := NewFactory(store, &mockArbiter{}, lggr, nil) | ||
| require.NoError(t, err) | ||
|
|
||
| err = f.Start(context.Background()) | ||
| require.NoError(t, err) | ||
|
|
||
| name := f.Name() | ||
| require.NotEmpty(t, name) | ||
|
|
||
| report := f.HealthReport() | ||
| require.NotNil(t, report) | ||
| require.Contains(t, report, name) | ||
|
|
||
| err = f.Close() | ||
| require.NoError(t, err) | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.