Skip to content

Add support for replication service#380

Merged
idanovo merged 7 commits intomasterfrom
feat/add-replication-2
Mar 2, 2026
Merged

Add support for replication service#380
idanovo merged 7 commits intomasterfrom
feat/add-replication-2

Conversation

@idanovo
Copy link
Contributor

@idanovo idanovo commented Feb 25, 2026

No description provided.

@idanovo idanovo requested a review from guy-har February 25, 2026 15:06
Copy link
Contributor

@guy-har guy-har left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments, overall looks
Mainly blocking because:

  1. I think we should use the next image version (which has many fixes) before enabling replicatoin
  2. Just validate replication is supported/recomended to work with more than replica

nit: can we move the replication files into a directory like done in mds?

What about rediness and liveness probes? maybe we should have a follow up issue

# https://docs.lakefs.io/latest/howto/mirroring/
replication:
enabled: false
replicas: 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it recommended to have more than one replication replica? If not let's not make it configurable, if there is a warning, let's add it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea TBH, never tested it with more than one replica.
It's configurable- the user will have to override replication.replicas

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that increasing reources make more sense than increasing replica count here

type: application
version: 1.7.20
version: 1.7.21
appVersion: 1.78.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should wait for the next version, which should be released today/tomorrow. WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's necessary
I don't want to be a blocker for whoever is going to release a new version.
Replication app version can always be changed by the user or us

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can set the default tag to be 0.1.17, which will be released tomorrow instead

Comment on lines +40 to +44
- name: REPLICATION_LAKEFS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: {{ .Values.replication.extraEnvVarsSecret }}
key: source_lakefs_access_key_id
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this should a env var or a secret, but it's fine like this

@idanovo
Copy link
Contributor Author

idanovo commented Mar 1, 2026

Few comments, overall looks Mainly blocking because:

  1. I think we should use the next image version (which has many fixes) before enabling replicatoin
  2. Just validate replication is supported/recomended to work with more than replica

nit: can we move the replication files into a directory like done in mds?

What about rediness and liveness probes? maybe we should have a follow up issue

@guy-har

  1. I don't think we should, I set the default version to be 0.1.17, which will be released today/tomorrow
  2. AFAIU, it's supported but wouldn't make any difference as both replicas will do the same things in the same order.
  3. I moved replication to a new dir, thanks!
  4. Add rediness and liveness probes, thanks!!

@idanovo idanovo requested a review from guy-har March 1, 2026 13:17
@idanovo idanovo merged commit 765d131 into master Mar 2, 2026
4 checks passed
@idanovo idanovo deleted the feat/add-replication-2 branch March 2, 2026 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants