🐛 Fix deprecation conditions showing install errors and improve condition semantics#2296
Conversation
✅ Deploy Preview for olmv1 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
There was a problem hiding this comment.
Pull Request Overview
This PR refactors deprecation status handling in ClusterExtension reconciliation to ensure deprecation conditions accurately reflect catalog data and installed bundle state, preventing install/validation errors from leaking into deprecation conditions.
- Moved deprecation status updates to a deferred function that runs at the end of reconciliation
- Changed BundleDeprecated condition to use
Unknownstatus withReasonAbsentwhen no bundle is installed - Updated test expectations to handle multiple possible error messages for sourceType validation
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| internal/operator-controller/controllers/clusterextension_controller.go | Refactored deprecation status logic with deferred updates and new condition semantics for uninstalled bundles |
| internal/operator-controller/controllers/clusterextension_controller_test.go | Added tests for deprecation handling with resolution failures and applier failures; updated test expectations for BundleDeprecated conditions |
| test/e2e/cluster_extension_install_test.go | Updated e2e tests to verify deprecation conditions in success and failure scenarios |
| internal/operator-controller/controllers/clusterextension_admission_test.go | Enhanced sourceType validation test to handle multiple possible error message formats |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
0891f37 to
4f61e6a
Compare
4f61e6a to
2b26273
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2296 +/- ##
==========================================
+ Coverage 69.48% 74.02% +4.53%
==========================================
Files 101 101
Lines 7891 7991 +100
==========================================
+ Hits 5483 5915 +432
+ Misses 1969 1626 -343
- Partials 439 450 +11
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
2b26273 to
ad9ccfd
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
internal/operator-controller/controllers/clusterextension_controller_test.go:1
- The assertion
require.Contains(ct, []string{...}, cond.Reason)is semantically backwards. TheContainsfunction expects a slice as the second argument and the search item as the first. This should berequire.Contains(ct, cond.Reason, []string{ocv1.ReasonFailed, ocv1.ReasonAbsent})or userequire.True(ct, slices.Contains([]string{ocv1.ReasonFailed, ocv1.ReasonAbsent}, cond.Reason)).
package controllers_test
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ad9ccfd to
a6d0678
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
a6d0678 to
287bede
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
287bede to
b2da76f
Compare
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Not sure if this has been addressed in other comments (if so, please update the PR description!). I noticed that we have
|
|
One thing I didn't see answered in the description is the open question in the original TODO comment about what happens when the package and/or channel is present in multiple catalogs, but we didn't resolve a bundle:
|
|
Is there an RFC for this? It kinda feels like we should have something written up in that form? I also wonder if we should tackle the question of whether to explicitly include the Deprecation conditions when we would set them as False. They add a lot of noise to the YAML/JSON and |
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Hi @joelanford Thanks for the feedback! You're absolutely right - we should keep the TODO and open question for item 2. I've done that. See the updated TODO with scenarios here: internal/operator-controller/controllers/clusterextension_controller.go The core logic for handling multiple catalogs hasn't changed - the resolver still picks one. What changed is we now correctly handle the failure cases and don't lose deprecation information when things go wrong. For that yes, I think we need a follow up PR/discussion. ( out of scope ) Just to clarify the scope: this PR solves item 1 (use installed bundle instead of resolved) and #2008 (install errors leaking into deprecation conditions). The open question about conflicting deprecations from multiple catalogs is a separate edge case that deserves its own focused discussion and PR (IHMO, lets go step by step). Key Differences (What This PR Fixes)
Hope this addresses your concern! Let me know if you'd like me to clarify anything else. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Hey @joelanford — all your comments have been addressed. We also have an RFC here: https://docs.google.com/document/d/1_XtISZ74rsd9-Zh9oQje1mEHjAfx0CHlf_50UJkXHTQ I think it’s ready to 🚀 now. |
| if installedBundleName != "" { | ||
| bundleReason = ocv1.ReasonDeprecationStatusUnknown | ||
| } | ||
| setDeprecationCondition(ext, ocv1.TypeDeprecated, metav1.ConditionUnknown, ocv1.ReasonDeprecationStatusUnknown, "") |
There was a problem hiding this comment.
should not we put in Reason some explanation why the status is equal to Unknown? Something like NoCatalogData? And also add the appropriate human readable message, instead of the empty string?
There was a problem hiding this comment.
It seems a good idea. Added
# When catalog is unavailable:
- type: PackageDeprecated
status: Unknown
reason: DeprecationStatusUnknown
message: "deprecation status unknown: catalog data unavailable"
# When no bundle installed:
- type: BundleDeprecated
status: Unknown
reason: Absent
message: "no bundle installed yet"
| return info | ||
| } | ||
|
|
||
| // setDeprecationCondition sets a single deprecation condition with less boilerplate. |
There was a problem hiding this comment.
where do we save some boilerplate? setDeprecationCondtion is literary a wrapper around SetStatusCondition. IMHO, I see no added value,
There was a problem hiding this comment.
The diff is that we do not need add in all places
SetStatusCondition(&ext.Status.Conditions, metav1.Condition{
Type: condType,
Status: status,
Reason: reason,
Message: message,
ObservedGeneration: ext.GetGeneration(),
})
Example
SetStatusCondition(&ext.Status.Conditions, metav1.Condition{
Type: ocv1.TypeDeprecated,
Status: metav1.ConditionUnknown,
Reason: ocv1.ReasonDeprecationStatusUnknown,
Message: "deprecation status unknown: catalog data unavailable",
ObservedGeneration: ext.GetGeneration(),
})
SetStatusCondition(&ext.Status.Conditions, metav1.Condition{
Type: ocv1.TypePackageDeprecated,
Status: metav1.ConditionUnknown,
Reason: ocv1.ReasonDeprecationStatusUnknown,
Message: "deprecation status unknown: catalog data unavailable",
ObservedGeneration: ext.GetGeneration(),
})
SetStatusCondition(&ext.Status.Conditions, metav1.Condition{
Type: ocv1.TypeChannelDeprecated,
Status: metav1.ConditionUnknown,
Reason: ocv1.ReasonDeprecationStatusUnknown,
Message: "deprecation status unknown: catalog data unavailable",
ObservedGeneration: ext.GetGeneration(),
})
SetStatusCondition(&ext.Status.Conditions, metav1.Condition{
Type: ocv1.TypeBundleDeprecated,
Status: metav1.ConditionUnknown,
Reason: bundleReason,
Message: bundleMessage,
ObservedGeneration: ext.GetGeneration(),
})
instead we have
setDeprecationCondition(ext, ocv1.TypeDeprecated, metav1.ConditionUnknown, ocv1.ReasonDeprecationStatusUnknown, "deprecation status unknown: catalog data unavailable")
setDeprecationCondition(ext, ocv1.TypePackageDeprecated, metav1.ConditionUnknown, ocv1.ReasonDeprecationStatusUnknown, "deprecation status unknown: catalog data unavailable")
setDeprecationCondition(ext, ocv1.TypeChannelDeprecated, metav1.ConditionUnknown, ocv1.ReasonDeprecationStatusUnknown, "deprecation status unknown: catalog data unavailable")
setDeprecationCondition(ext, ocv1.TypeBundleDeprecated, metav1.ConditionUnknown, bundleReason, bundleMessage)
But if you do not see value in the helper I have to keep the directly call.
DONE 👍
| } else { | ||
| resolvedRevisionMetadata = state.revisionStates.RollingOut[0] | ||
| // Resolve a new bundle from the catalog | ||
| l.Info("resolving bundle") |
There was a problem hiding this comment.
can we have this on debug level?
| } | ||
|
|
||
| longDeprecationMsg := strings.Join(deprecationMessages, "; ") | ||
| longDeprecationMsg := strings.Join(deprecationMessages, "\n") |
There was a problem hiding this comment.
why is this change necessary?
There was a problem hiding this comment.
On main (before):
- Controller code:
strings.Join(deprecationMessages, ";")← no space - Test code:
strings.Join(deprecationMessages, "; ")← has space (already inconsistent!)
In this PR (after):
- Controller code:
strings.Join(...Messages, "\n") - Test code:
strings.Join(deprecationMessages, "\n") - Both now use newlines ✅
Note that, this test builds one long deprecation message to check truncation. The test needs to use the same separator as the controller code to accurately validate truncation behavior.
On main, they were already inconsistent (";" vs "; "). This PR fixes that and improves readability by using newlines.
Why newlines are better
- With semicolons (hard to read):
- type: PackageDeprecated
status: "True"
message: "package 'foo' is deprecated;channel 'alpha' is deprecated;bundle 'bar' is deprecated"- With newlines (clearer):
- type: PackageDeprecated
status: "True"
message: |
package 'foo' is deprecated
channel 'alpha' is deprecated
bundle 'bar' is deprecatedEach warning on its own line - much easier to read in kubectl describe. And k8s API conventions recommend human-readable condition messages. Multiple issues should be clearly distinguishable. Also, as far I understand other k8s resources (Pods, Events, Jobs) all use newlines for multi-line messages, not semicolons.
**PS:**If I properly remember, the newline change was suggested by Copilot during development.
| var bm *ocv1.BundleMetadata | ||
| if state.revisionStates.Installed != nil { | ||
| bm = &state.revisionStates.Installed.BundleMetadata | ||
| } | ||
| resolvedBundle, resolvedBundleVersion, resolvedDeprecation, err := r.Resolve(ctx, ext, bm) | ||
|
|
||
| // Get the installed bundle name for deprecation status. | ||
| // BundleDeprecated should reflect what's currently running, not what we're trying to install. | ||
| installedBundleName := "" | ||
| if state.revisionStates.Installed != nil { | ||
| installedBundleName = state.revisionStates.Installed.Name | ||
| } | ||
|
|
||
| // Set deprecation status based on resolution results: | ||
| // - If resolution succeeds: hasCatalogData=true, deprecation shows catalog data (nil=not deprecated) | ||
| // - If resolution fails but returns deprecation: hasCatalogData=true, show package/channel deprecation warnings | ||
| // - If resolution fails with nil deprecation: hasCatalogData=false, all conditions go Unknown | ||
| // | ||
| // TODO: Open question - what if different catalogs have different opinions of what's deprecated? | ||
| // If we can't resolve a bundle, how do we know which catalog to trust for deprecation information? | ||
| // Perhaps if the package shows up in multiple catalogs and deprecations don't match, we can set | ||
| // the deprecation status to unknown? Or perhaps we somehow combine the deprecation information from | ||
| // all catalogs? This needs a follow-up discussion and PR. | ||
| hasCatalogData := err == nil || resolvedDeprecation != nil | ||
| SetDeprecationStatus(ext, installedBundleName, resolvedDeprecation, hasCatalogData) | ||
|
|
||
| if err != nil { | ||
| // Note: We don't distinguish between resolution-specific errors and generic errors | ||
| setStatusProgressing(ext, err) | ||
| setInstalledStatusFromRevisionStates(ext, state.revisionStates) | ||
| ensureFailureConditionsWithReason(ext, ocv1.ReasonFailed, err.Error()) | ||
| return nil, err | ||
| } | ||
|
|
||
| state.resolvedRevisionMetadata = &RevisionMetadata{ | ||
| Package: resolvedBundle.Package, | ||
| Image: resolvedBundle.Image, | ||
| // TODO: Right now, operator-controller only supports registry+v1 bundles and has no concept | ||
| // of a "release" field. If/when we add a release field concept or a new bundle format |
There was a problem hiding this comment.
what tests are covering the added logic?
There was a problem hiding this comment.
Asked for CLAUDE answered that ;-P
Following its answer
Test Coverage for Added Deprecation Logic
Changes: Main vs This PR
What changed in clusterextension_reconcile_steps.go
On main branch:
// main branch
l.Info("resolving bundle")
var bm *ocv1.BundleMetadata
if state.revisionStates.Installed != nil {
bm = &state.revisionStates.Installed.BundleMetadata
}
resolvedBundle, resolvedBundleVersion, resolvedDeprecation, err := r.Resolve(ctx, ext, bm)
if err != nil {
// return error immediately - no deprecation set
setStatusProgressing(ext, err)
setInstalledStatusFromRevisionStates(ext, state.revisionStates)
ensureAllConditionsWithReason(ext, ocv1.ReasonFailed, err.Error())
return nil, err
}
// Only set deprecation after successful resolution
// TODO: deprecation status should reflect currently installed bundle, not resolved bundle
SetDeprecationStatus(ext, resolvedBundle.Name, resolvedDeprecation) // ← uses RESOLVED bundle name
resolvedRevisionMetadata = &RevisionMetadata{
Package: resolvedBundle.Package,
Image: resolvedBundle.Image,
BundleMetadata: bundleutil.MetadataFor(resolvedBundle.Name, resolvedBundleVersion.AsLegacyRegistryV1Version()),
}In this PR:
// This PR
l.V(1).Info("resolving bundle") // ← changed to debug level
var bm *ocv1.BundleMetadata
if state.revisionStates.Installed != nil {
bm = &state.revisionStates.Installed.BundleMetadata
}
resolvedBundle, resolvedBundleVersion, resolvedDeprecation, err := r.Resolve(ctx, ext, bm)
// Get the installed bundle name for deprecation status.
// BundleDeprecated should reflect what's currently running, not what we're trying to install.
installedBundleName := ""
if state.revisionStates.Installed != nil {
installedBundleName = state.revisionStates.Installed.Name // ← uses INSTALLED bundle name
}
// Set deprecation status based on resolution results:
// - If resolution succeeds: hasCatalogData=true, deprecation shows catalog data (nil=not deprecated)
// - If resolution fails but returns deprecation: hasCatalogData=true, show package/channel deprecation warnings
// - If resolution fails with nil deprecation: hasCatalogData=false, all conditions go Unknown
hasCatalogData := err == nil || resolvedDeprecation != nil // ← NEW logic
SetDeprecationStatus(ext, installedBundleName, resolvedDeprecation, hasCatalogData) // ← Set BEFORE error check
if err != nil {
// deprecation already set above
setStatusProgressing(ext, err)
setInstalledStatusFromRevisionStates(ext, state.revisionStates)
ensureFailureConditionsWithReason(ext, ocv1.ReasonFailed, err.Error())
return nil, err
}
state.resolvedRevisionMetadata = &RevisionMetadata{
Package: resolvedBundle.Package,
Image: resolvedBundle.Image,
BundleMetadata: bundleutil.MetadataFor(resolvedBundle.Name, resolvedBundleVersion.AsLegacyRegistryV1Version()),
}Key Changes
| Change | Main | This PR |
|---|---|---|
| Bundle name used | resolvedBundle.Name (what we're trying to install) |
installedBundleName (what's currently running) |
| When set | After successful resolution only | Before error check (always) |
| Catalog unavailable | No deprecation set (conditions stay False) | Deprecation set to Unknown |
| hasCatalogData logic | Not present | err == nil || resolvedDeprecation != nil |
| Log level | l.Info("resolving bundle") |
l.V(1).Info("resolving bundle") |
Test Coverage
New Tests Added in This PR
File: clusterextension_controller_test.go
1. TestClusterExtensionResolutionFailsWithDeprecationData
- Line: 191
- Covers:
hasCatalogData = truewhenerr != nil && resolvedDeprecation != nil- Deprecation shown even when resolution fails
- Package/channel deprecation visible despite installation failure
2. TestClusterExtensionUpgradeShowsInstalledBundleDeprecation
- Line: 254
- Covers:
- Uses installed bundle name (not resolved)
- v1.0.0 installed (deprecated), v2.0.0 resolved (not deprecated)
- BundleDeprecated shows True for v1.0.0 (actual state, not desired)
- This is the key behavioral change
3. TestClusterExtensionResolutionFailsWithoutCatalogDeprecationData
- Line: 352
- Covers:
hasCatalogData = falsewhenerr != nil && resolvedDeprecation == nil- All deprecation conditions → Unknown (not False)
- Uses installed bundle name when bundle exists
4. TestClusterExtensionBoxcutterApplierFailsDoesNotLeakDeprecationErrors
- Line: 641
- Covers:
- Early return for rolling out bundles
- Deprecation set to Unknown during rollout
- Apply errors stay in Progressing condition only
5. TestSetDeprecationStatus
- Line: 1264
- Covers: (11 test cases)
- Direct unit testing of
SetDeprecationStatusfunction - All combinations of
hasCatalogDataand deprecation states - Installed vs no bundle scenarios
- Reason values (Deprecated, DeprecationStatusUnknown, Absent)
- Direct unit testing of
Modified Tests in This PR
File: common_controller_test.go
TestClusterExtensionDeprecationMessageTruncation
- Changed: Separator from
"; "→"\n"to match production code - Covers: Message truncation with newline-separated deprecation warnings
Coverage Summary
| Logic Change | Test Coverage |
|---|---|
| Use installed bundle (not resolved) | ✅ TestClusterExtensionUpgradeShowsInstalledBundleDeprecation ✅ TestSetDeprecationStatus ✅ TestClusterExtensionResolutionFailsWithoutCatalogDeprecationData |
hasCatalogData = err == nil || resolvedDeprecation != nil |
✅ TestClusterExtensionResolutionFailsWithDeprecationData ✅ TestClusterExtensionResolutionFailsWithoutCatalogDeprecationData |
| Set deprecation before error check | ✅ TestClusterExtensionResolutionFailsWithDeprecationData ✅ TestClusterExtensionResolutionFailsWithoutCatalogDeprecationData |
| Early return for rolling out bundles | ✅ TestClusterExtensionBoxcutterApplierFailsDoesNotLeakDeprecationErrors |
| Deprecation conditions absent when not deprecated | ✅ TestSetDeprecationStatus (11 cases) |
| Unknown when catalog unavailable | ✅ TestClusterExtensionResolutionFailsWithoutCatalogDeprecationData ✅ TestSetDeprecationStatus |
| Newline separator for messages | ✅ TestClusterExtensionDeprecationMessageTruncation |
Total Test Coverage
- 5 new integration/reconciliation tests
- 11 unit test cases in TestSetDeprecationStatus
- 1 modified test for message formatting
Total: 17 test cases covering all code paths and behavioral changes.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 13 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: pedjak The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
/hold We will need to change here to set False always we should always set something |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
/hold cancel Changed to keep Deprecated False as before. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This fixes three problems: 1. Install errors were leaking into deprecation conditions 2. Catalog unavailable showed "False" instead of "Unknown" 3. BundleDeprecated was checking the wrong bundle (resolved vs installed) 4. Ensure that we have Deprecated False when has no deprecation Also improved the code: - Simpler logic (clear all, then add only what's needed) - Better reason values - Comprehensive test coverage for all scenarios Assisted-by: Cursor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
/lgtm - we did it! <3333 XDDD thank you!!! |
|
crd-diff failed due to changes in field description - the changes don't have no semantic impact and further clarify the deprecated condition behavior - setting override |
|
/lgtm |
Fixes deprecation conditions showing install errors instead of actual deprecation status.
Problems Fixed
1. Install errors leaked into deprecation conditions
When installation failed, the error appeared in ALL conditions including deprecation ones:
Users couldn't tell if something was actually deprecated.
2. Wrong bundle checked during upgrades
When upgrading v1.0.0 → v1.0.1, deprecation showed v1.0.1's status even when the upgrade failed.
This violates Kubernetes convention: status must show actual state, not desired state.
Changes Made
What we do now
status: Truestatus: Unknownstatus: Unknown, reason: AbsentKey improvements
Install errors stay separate
ProgressingandInstalledconditions onlyUse installed bundle (not resolved)
Correct reason values
Deprecated→ whenTrueDeprecationStatusUnknown→ whenUnknownAbsent→ when no bundle installed (BundleDeprecated only)Before/After Comparison
False(misleading) ❌Unknown(honest) ✅Deprecated❌Examples
Example 1: Install Error (Bug #2008)
Before:
After:
Example 2: Catalog Removed
Before:
After: