Confluence Data Center

[zmbc]

Is your feature request related to a problem? Please describe.

While https://docs.opensearch.org/latest/data-prepper/pipelines/configuration/sources/atlassian-confluence/ supports Confluence Cloud, it doesn't appear to support Confluence Data Center, because this code rejects any local address:

data-prepper/data-prepper-plugins/saas-source-plugins/source-crawler/src/main/java/org/opensearch/dataprepper/plugins/source/source_crawler/utils/AddressValidation.java

Lines 54 to 59 in 49ce4f5

	public static void validateInetAddress(@NonNull final InetAddress address) {
	if (address.isMulticastAddress() || address.isAnyLocalAddress() || address.isLinkLocalAddress()
	|| address.isSiteLocalAddress() || address.isLoopbackAddress()) {
	throw new BadRequestException(INVALID_URL);
	}
	}

Describe the solution you'd like

Make this behavior configurable so OpenSearch can be used with Confluence Data Center.

Describe alternatives you've considered (Optional)

Additional context

[zmbc]

Another related issue I ran into: the current Confluence source does not support bearer-token authentication. I've made a really hacky set of patches with the help of GitHub Copilot that successfully work around both these issues.

[dlvenable]

@zmbc , Thank you for raising this request. We are definitely open to supporting the bearer token. I believe that we have other sources that do, so we will want to be sure to follow the same conventions.

For Confluence Data Center, what kind of hosts do you provide? Are these running on an internal network? This IP address validation check is important. We may need to make it configurable so that this check can continue to run.

We are happy to receive PRs for these features.

[zmbc]

Yes, Confluence Data Center typically runs on an internal network. I think it would be fine to make configurable.

[dlvenable]

@zmbc , That would work. Are you able to work on these two changes?

[zmbc]

@dlvenable I don't know much Java but if you're okay with reviewing an amateurish PR I could try!

[srikanthpadakanti]

@dlvenable You can assign this to me if @zmbc is okay with it .. Thanks.