Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/workflows/dependency-submission.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,24 @@
# --------------------------------------------------------------------
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed
# with this work for additional information regarding copyright
# ownership. The ASF licenses this file to You under the Apache
# License, Version 2.0 (the "License"); you may not use this file
# except in compliance with the License. You may obtain a copy of the
# License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing
# permissions and limitations under the License.
#
# --------------------------------------------------------------------
# Apache Cloudberry PXF Dependency Submission Workflow
# --------------------------------------------------------------------
name: Dependency Submission

on:
Expand Down
21 changes: 21 additions & 0 deletions .github/workflows/pxf-ci.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,24 @@
# --------------------------------------------------------------------
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed
# with this work for additional information regarding copyright
# ownership. The ASF licenses this file to You under the Apache
# License, Version 2.0 (the "License"); you may not use this file
# except in compliance with the License. You may obtain a copy of the
# License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing
# permissions and limitations under the License.
#
# --------------------------------------------------------------------
# Apache Cloudberry PXF CI Workflow
# --------------------------------------------------------------------
name: PXF CI Pipeline

on:
Expand Down
19 changes: 19 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,22 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

Apache Cloudberry community welcomes contributions from anyone, new and
experienced! We appreciate your interest in contributing. This guide will help
you get started with the contribution.
Expand Down
9 changes: 9 additions & 0 deletions DISCLAIMER
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Apache Cloudberry is an effort undergoing incubation at The Apache
Software Foundation (ASF), sponsored by the Apache
Incubator. Incubation is required of all newly accepted projects until
a further review indicates that the infrastructure, communications,
and decision making process have stabilized in a manner consistent
with other successful ASF projects. While incubation status is not
necessarily a reflection of the completeness or stability of the code,
it does indicate that the project has yet to be fully endorsed by the
ASF.
78 changes: 78 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,84 @@ This product is derived from software originally developed by:
notices and license terms. Your use of these subcomponents is subject to the
terms and conditions of the subcomponent's license, as noted in the LICENSE file.


The Greenplum Platform Extension Framework includes:

----------------------------
Apache License - Version 2.0

The following files are licensed under the Apache License, Version 2.0:

FDW Module:
fdw/libchurl.c
fdw/libchurl.h
fdw/pxf_bridge.c
fdw/pxf_bridge.h
fdw/pxf_filter.c
fdw/pxf_filter.h
fdw/pxf_header.c
fdw/pxf_header.h

External Table Module:
external-table/src/gpdbwritableformatter.c
external-table/src/libchurl.c
external-table/src/libchurl.h
external-table/src/pxfbridge.c
external-table/src/pxfbridge.h
external-table/src/pxffilters.c
external-table/src/pxffilters.h
external-table/src/pxfheaders.c
external-table/src/pxfheaders.h
external-table/src/pxfprotocol.c
external-table/src/pxfuriparser.c
external-table/src/pxfuriparser.h
external-table/test/pxffilters_test.c
external-table/test/pxfheaders_test.c
external-table/test/pxfprotocol_test.c
external-table/test/pxfuriparser_test.c

Server Module (Java Sources):
server/build.gradle
server/gradle.properties
server/settings.gradle
server/pxf-api/src/main/java/org/apache/cloudberry/pxf/api/*.java
server/pxf-api/src/test/java/org/apache/cloudberry/pxf/api/**/*.java
server/pxf-hbase/src/main/java/org/apache/cloudberry/pxf/plugins/hbase/**/*.java
server/pxf-hbase/src/test/java/org/apache/cloudberry/pxf/plugins/hbase/**/*.java
server/pxf-hdfs/src/main/java/org/apache/cloudberry/pxf/plugins/hdfs/**/*.java
server/pxf-hdfs/src/test/java/org/apache/cloudberry/pxf/plugins/hdfs/**/*.java
server/pxf-hive/src/main/java/org/apache/cloudberry/pxf/plugins/hive/**/*.java
server/pxf-hive/src/test/java/org/apache/cloudberry/pxf/plugins/hive/**/*.java
server/pxf-jdbc/src/main/java/org/apache/cloudberry/pxf/plugins/jdbc/**/*.java
server/pxf-jdbc/src/test/java/org/apache/cloudberry/pxf/plugins/jdbc/**/*.java
server/pxf-json/src/main/java/org/apache/cloudberry/pxf/plugins/json/**/*.java
server/pxf-json/src/test/java/org/apache/cloudberry/pxf/plugins/json/**/*.java
server/pxf-service/src/main/java/org/apache/cloudberry/pxf/service/**/*.java
server/pxf-service/src/test/java/org/apache/cloudberry/pxf/service/**/*.java

Documentation Templates:
docs/content/*.html.md.erb

Configuration Files:
server/pxf-api/src/test/resources/pxf-profiles-default.xml
server/pxf-hive/src/test/resources/pxf-profiles-default.xml
server/pxf-jdbc/src/test/resources/log4j.properties
server/pxf-json/src/test/resources/log4j.properties
server/pxf-service/src/main/resources/pxf-profiles-default.xml
server/pxf-service/src/templates/conf/pxf-profiles.xml
server/pxf-service/src/test/resources/pxf-profiles-default.xml

CI/Test Templates:
automation/src/test/resources/templates/zk/zoo.cfg
ci/singlecluster/templates/hadoop/etc/hadoop/core-site.xml
ci/singlecluster/templates/hadoop/etc/hadoop/hdfs-site.xml
ci/singlecluster/templates/hadoop/etc/hadoop/yarn-env.sh
ci/singlecluster/templates/hbase/conf/hbase-env.sh
ci/singlecluster/templates/hbase/conf/hbase-site.xml
ci/singlecluster/templates/ranger/install.properties
ci/singlecluster/templates/tez/conf/tez-site.xml
ci/singlecluster/templates/usersync/install.properties

=======================================================================

This product bundles Gradle Wrapper, which is licensed under
Expand Down
42 changes: 22 additions & 20 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,9 @@ install-server:

stage:
rm -rf build/stage
make -C $(SOURCE_EXTENSION_DIR) stage
make -C cli stage
make -C server stage
ifneq ($(SKIP_EXTERNAL_TABLE_PACKAGE_REASON),)
@echo "Skipping staging FDW extension because $(SKIP_EXTERNAL_TABLE_PACKAGE_REASON)"
$(eval PXF_MODULES := $(filter-out external-table,$(PXF_MODULES)))
Expand All @@ -95,12 +98,12 @@ endif
set -e ;\
GP_MAJOR_VERSION=$$(cat $(SOURCE_EXTENSION_DIR)/build/metadata/gp_major_version) ;\
GP_BUILD_ARCH=$$(cat $(SOURCE_EXTENSION_DIR)/build/metadata/build_arch) ;\
PXF_PACKAGE_NAME=pxf-cbdb$${GP_MAJOR_VERSION}-$${PXF_VERSION}-$${GP_BUILD_ARCH} ;\
PXF_PACKAGE_NAME=pxf-cloudberry$${GP_MAJOR_VERSION}-$${PXF_VERSION}-$${GP_BUILD_ARCH} ;\
mkdir -p build/stage/$${PXF_PACKAGE_NAME} ;\
cp -a $(SOURCE_EXTENSION_DIR)/build/stage/* build/stage/$${PXF_PACKAGE_NAME} ;\
cp -a cli/build/stage/* build/stage/$${PXF_PACKAGE_NAME} ;\
cp -a server/build/stage/* build/stage/$${PXF_PACKAGE_NAME} ;\
echo $$(git rev-parse --verify HEAD) > build/stage/$${PXF_PACKAGE_NAME}/pxf/commit.sha ;\
echo $$(git rev-parse --verify HEAD) > build/stage/$${PXF_PACKAGE_NAME}/commit.sha ;\
cp package/install_binary build/stage/$${PXF_PACKAGE_NAME}/install_component ;\
echo "===> PXF staging is complete <==="

Expand All @@ -115,16 +118,15 @@ gppkg-rpm: rpm
mkdir -p gppkg/deps
GP_MAJOR_VERSION=$$(cat $(SOURCE_EXTENSION_DIR)/build/metadata/gp_major_version)
cat package/gppkg_spec.yml.in | sed "s,#arch,`arch`," | sed "s,#os,$(TEST_OS)," | sed "s,#gppkgver,1.0," | sed "s,#gpver,1," > gppkg/gppkg_spec.yml
find build/rpmbuild/RPMS -name pxf-cbdb$(GP_MAJOR_VERSION)-*.rpm -exec cp {} gppkg/ \;
source $(GPHOME)/greenplum_path.sh && gppkg --build gppkg
find build/rpmbuild/RPMS -name pxf-cloudberry$(GP_MAJOR_VERSION)-*.rpm -exec cp {} gppkg/ \;
source $(GPHOME)/greenplum_path.sh || source $(GPHOME)/cloudberry-env.sh && gppkg --build gppkg

rpm:
make -C $(SOURCE_EXTENSION_DIR) stage
make -C cli stage
make -C server stage
rpm: stage
set -e ;\
GP_MAJOR_VERSION=$$(cat $(SOURCE_EXTENSION_DIR)/build/metadata/gp_major_version) ;\
PXF_FULL_VERSION=$${PXF_VERSION} ;\
GP_BUILD_ARCH=$$(cat $(SOURCE_EXTENSION_DIR)/build/metadata/build_arch) ;\
PXF_PACKAGE_NAME=pxf-cloudberry$${GP_MAJOR_VERSION}-${PXF_VERSION}-$${GP_BUILD_ARCH} ;\
PXF_FULL_VERSION=${PXF_VERSION} ;\
PXF_MAIN_VERSION=$$(echo $${PXF_FULL_VERSION} | sed -E 's/(-SNAPSHOT|-rc[0-9]+)$$//') ;\
if [[ $${PXF_FULL_VERSION} == *"-SNAPSHOT" ]]; then \
PXF_RELEASE=SNAPSHOT; \
Expand All @@ -135,7 +137,7 @@ rpm:
fi ;\
rm -rf build/rpmbuild ;\
mkdir -p build/rpmbuild/{BUILD,RPMS,SOURCES,SPECS} ;\
cp -a build/stage/$${PXF_PACKAGE_NAME}/pxf/* build/rpmbuild/SOURCES ;\
cp -a build/stage/$${PXF_PACKAGE_NAME}/* build/rpmbuild/SOURCES ;\
cp package/*.spec build/rpmbuild/SPECS/ ;\
rpmbuild \
--define "_topdir $${PWD}/build/rpmbuild" \
Expand All @@ -150,7 +152,7 @@ rpm-tar: rpm
mkdir -p build/{stagerpm,distrpm}
set -e ;\
GP_MAJOR_VERSION=$$(cat $(SOURCE_EXTENSION_DIR)/build/metadata/gp_major_version) ;\
PXF_RPM_FILE=$$(find build/rpmbuild/RPMS -name pxf-cbdb$${GP_MAJOR_VERSION}-*.rpm) ;\
PXF_RPM_FILE=$$(find build/rpmbuild/RPMS -name cloudberry-pxf-*.rpm) ;\
PXF_RPM_BASE_NAME=$$(basename $${PXF_RPM_FILE%*.rpm}) ;\
PXF_PACKAGE_NAME=$${PXF_RPM_BASE_NAME%.*} ;\
mkdir -p build/stagerpm/$${PXF_PACKAGE_NAME} ;\
Expand All @@ -165,24 +167,24 @@ deb: stage
PXF_MAIN_VERSION=$${PXF_VERSION//-SNAPSHOT/} ;\
if [[ $${PXF_VERSION} == *"-SNAPSHOT" ]]; then PXF_RELEASE=SNAPSHOT; else PXF_RELEASE=1; fi ;\
rm -rf build/debbuild ;\
mkdir -p build/debbuild/usr/local/pxf-cbdb$${GP_MAJOR_VERSION}/$(TARGET_EXTENSION_DIR) ;\
cp -a $(SOURCE_EXTENSION_DIR)/build/stage/* build/debbuild/usr/local/pxf-cbdb$${GP_MAJOR_VERSION}/$(TARGET_EXTENSION_DIR) ;\
cp -a cli/build/stage/pxf/* build/debbuild/usr/local/pxf-cbdb$${GP_MAJOR_VERSION} ;\
cp -a server/build/stage/pxf/* build/debbuild/usr/local/pxf-cbdb$${GP_MAJOR_VERSION} ;\
echo $$(git rev-parse --verify HEAD) > build/debbuild/usr/local/pxf-cbdb$${GP_MAJOR_VERSION}/commit.sha ;\
mkdir -p build/debbuild/usr/local/cloudberry-pxf/$(TARGET_EXTENSION_DIR) ;\
cp -a $(SOURCE_EXTENSION_DIR)/build/stage/* build/debbuild/usr/local/cloudberry-pxf/ ;\
cp -a cli/build/stage/* build/debbuild/usr/local/cloudberry-pxf ;\
cp -a server/build/stage/* build/debbuild/usr/local/cloudberry-pxf ;\
echo $$(git rev-parse --verify HEAD) > build/debbuild/usr/local/cloudberry-pxf/commit.sha ;\
mkdir build/debbuild/DEBIAN ;\
cp -a package/DEBIAN/* build/debbuild/DEBIAN/ ;\
sed -i -e "s/%VERSION%/$${PXF_MAIN_VERSION}-$${PXF_RELEASE}/" -e "s/%MAINTAINER%/${VENDOR}/" build/debbuild/DEBIAN/control ;\
sed -i -e "s/%VERSION%/$${PXF_MAIN_VERSION}-$${PXF_RELEASE}/" -e "s/%MAINTAINER%/${VENDOR}/" -e "s/%ARCH%/$$(dpkg --print-architecture)/" build/debbuild/DEBIAN/control ;\
dpkg-deb --build build/debbuild ;\
mv build/debbuild.deb build/pxf-cbdb$${GP_MAJOR_VERSION}-$${PXF_MAIN_VERSION}-$${PXF_RELEASE}-ubuntu18.04-amd64.deb
mv build/debbuild.deb build/cloudberry-pxf-$${PXF_MAIN_VERSION}-$${PXF_RELEASE}-$$(lsb_release -si | tr '[:upper:]' '[:lower:]')$$(lsb_release -sr)-$$(dpkg --print-architecture).deb

deb-tar: deb
rm -rf build/{stagedeb,distdeb}
mkdir -p build/{stagedeb,distdeb}
set -e ;\
GP_MAJOR_VERSION=$$(cat $(SOURCE_EXTENSION_DIR)/build/metadata/gp_major_version) ;\
PXF_DEB_FILE=$$(find build/ -name pxf-cbdb$${GP_MAJOR_VERSION}*.deb) ;\
PXF_PACKAGE_NAME=$$(dpkg-deb --field $${PXF_DEB_FILE} Package)-$$(dpkg-deb --field $${PXF_DEB_FILE} Version)-ubuntu18.04 ;\
PXF_DEB_FILE=$$(find build/ -name cloudberry-pxf*.deb) ;\
PXF_PACKAGE_NAME=$$(dpkg-deb --field $${PXF_DEB_FILE} Package)-$$(dpkg-deb --field $${PXF_DEB_FILE} Version)-$$(lsb_release -si | tr '[:upper:]' '[:lower:]')$$(lsb_release -rs) ;\
mkdir -p build/stagedeb/$${PXF_PACKAGE_NAME} ;\
cp $${PXF_DEB_FILE} build/stagedeb/$${PXF_PACKAGE_NAME} ;\
cp package/install_deb build/stagedeb/$${PXF_PACKAGE_NAME}/install_component ;\
Expand Down
Loading
Loading