Skip to content

Commit c05da80

Browse files
sirkozuchsirkozuch
authored
Merge pull request #1 from keboola/DEV/pypi_package_structure
Merged initial pypi structure
2 parents 311338f + 04c6798 commit c05da80

File tree

14 files changed

+4047
-1
lines changed

14 files changed

+4047
-1
lines changed

‎.github/workflows/deploy.yml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
name: BUild & Upload Python Package to PYPI production
2+
3+
on:
4+
release:
5+
types: [ created ]
6+
branches:
7+
- main
8+
9+
jobs:
10+
deploy:
11+
runs-on: ubuntu-latest
12+
steps:
13+
- uses: actions/checkout@v2
14+
- name: Set up Python
15+
uses: actions/setup-python@v2
16+
with:
17+
python-version: '3.7'
18+
- name: Install dependencies
19+
run: |
20+
python -m pip install --upgrade pip
21+
pip install setuptools wheel twine
22+
pip install flake8 pytest
23+
pip install -r requirements.txt
24+
- name: Lint with flake8
25+
run: |
26+
# stop the build if there are Python syntax errors or undefined names
27+
flake8 src/ --config=flake8.cfg
28+
- name: Test with pytest
29+
run: |
30+
pytest tests
31+
- name: Build and publish
32+
env:
33+
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
34+
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
35+
run: |
36+
python setup.py sdist bdist_wheel
37+
twine upload dist/*

‎.github/workflows/deploy_to_test.yml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: Build & Deploy Python Package To Test PYPI
2+
3+
on:
4+
create:
5+
tags:
6+
- 0.*a
7+
- 1.*a
8+
9+
10+
jobs:
11+
deploy:
12+
runs-on: ubuntu-latest
13+
steps:
14+
- uses: actions/checkout@v2
15+
- name: Set up Python
16+
uses: actions/setup-python@v2
17+
with:
18+
python-version: '3.7'
19+
- name: Install dependencies
20+
run: |
21+
python -m pip install --upgrade pip
22+
pip install setuptools wheel twine
23+
pip install flake8 pytest
24+
pip install -r requirements.txt
25+
- name: Lint with flake8
26+
run: |
27+
# stop the build if there are Python syntax errors or undefined names
28+
flake8 src/ --config=flake8.cfg
29+
- name: Test with pytest
30+
run: |
31+
pytest tests
32+
- name: Build and publish
33+
env:
34+
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME_TEST }}
35+
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD_TEST }}
36+
run: |
37+
python setup.py sdist bdist_wheel
38+
twine upload --repository testpypi dist/*

‎.github/workflows/push_dev.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: Build & Test
2+
3+
on: [ push ]
4+
5+
jobs:
6+
build:
7+
8+
runs-on: ubuntu-latest
9+
strategy:
10+
matrix:
11+
python-version: [ 3.7 ]
12+
13+
steps:
14+
- uses: actions/checkout@v2
15+
- name: Set up Python ${{ matrix.python-version }}
16+
uses: actions/setup-python@v2
17+
with:
18+
python-version: ${{ matrix.python-version }}
19+
- name: Install dependencies
20+
run: |
21+
python -m pip install --upgrade pip
22+
pip install flake8 pytest
23+
pip install -r requirements.txt
24+
- name: Lint with flake8
25+
run: |
26+
# stop the build if there are Python syntax errors or undefined names
27+
flake8 src/ --config=flake8.cfg
28+
- name: Test with pytest
29+
run: |
30+
pytest tests

‎.gitignore‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,10 @@ share/python-wheels/
2727
*.egg
2828
MANIFEST
2929

30+
31+
src/test.py
32+
33+
3034
# PyInstaller
3135
# Usually these files are written by a python script from a template
3236
# before PyInstaller builds the exe, so as to inject date/other infos into it.
@@ -127,3 +131,12 @@ dmypy.json
127131

128132
# Pyre type checker
129133
.pyre/
134+
135+
# VSCode
136+
.vscode/
137+
138+
# IntelliJ
139+
.idea/
140+
141+
# MacOS files
142+
.DS_Store

‎README.md‎

Lines changed: 202 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,202 @@
1-
# python-http-client
1+
# Python HTTP Client
2+
3+
## Introduction
4+
5+
This library serves as tool to work effectively when sending requests to external services. The library wraps on top of the `requests` library and implements a couple useful method, such as in-built retry, exception raising, etc.
6+
7+
It is being developed by the Keboola Data Services team and officially supported by Keboola. It aims to simplify the Keboola Component creation process, by removing the necessity to write complicated code to work with the APIs effectively.
8+
9+
## Links
10+
11+
- API Documentation: [API Docs](https://github.com/keboola/python-http-client/blob/main)
12+
- Source code: [https://github.com/keboola/python-http-client](https://github.com/keboola/python-http-client)
13+
- PYPI project code: [https://test.pypi.org/project/keboola.http-client](https://test.pypi.org/project/keboola.http-client)
14+
- Documentation: [https://developers.keboola.com/extend/component/python-component-library](https://developers.keboola.com/extend/component/python-component-library)
15+
16+
## Quick Start
17+
18+
### Installation
19+
20+
The package may be installed via PIP:
21+
22+
```
23+
pip install keboola.http-client
24+
```
25+
26+
### Structure and Functionality
27+
28+
The package contains a single core module:
29+
- `keboola.http_client` - Contains the `HttpClient` class for easy manipulation with APIs and external services
30+
31+
### `HttpClient`
32+
33+
The core class that serves as a tool to communicate with external services. The class is a wrapper around the `requests` library with implemented retry mechanism, and automatic error handling in case of HTTP error returned.
34+
35+
For each HTTP method, following methods are implemented in the `HttpClient`:
36+
- GET - `get()`, `get_raw()`
37+
- POST - `post()`, `post_raw()`
38+
- PATCH - `patch()`, `patch_raw()`
39+
- UPDATE - `update()`, `update_raw()`
40+
- PUT - `put()`, `put_raw()`
41+
- DELETE - `delete()`, `delete_raw()`
42+
43+
The difference between `_raw()` methods and their non-`_raw()` counterparts is, that raw methods will return `requests.Response` object, while non-raw methods will return a json body if the request is successful and raise an error if an HTTP error is encountered.
44+
45+
All abovementioned methods support all parameters supported by `requests.request()` functions - as described in the [documentation](https://requests.readthedocs.io/en/latest/api/#main-interface).
46+
47+
#### Initialization
48+
49+
The core class is `keboola.http_client.HttpClient`, which can be initialized by specifying the `base_url` parameter:
50+
51+
```python
52+
from keboola.http_client import HttpClient
53+
54+
BASE_URL = 'https://connection.keboola.com/v2/storage/'
55+
cl = HttpClient(BASE_URL)
56+
```
57+
58+
#### Default arguments
59+
60+
For `HttpClient`, it is possible to define default arguments, which will be sent with every request. It's possible to define `default_http_header`, `auth_header` and `default_params` - a default header, a default authentication header and default parameters, respectively.
61+
62+
```python
63+
from keboola.http_client import HttpClient
64+
65+
BASE_URL = 'https://connection.keboola.com/v2/storage/'
66+
AUTH_HEADER = {
67+
'x-storageapi-token': '1234-STORAGETOKENSTRING'
68+
}
69+
DEFAULT_PARAMS = {
70+
'include': 'columns'
71+
}
72+
DEFAULT_HEADER = {
73+
'Content-Type': 'application/json'
74+
}
75+
76+
cl = HttpClient(BASE_URL, default_http_header=DEFAULT_HEADER,
77+
auth_header=AUTH_HEADER, default_params=DEFAULT_PARAMS)
78+
```
79+
80+
#### Basic authentication
81+
82+
By specifying the `auth` argument, the `HttpClient` will utilize the basic authentication.
83+
84+
```python
85+
from keboola.http_client import HttpClient
86+
87+
BASE_URL = 'https://connection.keboola.com/v2/storage/'
88+
USERNAME = 'TestUser'
89+
PASSWORD = '@bcd1234'
90+
91+
cl = HttpClient(BASE_URL, auth=(USERNAME, PASSWORD))
92+
```
93+
94+
#### Simple POST request
95+
96+
Making a simple POST request using `post_raw()` method.
97+
98+
```python
99+
from keboola.http_client import HttpClient
100+
101+
BASE_URL = 'https://www.example.com/change'
102+
cl = HttpClient(BASE_URL)
103+
104+
data = {'attr_1': 'value_1', 'attr_2': 'value_2'}
105+
header = {'content-type': 'application/json'}
106+
response = cl.post_raw(data=data, headers=header)
107+
108+
if response.ok is not True:
109+
raise ValueError(response.json())
110+
else:
111+
print(response.json())
112+
```
113+
114+
Making a simple POST request using `post()` method.
115+
116+
```python
117+
from keboola.http_client import HttpClient
118+
119+
BASE_URL = 'https://www.example.com/change'
120+
cl = HttpClient(BASE_URL)
121+
122+
data = {'attr_1': 'value_1', 'attr_2': 'value_2'}
123+
header = {'content-type': 'application/json'}
124+
response = cl.post(data=data, headers=header)
125+
```
126+
127+
#### Working with URL paths
128+
129+
Each of the methods takes an optional positional argument `endpoint_path`. If specified, the value of the `endpoint_path` will be appended to the URL specified in the `base_url` parameter, when initializing the class. When appending the `endpoint_path`, the [`urllib.parse.urljoin()`](https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urljoin) function is used.
130+
131+
The below code will send a POST request to the URL `https://example.com/api/v1/events`:
132+
133+
```python
134+
from keboola.http_client import HttpClient
135+
136+
BASE_URL = 'https://example.com/api/v1'
137+
cl = HttpClient(BASE_URL)
138+
139+
header = {'token': 'token_value'}
140+
cl.post_raw('events', headers=header)
141+
```
142+
143+
It is also possible to override this behavior by using parameter `is_absolute_path=True`. If specified, the value of `endpoint_path` will not be appended to the `base_url` parameter, but will rather be used as an absolute URL to which the HTTP request will be made.
144+
145+
In the below code, the `base_url` parameter is set to `https://example.com/api/v1`, but the base URL will be overriden by specifying `is_absolute_path=True` and the HTTP request will be made to the URL specified in the `post()` request - `https://anothersite.com/v2`.
146+
147+
```python
148+
from keboola.http_client import HttpClient
149+
150+
BASE_URL = 'https://example.com/api/v1'
151+
cl = HttpClient(BASE_URL)
152+
153+
header = {'token': 'token_value'}
154+
cl.post_raw('https://anothersite.com/v2', headers=header, is_absolute_path=True)
155+
```
156+
157+
#### Raw request Example
158+
159+
A simple request made with default authentication header and parameters.
160+
161+
```python
162+
import os
163+
from keboola.http_client import HttpClient
164+
165+
BASE_URL = 'https://connection.keboola.com/v2/'
166+
TOKEN = os.environ['TOKEN']
167+
168+
cl = HttpClient(BASE_URL, auth_header={'x-storageapi-token': TOKEN})
169+
170+
request_params = {'exclude': 'components'}
171+
response = cl.get_raw('storage', params=request_params)
172+
173+
if response.ok is True:
174+
print(response.json())
175+
```
176+
177+
#### Building HTTP client based on HTTPClient Example
178+
179+
This example demonstrates the default use of the HTTPClient as a base for REST API clients.
180+
181+
```python
182+
from keboola.http_client import HttpClient
183+
184+
BASE_URL = 'https://connection.eu-central-1.keboola.com/v2/storage'
185+
MAX_RETRIES = 10
186+
187+
188+
class KBCStorageClient(HttpClient):
189+
190+
def __init__(self, storage_token):
191+
HttpClient.__init__(self, base_url=BASE_URL, max_retries=MAX_RETRIES, backoff_factor=0.3,
192+
status_forcelist=(429, 500, 502, 504),
193+
auth_header={"X-StorageApi-Token": storage_token})
194+
195+
def get_files(self, show_expired=False):
196+
params = {"showExpired": show_expired}
197+
return self.get('files', params=params)
198+
199+
cl = KBCStorageClient("my_token")
200+
201+
print(cl.get_files())
202+
```

0 commit comments

Comments
 (0)