Merge branch 'master' into redshift-super-support

Author: Sergey Vasilyev

README.md

@@ -1,3 +1,7 @@
+**THE PROJECT IS MERGED INTO [data-diff](https://github.com/datafold/data-diff) ON 2023-04-20 AND IS MAINTAINED THERE.**
+
+**THIS REPO IS FOR HISTORIC REFERENCES ONLY.**
+
 # Sqeleton
 
 **Under construction!**
@@ -12,6 +16,8 @@ It consists of -
 
 It is comparable to other libraries such as SQLAlchemy or PyPika, in terms of API and intended audience. However there are several notable ways in which it is different.
 
+## Overview
+
 ### Built for performance
 
 - Multi-threaded by default -
@@ -31,11 +37,13 @@ This feature can be also used to inform the query-builder, either as an alternat
 
 The schema is used for validation when building expressions, making sure the names are correct, and that the data-types align.
 
+(Still WIP)
+
 ### Multi-database access
 
 Sqeleton is designed to work with several databases at the same time. Its API abstracts away as many implementation details as possible.
 
-Databases we support:
+Databases we fully support:
 
 - PostgreSQL >=10
 - MySQL
@@ -51,11 +59,28 @@ Databases we support:
 - DuckDB >=0.6
 - SQLite (coming soon)
 
-
-### Documentation
+## Documentation
 
 [Read the docs!](https://sqeleton.readthedocs.io)
 
+Or jump straight to the [introduction](https://sqeleton.readthedocs.io/en/latest/intro.html).
+
+### Install
+
+Install using pip:
+
+```bash
+pip install sqeleton
+```
+
+It is recommended to install the driver dependencies using pip's `[]` syntax:
+
+```bash
+pip install 'sqeleton[mysql, postgresql]'
+```
+
+Read more in [install / getting started.](https://sqeleton.readthedocs.io/en/latest/install.html)
+
 ### Basic usage
 
 ```python

docs/conn_editor.md

@@ -0,0 +1,37 @@
+# Connection Editor
+
+A common complaint among new users was the difficulty in setting up the connections.
+
+Connection URLs are admittedly confusing, and editing `.toml` files isn't always straight-forward either.
+
+To ease this initial difficulty, we added a `textual`-based TUI tool to sqeleton, that allows users to edit configuration files and test the connections while editing them.
+
+## Install
+
+This tool needs `textual` to run. You can install it using:
+
+```bash
+pip install 'sqeleton[tui]'
+```
+
+Make sure you also have drivers for whatever database connection you're going to edit!
+
+## Run
+
+Once everything is installed, you can run the editor with the following command:
+
+```bash
+sqeleton conn-editor <conf_path.toml>
+```
+
+Example:
+
+```bash
+sqeleton conn-editor ~/dbs.toml
+```
+
+The available actions and hotkeys will be listed in the status bar.
+
+Note: using the connection editor will delete comments and reformat the file!
+
+We recommend backing up the configuration file before editing it.

docs/index.rst

@@ -3,8 +3,10 @@
    :caption: API Reference
    :hidden:
 
+   install
    intro
    supported-databases
+   conn_editor
    python-api
 
 Sqeleton
@@ -26,8 +28,10 @@ For more information, `See our README <https://github.com/datafold/sqeleton#read
 Resources
 ---------
 
+- :doc:`install`
 - :doc:`intro`
 - :doc:`supported-databases`
+- :doc:`conn_editor`
 - :doc:`python-api`
 - Source code (git): `<https://github.com/datafold/sqeleton>`_
 

docs/install.md

@@ -0,0 +1,58 @@
+# Install / Get started
+
+Sqeleton can be installed using pip:
+
+```
+pip install sqeleton
+```
+
+## Database drivers
+
+To ensure that the database drivers are compatible with sqeleton, we recommend installing them along with sqeleton, using pip's `[]` syntax:
+
+- `pip install 'sqeleton[mysql]'`
+
+- `pip install 'sqeleton[postgresql]'`
+
+- `pip install 'sqeleton[snowflake]'`
+
+- `pip install 'sqeleton[presto]'`
+
+- `pip install 'sqeleton[oracle]'`
+
+- `pip install 'sqeleton[trino]'`
+
+- `pip install 'sqeleton[clickhouse]'`
+
+- `pip install 'sqeleton[vertica]'`
+
+- For BigQuery, see: https://pypi.org/project/google-cloud-bigquery/
+
+_Some drivers have dependencies that cannot be installed using `pip` and still need to be installed manually._
+
+
+It is also possible to install several databases at once. For example:
+
+```bash
+pip install 'sqeleton[mysql, postgresql]'
+```
+
+Note: Some shells use `"` for escaping instead, like:
+
+```bash
+pip install "sqeleton[mysql, postgresql]"
+```
+
+## Connection editor
+
+Sqeleton provides a TUI connection editor, that can be installed using:
+
+```bash
+pip install 'sqeleton[tui]'
+```
+
+Read more [here](conn_editor.md).
+
+## What's next?
+
+Read the [introduction](intro.md) and start coding!

docs/intro.md

@@ -440,12 +440,12 @@ from sqeleton.abcs.mixins import AbstractMixin_NormalizeValue, AbstractMixin_Ran
 
 connect = sqeleton.connect.load_mixins(AbstractMixin_NormalizeValue)
 ddb = connect("duckdb://:memory:")
-print(ddb.dialect.normalize_boolean("bool", None) == "bool::INTEGER::VARCHAR")
+print(ddb.dialect.normalize_boolean("bool", None))
 # Outputs:
 #   bool::INTEGER::VARCHAR
 ```
 
-Each database is already aware of the available mixin implementation, because it was defined with the `MIXINS` attribute. We're only using the abstract mixins to select the mixins we want to use.
+Each database is already aware of the available mixin implementations, because it was defined with the `MIXINS` attribute. We're only using the abstract mixins to select the mixins we want to use.
 
 #### List of mixins
 
@@ -463,6 +463,12 @@ List of available abstract mixins:
 
 - `AbstractMixin_TimeTravel` - Only snowflake & bigquery
 
+- `AbstractMixin_OptimizerHints` - Only oracle & mysql
+
+More will be added in the future.
+
+Note that it's still possible to use user-defined mixins that aren't on this list.
+
 #### Unimplemented Mixins
 
 Trying to load a mixin that isn't implemented by all databases, will fail: