Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
585 changes: 553 additions & 32 deletions .librarian/config.yaml

Large diffs are not rendered by default.

868 changes: 842 additions & 26 deletions .librarian/state.yaml

Large diffs are not rendered by default.

4,763 changes: 4,763 additions & 0 deletions librarian.yaml

Large diffs are not rendered by default.

20 changes: 9 additions & 11 deletions packages/bigframes/.repo-metadata.json
Original file line number Diff line number Diff line change
@@ -1,16 +1,14 @@
{
"name": "bigframes",
"name_pretty": "A unified Python API in BigQuery",
"product_documentation": "https://cloud.google.com/bigquery",
"api_id": "bigquery.googleapis.com",
"api_shortname": "bigquery",
"client_documentation": "https://cloud.google.com/python/docs/reference/bigframes/latest",
"distribution_name": "bigframes",
"issue_tracker": "https://github.com/googleapis/python-bigquery-dataframes/issues",
"release_level": "preview",
"language": "python",
"library_type": "INTEGRATION",
"repo": "googleapis/google-cloud-python",
"distribution_name": "bigframes",
"api_id": "bigquery.googleapis.com",
"default_version": "",
"codeowner_team": "@googleapis/bigquery-dataframe-team",
"api_shortname": "bigquery"
}
"name": "bigframes",
"name_pretty": "A unified Python API in BigQuery",
"product_documentation": "https://cloud.google.com/bigquery",
"release_level": "preview",
"repo": "googleapis/google-cloud-python"
}
1 change: 0 additions & 1 deletion packages/bigframes/docs/README.rst

This file was deleted.

94 changes: 94 additions & 0 deletions packages/bigframes/docs/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
BigQuery DataFrames (BigFrames)
===============================


|GA| |pypi| |versions|

BigQuery DataFrames (also known as BigFrames) provides a Pythonic DataFrame
and machine learning (ML) API powered by the BigQuery engine. It provides modules
for many use cases, including:

* `bigframes.pandas <https://dataframes.bigquery.dev/reference/api/bigframes.pandas.html>`_
is a pandas API for analytics. Many workloads can be
migrated from pandas to bigframes by just changing a few imports.
* `bigframes.ml <https://dataframes.bigquery.dev/reference/index.html#ml-apis>`_
is a scikit-learn-like API for ML.
* `bigframes.bigquery.ai <https://dataframes.bigquery.dev/reference/api/bigframes.bigquery.ai.html>`_
are a collection of powerful AI methods, powered by Gemini.

BigQuery DataFrames is an `open-source package <https://github.com/googleapis/google-cloud-python>`_.

.. |GA| image:: https://img.shields.io/badge/support-GA-gold.svg
:target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#general-availability
.. |pypi| image:: https://img.shields.io/pypi/v/bigframes.svg
:target: https://pypi.org/project/bigframes/
.. |versions| image:: https://img.shields.io/pypi/pyversions/bigframes.svg
:target: https://pypi.org/project/bigframes/

Getting started with BigQuery DataFrames
----------------------------------------

The easiest way to get started is to try the
`BigFrames quickstart <https://cloud.google.com/bigquery/docs/dataframes-quickstart>`_
in a `notebook in BigQuery Studio <https://cloud.google.com/bigquery/docs/notebooks-introduction>`_.

To use BigFrames in your local development environment,

1. Run ``pip install --upgrade bigframes`` to install the latest version.

2. Setup `Application default credentials <https://cloud.google.com/docs/authentication/set-up-adc-local-dev-environment>`_
for your local development environment enviroment.

3. Create a `GCP project with the BigQuery API enabled <https://cloud.google.com/bigquery/docs/sandbox>`_.

4. Use the ``bigframes`` package to query data.

.. code-block:: python

import bigframes.pandas as bpd

bpd.options.bigquery.project = your_gcp_project_id # Optional in BQ Studio.
bpd.options.bigquery.ordering_mode = "partial" # Recommended for performance.
df = bpd.read_gbq("bigquery-public-data.usa_names.usa_1910_2013")
print(
df.groupby("name")
.agg({"number": "sum"})
.sort_values("number", ascending=False)
.head(10)
.to_pandas()
)

Documentation
-------------

To learn more about BigQuery DataFrames, visit these pages

* `Introduction to BigQuery DataFrames (BigFrames) <https://cloud.google.com/bigquery/docs/bigquery-dataframes-introduction>`_
* `Sample notebooks <https://github.com/googleapis/google-cloud-python/tree/main/notebooks>`_
* `API reference <https://dataframes.bigquery.dev/>`_
* `Source code (GitHub) <https://github.com/googleapis/google-cloud-python>`_

License
-------

BigQuery DataFrames is distributed with the `Apache-2.0 license
<https://github.com/googleapis/google-cloud-python/blob/main/LICENSE>`_.

It also contains code derived from the following third-party packages:

* `Ibis <https://ibis-project.org/>`_
* `pandas <https://pandas.pydata.org/>`_
* `Python <https://www.python.org/>`_
* `scikit-learn <https://scikit-learn.org/>`_
* `XGBoost <https://xgboost.readthedocs.io/en/stable/>`_
* `SQLGlot <https://sqlglot.com/sqlglot.html>`_

For details, see the `third_party
<https://github.com/googleapis/google-cloud-python/tree/main/third_party/bigframes_vendored>`_
directory.


Contact Us
----------

For further help and provide feedback, you can email us at `bigframes-feedback@google.com <https://mail.google.com/mail/?view=cm&fs=1&tf=1&to=bigframes-feedback@google.com>`_.
19 changes: 8 additions & 11 deletions packages/bigquery-magics/.repo-metadata.json
Original file line number Diff line number Diff line change
@@ -1,16 +1,13 @@
{
"name": "bigquery-magics",
"name_pretty": "Google BigQuery connector for Jupyter and IPython",
"product_documentation": "https://cloud.google.com/bigquery",
"api_id": "bigquery.googleapis.com",
"client_documentation": "https://googleapis.dev/python/bigquery-magics/latest/",
"distribution_name": "bigquery-magics",
"issue_tracker": "https://github.com/googleapis/python-bigquery-magics/issues",
"release_level": "stable",
"language": "python",
"library_type": "INTEGRATION",
"repo": "googleapis/google-cloud-python",
"distribution_name": "bigquery-magics",
"api_id": "bigquery.googleapis.com",
"requires_billing": false,
"default_version": "",
"codeowner_team": "@googleapis/cloud-sdk-python-team @googleapis/bigquery-dataframe-team"
}
"name": "bigquery-magics",
"name_pretty": "Google BigQuery connector for Jupyter and IPython",
"product_documentation": "https://cloud.google.com/bigquery",
"release_level": "preview",
"repo": "googleapis/google-cloud-python"
}
1 change: 0 additions & 1 deletion packages/bigquery-magics/docs/README.rst

This file was deleted.

103 changes: 103 additions & 0 deletions packages/bigquery-magics/docs/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
IPython Magics for BigQuery
===========================

|GA| |pypi| |versions|

Querying massive datasets can be time consuming and expensive without the
right hardware and infrastructure. Google `BigQuery`_ solves this problem by
enabling super-fast, SQL queries against append-mostly tables, using the
processing power of Google's infrastructure.

- `Library Documentation`_
- `Product Documentation`_

.. |GA| image:: https://img.shields.io/badge/support-GA-gold.svg
:target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#general-availability
.. |pypi| image:: https://img.shields.io/pypi/v/bigquery-magics.svg
:target: https://pypi.org/project/bigquery-magics/
.. |versions| image:: https://img.shields.io/pypi/pyversions/bigquery-magics.svg
:target: https://pypi.org/project/bigquery-magics/
.. _BigQuery: https://cloud.google.com/bigquery/what-is-bigquery
.. _Library Documentation: https://googleapis.dev/python/bigquery-magics/latest
.. _Product Documentation: https://cloud.google.com/bigquery/docs/reference/v2/

Quick Start
-----------

In order to use this library, you first need to go through the following steps:

1. `Select or create a Cloud Platform project.`_
2. `Enable billing for your project.`_
3. `Enable the Google Cloud BigQuery API.`_
4. `Setup Authentication.`_

.. _Select or create a Cloud Platform project.: https://console.cloud.google.com/project
.. _Enable billing for your project.: https://cloud.google.com/billing/docs/how-to/modify-project#enable_billing_for_a_project
.. _Enable the Google Cloud BigQuery API.: https://cloud.google.com/bigquery
.. _Setup Authentication.: https://googleapis.dev/python/google-api-core/latest/auth.html

Installation
~~~~~~~~~~~~

Install this library in a `virtualenv`_ using pip. `virtualenv`_ is a tool to
create isolated Python environments. The basic problem it addresses is one of
dependencies and versions, and indirectly permissions.

With `virtualenv`_, it's possible to install this library without needing system
install permissions, and without clashing with the installed system
dependencies.

.. _`virtualenv`: https://virtualenv.pypa.io/en/latest/


Supported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^
Python >= 3.9

Unsupported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Python <= 3.8.


Mac/Linux
^^^^^^^^^

.. code-block:: console

pip install virtualenv
virtualenv <your-env>
source <your-env>/bin/activate
<your-env>/bin/pip install bigquery-magics


Windows
^^^^^^^

.. code-block:: console

pip install virtualenv
virtualenv <your-env>
<your-env>\Scripts\activate
<your-env>\Scripts\pip.exe install bigquery-magics

Example Usage
-------------

To use these magics, you must first register them. Run the ``%load_ext bigquery_magics``
in a Jupyter notebook cell.

.. code-block::

%load_ext bigquery_magics

Perform a query
~~~~~~~~~~~~~~~

.. code:: python

%%bigquery
SELECT name, SUM(number) as count
FROM 'bigquery-public-data.usa_names.usa_1910_current'
GROUP BY name
ORDER BY count DESC
LIMIT 3
17 changes: 7 additions & 10 deletions packages/db-dtypes/.repo-metadata.json
Original file line number Diff line number Diff line change
@@ -1,15 +1,12 @@
{
"api_id": "bigquery.googleapis.com",
"client_documentation": "https://googleapis.dev/python/db-dtypes/latest/index.html",
"distribution_name": "db-dtypes",
"language": "python",
"library_type": "INTEGRATION",
"name": "db-dtypes",
"name_pretty": "Pandas Data Types for SQL systems (BigQuery, Spanner)",
"product_documentation": "https://pandas.pydata.org/pandas-docs/stable/ecosystem.html#ecosystem-extensions",
"client_documentation": "https://googleapis.dev/python/db-dtypes/latest/index.html",
"release_level": "stable",
"language": "python",
"library_type": "INTEGRATION",
"repo": "googleapis/google-cloud-python",
"distribution_name": "db-dtypes",
"api_id": "bigquery.googleapis.com",
"default_version": "",
"codeowner_team": "@googleapis/python-core-client-libraries",
"api_description": "Pandas extension data types for data from SQL systems such as BigQuery."
}
"repo": "googleapis/google-cloud-python"
}
1 change: 0 additions & 1 deletion packages/db-dtypes/docs/README.rst

This file was deleted.

63 changes: 63 additions & 0 deletions packages/db-dtypes/docs/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
Pandas Data Types for SQL systems (BigQuery, Spanner)
=====================================================

|ga| |pypi| |versions|

`Pandas extension data types`_ for data from SQL systems such as `BigQuery`_.

- `Library Documentation`_

.. |ga| image:: https://img.shields.io/badge/support-GA-gold.svg
:target: https://github.com/googleapis/google-cloud-python/blob/main/README.rst#general-availability
.. |pypi| image:: https://img.shields.io/pypi/v/db-dtypes.svg
:target: https://pypi.org/project/db-dtypes/
.. |versions| image:: https://img.shields.io/pypi/pyversions/db-dtypes.svg
:target: https://pypi.org/project/db-dtypes/
.. _Pandas extension data types: https://pandas.pydata.org/docs/development/extending.html#extension-types
.. _BigQuery: https://cloud.google.com/bigquery/docs/
.. _Library Documentation: https://googleapis.dev/python/db-dtypes/latest


Installation
------------

Install this library in a `virtualenv`_ using pip. `virtualenv`_ is a tool to
create isolated Python environments. The basic problem it addresses is one of
dependencies and versions, and indirectly permissions.

With `virtualenv`_, it's possible to install this library without needing system
install permissions, and without clashing with the installed system
dependencies.

.. _`virtualenv`: https://virtualenv.pypa.io/en/latest/


Supported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^
Python >= 3.9

Unsupported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Python <= 3.8.


Mac/Linux
^^^^^^^^^

.. code-block:: console

pip install virtualenv
virtualenv <your-env>
source <your-env>/bin/activate
<your-env>/bin/pip install db-dtypes


Windows
^^^^^^^

.. code-block:: console

pip install virtualenv
virtualenv <your-env>
<your-env>\Scripts\activate
<your-env>\Scripts\pip.exe install db-dtypes
17 changes: 8 additions & 9 deletions packages/django-google-spanner/.repo-metadata.json
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
{
"name": "django-google-spanner",
"name_pretty": "Cloud Spanner Django",
"product_documentation": "https://cloud.google.com/spanner/docs/",
"api_shortname": "django-google-spanner",
"client_documentation": "https://googleapis.dev/python/django-google-spanner/latest",
"distribution_name": "django-google-spanner",
"issue_tracker": "https://issuetracker.google.com/issues?q=componentid:190851%2B%20status:open",
"release_level": "stable",
"language": "python",
"library_type": "INTEGRATION",
"repo": "googleapis/google-cloud-python",
"distribution_name": "django-google-spanner",
"requires_billing": true,
"api_shortname": "django-google-spanner"
}
"name": "django-google-spanner",
"name_pretty": "Cloud Spanner Django",
"product_documentation": "https://cloud.google.com/spanner/docs/",
"release_level": "stable",
"repo": "googleapis/google-cloud-python"
}
Loading
Loading