diff --git a/docs/cells.md b/docs/cells.md new file mode 100644 index 000000000..c668994f8 --- /dev/null +++ b/docs/cells.md @@ -0,0 +1,144 @@ +### Cells ### + +Cells represents an abstraction of the web, tightly coupled with object database, which allows you to create "reactive" interface in python.... + + +### Preamble ### + +In order to get ourselves up and running we'll need a few things: + +* Object Database installed (see the [installation docs](../INSTALLATION.md) for more details) +* an object_database engine service instance (see [here](./object_engine.md) for more information) +* build the bundle for the services landing page + * `cd object_database/object_database/web/content` + * create a node environment and source it + * run `npm install && npm run build` +* a configured and installed web service instance, which will be responsible for building and serving the web application +* and an installed instance of the service we'd like to run + +After this we can start up our web server and service. + +Lets walk through it (as in the other examples we'll use "TOKEN" for our special token): + +In a python virtual environment boot up Object Database engine like so: +``` + object_database_service_manager \ + localhost \ + localhost \ + 8000 \ + Master \ + --run_db \ + --service-token TOKEN \ + --source ./odb/source \ + --storage ./odb/storage +``` + +In another python virtual environment instance run the following: +``` +# export our special Object Database token +export ODB_AUTH_TOKEN=TOKEN + +# install the ActiveWebService +object_database_service_config install \ +--class object_database.web.ActiveWebService.ActiveWebService \ +--placement Master + +# configure ActiveWebService +object_database_service_config configure ActiveWebService \ +--port 8080 --hostname localhost --internal-port 8081 + +# check to make sure it is listed +object_database_service_config list + +# start it up +object_database_service_config start ActiveWebService + +# check to see that it is running +object_database_service_config instances +``` + +NOTE: you can always open [http://localhost:8080/](http://localhost:8080/) in your browser to see the running services and click to see what they are. Also, if you get tired of running the above commands, there is a small bash script found [here](./examples/aws_start.sh). + +#### Running a boring web app #### + +Run the following in your virtual environment: + +``` +object_database_service_config install --class object_database.service_manager.ServiceBase.ServiceBase --placement Master +# check to see it is installed +object_database_service_config list +# start +object_database_service_config start ServiceBase +# check to see it is running +object_database_service_config instances +``` + +(NOTE: you might need to change the paths to the [ServiceBase](https://github.com/APrioriInvestments/object_database/blob/dev/object_database/service_manager/ServiceBase.py) file depending on the directory you are running this from.) + +If you head to [http://localhost:8080/services/ServiceBase](http://localhost:8080/services/ServiceBase) you will see our really boring service. The simple text holder card is what [ServiceBase.serviceDisplay](https://github.com/APrioriInvestments/object_database/blob/dev/object_database/service_manager/ServiceBase.py#L67) returns. In the next example, we'll subclass `ServiceBase` and change this method to do something more interesting. + + +#### Running a more interesting web app #### + +Take a look at [cells.py]('./examples/cells.py'). You'll see we made a subclass of `ServiceBase` and overrode its `.serviceDisplay` method. There is card with a header and some buttons which all route you to the corresponding URI (in our case the list of services we have running), plus a little bit of styling. + +Lets install our more interesting app like above: +``` +object_database_service_config install --class docs.examples.cells.SomethingMoreInteresting --placement Master +object_database_service_config start SomethingMoreInteresting +``` + +Note: when you make changes to `SomethingMoreInteresting` you need to reinstall it, with the above command. If you see a message like `Cannot set codebase of locked service 'SomethingMoreInteresting'` then click the "Locked" icon in [http://localhost:8080/services]([http://localhost:8080/services) to unlock it, then reinstall. + + +You can learn more about cells by perusing the [cells](https://github.com/APrioriInvestments/object_database/tree/dev/object_database/web/cells) directory or taking a look at the [cells test example](https://github.com/APrioriInvestments/object_database/blob/dev/object_database/web/CellsTestService.py#L63) + +#### Running an ODB web app #### + +But before we wrap up, we should really build a cells examples which works with Object Database, since that's the main point here. + +I am going to skip over details of how ODB works. Please here [here](https://github.com/APrioriInvestments/object_database/blob/dev/docs/object_engine.md) for an introduction. + +We'll be building an `AnODBService` which you can find [here](https://github.com/APrioriInvestments/object_database/blob/daniel-examples/docs/examples/cells_odb.py). You'll see we need to define +* a [schema](https://github.com/APrioriInvestments/object_database/blob/daniel-examples/docs/examples/cells_odb.py#L12) +* how our app will [interact with ODB](https://github.com/APrioriInvestments/object_database/blob/daniel-examples/docs/examples/cells_odb.py#L23) +* and the [UI](https://github.com/APrioriInvestments/object_database/blob/daniel-examples/docs/examples/cells_odb.py#L40) for the app itself. + +The app will send and recieve messages from the database, and update the UI which consists largely of a Panel and Table cell. The key departure here from the previous examples is the lambda functions passed to the cells. Instead of returning something like a string (which then tells the service to route to the correponding URI), these interact with the DB via the `Message` class. + +As before we'll need to install and start the service: +Lets install our more interesting app like above: +``` +object_database_service_config install --class docs.examples.cells_db.AnODBService --placement Master +object_database_service_config start AnODBService +``` + +We'll learn more about cells and how to develop them in the upcoming `cells_dev.md` doc. + + +#### ODB Cells Playground #### + +ODB provides a playground where you can explore and see examples of various cells in action. Running `object_database_webtest` and then heading to [http://localhost:8000/services](http://localhost:8000) you will see a cells test service. If you update the code in the editor and press ctrl-n-enter the cell will refresh in the browser. This is one of the better way to explore cells. + +#### A Note About slots #### + +In the ODB playground (mentioned above) you will see a number of examples with `cells.Slot()`'s and `cells.Subscribed()`'s. A cell which is subscribed to a slot will automatically update when the contents of the slot change. This can happen from within the UI itselt on the client side, or from the ODB on the server side. Slots allow you to build reactive web applications. + +The slot API is very simple `slot.set([value])` sets the value and `slot.get([value])` retrieves it. + +Here is an example of a dropdown cell element which displays the contents of a slot but also changes those contents based on the user selection. + +```python +slot = cells.Slot("you haven't picked anything yet") + +dropdown = cells.Subscribed( + lambda: + cells.Dropdown( + str(slot.get()), + ["option 1", "option 2", "option 3"], + lambda i: slot.set(i) + ) + ) +``` + +The dropdown selection calls `lambda i: slot(i)` which takes the selected value and sets the slot, while the display value is defined by `slot.get()`. The dropdown is notified when the slot value changes, calls .`get()` on it and udpates the display. diff --git a/docs/cells_dev.md b/docs/cells_dev.md new file mode 100644 index 000000000..e9d4a5677 --- /dev/null +++ b/docs/cells_dev.md @@ -0,0 +1,193 @@ +## Cells Development ## + +This document is intended for those interested in developing cells. We'll see how to customize currently available cells, as well as develop new ones to add to the object database ecosystem. + +We'll assume you have a decent idea about how things fit together (cells, object database, typed python and so on). But if you need a refresher on specifics take a look at the corresponding docs [here](https://github.com/APrioriInvestments/object_database/tree/docs). We'll also assume that you have gone through the [cells.md](./cells.md) doc and will be using the services developed there as a starting point. + + +### Basic Overview ### + +Leaving the server and related backend infrastructure aside, cells consist of two main components. The [python classes](https://github.com/APrioriInvestments/object_database/tree/dev/object_database/web/cells) which are the cells themselves and the corresponding [JS classes](https://github.com/APrioriInvestments/object_database/tree/dev/object_database/web/content/src/components) which are responsible for generated the html/js/css etc. Every python cell has a corresponding JS cell, but not all of these strictly generate DOM elements. Some are utilities for styling, layouts, events etc. + +For some starter examples take a look at the following: +* border: [python](https://github.com/APrioriInvestments/object_database/blob/dev/object_database/web/cells/border.py) and [JS](https://github.com/APrioriInvestments/object_database/blob/dev/object_database/web/content/src/components/Border.js) +* layout/flex: [python](https://github.com/APrioriInvestments/object_database/blob/dev/object_database/web/cells/flex.py) and [JS](https://github.com/APrioriInvestments/object_database/blob/dev/object_database/web/content/src/components/Flex.js) +* key events: [JS](https://github.com/APrioriInvestments/object_database/blob/dev/object_database/web/content/src/components/KeyAction.js) + + +### Making changes ### + +In our [cells.md](./cells.md) doc we made a [__SlightlyMoreInteresting__ service](./examples/cells.py) where we strung together a number of button cells which linked to the different installed services. + +The first task will be to modify the button. Lets take a look at the [Button.build()](https://github.com/APrioriInvestments/object_database/blob/dev/object_database/web/content/src/components/Button.js#L32). You'll see that we use 'hyperscript' h() notation to build the actual DOM elements. + +Suppose you decide that all buttons need to have a padding, as an inline style. For this simply add `style: "padding: 5px"` argument to h(), then rebuild the bundle (`npm run build` in the [content](https://github.com/APrioriInvestments/object_database/tree/dev/object_database/web/content) directory) and refresh. You should see the button names padded with 5px. + +Of course this is a hardcoded change, which you cannot control from the python side, so lets try to make this a bit more configurable. + +Lets go to our [python Button]() class and a `padding="5px"` keyword argument. Then we'll make sure that to export the data to the JS side which happens in the `Button.recalculate()` method. + +Your python Button classshould now look like this: +``` +class Button(Clickable): + def __init__(self, *args, small=False, active=True, style="primary", + padding="5px", **kwargs): + Clickable.__init__(self, *args, **kwargs) + self.small = small + self.active = active + self.style = style + self.padding = padding + + ... + + def recalculate(self): + super().recalculate() + + self.exportData["small"] = bool(self.small) + self.exportData["active"] = bool(self.active) + self.exportData["style"] = self.style + self.exportData["padding"] = self.padding + ... +``` + +The `padding` argument will now be sent along with props, which subsequently can be handled on the JS side as needed. For example, the JS Button class could now look like this: +``` +class Button extends ConcreteCell { + constructor(props, ...args){ + super(props, ...args); + + // Bind context to methods + this.onClick = this.onClick.bind(this); + this._getHTMLClasses = this._getHTMLClasses.bind(this); + + this.buttonDiv = null; + // Our new padding arg! + self.padding = props.padding; + } + +... + + build() { + this.buttonDiv = h('div', { + id: this.getElementId(), + "data-cell-id": this.identity, + "data-cell-type": "Button", + class: this._getHTMLClasses(), + style: `padding: ${self.padding}`, // using padding here! + onclick: this.onClick, + tabindex: -1, + onmousedown: (event) => { + // prevent the event from causing us to focus since we just want a + // click + event.preventDefault(); + } + }, [this.renderChildNamed('content')]); + + let res = h( + 'div', + {'class': 'allow-child-to-fill-space button-holder'}, + [this.buttonDiv] + ); + + this.applySpacePreferencesToClassList(this.buttonDiv); + this.applySpacePreferencesToClassList(res); + + return res; + } +... +``` + +Of course you could add more logic, as well as change things beoynd styling (the DOM element itself, event handling and so on). + +### Making a new cell ### + +If the current collections of cells is not sufficient, you can always make a new one. + +There are three steps here: +* create a Python cells class +* create the corresponding JS cells class +* register the classes at the object database system and module levels, as well as with the web bundle + +Lets create an `OurNewCell` cell that displays some text with a few basic options. + +Create a file `our_new_cell.py` in the [cells directory](https://github.com/APrioriInvestments/object_database/tree/dev/object_database/web/cells) with the following code: + +``` +from object_database.web.cells.cell import Cell + +class OurNewCell(Cell): + def __init__(self, text, makeBold=False): + super().__init__() + self.text = test + self.bold = makeBold + + def recalculate(self): + self.exportData["text"] = self.text + self.exportData["bold"] = self.bold +``` + +Create a file `OurNewCell.js` in the js [components directory](https://github.com/APrioriInvestments/object_database/tree/dev/object_database/web/content/src/components) with the following code: + +``` +import {ConcreteCell} from './ConcreteCell'; +import {makeDomElt as h} from './Cell'; + +class OurNewCell extends ConcreteCell { + constructor(props, ...args){ + super(props, ...args); + this.bold = props.bold; + this.text = props.text; + } + + build() { + let style = "text-align: center"; + if(this.bold){ + style += "; font-weight: bold"; + }; + let res = h( + 'div', + {style: style}, + [this.text] + ); + return res; + } +} + +export {OurNewCell, OurNewCell as default}; +``` + +Update the various registers with your new cells class: +* [JS components registry](https://github.com/APrioriInvestments/object_database/blob/b20b6c280b09f7381c9ac9900945a33e234eb621/object_database/web/content/ComponentRegistry.js) +* [object database module init](https://github.com/APrioriInvestments/object_database/blob/dev/object_database/web/cells/__init__.py) + +Now lets update our [cells.py](./examples/cells.py) examples we used in [cells.md](./cells.md) to use our new cell: +``` +import object_database.web.cells as cells +from object_database import ServiceBase + + +class SomethingMoreInteresting(ServiceBase): + def initialize(self): + self.buttonName = "click me" + return + + @staticmethod + def serviceDisplay(serviceObject, instance=None, objType=None, queryArgs=None): + return cells.Card( + cells.Panel( + cells.OurNewCell("This is our new cell", makeBold=True) + + cells.Button("Reload", lambda: "") + + cells.Button("Service Base", lambda: "ServiceBase") + + cells.Button("Active Web Service", lambda: "ActiveWebService") + ), + header="This is a 'card' cell with some buttons", + padding="10px" + ) +``` + +The last step is to rebuild the bundle `npm run build` in the [components](https://github.com/APrioriInvestments/object_database/tree/dev/object_database/web/content/src/components) directory and then reinstall our service as before: +``` +object_database_service_config install --class docs.examples.cells.SomethingMoreInteresting --placement Master +``` + +You should see your new cell appear when you click on the `SomethingMoreInteresting` service. diff --git a/docs/examples/__init__.py b/docs/examples/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/docs/examples/aws_start.sh b/docs/examples/aws_start.sh new file mode 100644 index 000000000..e17134a7f --- /dev/null +++ b/docs/examples/aws_start.sh @@ -0,0 +1,22 @@ +## Active Web Service startup script + +# export our special Object Database token +export ODB_AUTH_TOKEN=TOKEN + +# install the ActiveWebService +object_database_service_config install \ +--class object_database.web.ActiveWebService.ActiveWebService \ +--placement Master + +# configure ActiveWebService +object_database_service_config configure ActiveWebService \ +--port 8080 --hostname localhost --internal-port 8081 + +# check to make sure it is listed +object_database_service_config list + +# start it up +object_database_service_config start ActiveWebService + +# check to see that it is running +object_database_service_config instances diff --git a/docs/examples/cells.py b/docs/examples/cells.py new file mode 100644 index 000000000..3cb393d8f --- /dev/null +++ b/docs/examples/cells.py @@ -0,0 +1,20 @@ +import object_database.web.cells as cells +from object_database import ServiceBase + + +class SomethingMoreInteresting(ServiceBase): + def initialize(self): + self.buttonName = "click me" + return + + @staticmethod + def serviceDisplay(serviceObject, instance=None, objType=None, queryArgs=None): + return cells.Card( + cells.Panel( + cells.Button("Reload", lambda: "") + + cells.Button("Service Base", lambda: "ServiceBase") + + cells.Button("Active Web Service", lambda: "ActiveWebService") + ), + header="This is a 'card' cell with some buttons", + padding="10px" + ) diff --git a/docs/examples/cells_odb.py b/docs/examples/cells_odb.py new file mode 100644 index 000000000..8512d90a6 --- /dev/null +++ b/docs/examples/cells_odb.py @@ -0,0 +1,81 @@ +from object_database import Schema, ServiceBase +import object_database.web.cells as cells +import time +import random + +# define a 'schema', which is a collection of classes we can subscribe to as a group +schema = Schema("cells_obd") + +# define a type of entry in odb. We'll have one instance of this class for each +# message in the database +@schema.define +class Message: + timestamp = float + message = str + lifetime = float + + +class AnODBService(ServiceBase): + def initialize(self): + # make sure we're subscribed to all objects in our schema. + self.db.subscribeToSchema(schema) + + def doWork(self, shouldStop): + # this is the main entrypoint for the service - it gets to do work here. + while not shouldStop.is_set(): + #wake up every 100ms and look at the objects in the ODB. + time.sleep(.1) + + # delete any messages more than 10 seconds old + with self.db.transaction(): + # get all the messages + messages = Message.lookupAll() + + for m in messages: + if m.timestamp < time.time() - m.lifetime: + # this will actually delete the object from the ODB. + m.delete() + + @staticmethod + def serviceDisplay(serviceObject, instance=None, objType=None, queryArgs=None): + # make sure cells has loaded these classes in the database and subscribed + # to all the objects. + cells.ensureSubscribedSchema(schema) + + def newMessage(): + # calling the constructor creates a new message object. Even though we + # orphan it immediately, we can always get it back by calling + # Message.lookupAll() + # because ODB objects have an explicit lifetime (they have to be destroyed) + Message(timestamp=time.time(), message=editBox.currentText.get(), lifetime=20) + + # reset our edit box so we can type again + editBox.currentText.set("") + + # define an 'edit box' cell. The user can type into this. + editBox = cells.SingleLineTextBox(onEnter=lambda newText: newMessage()) + + return cells.Panel( + editBox >> cells.Button( + "New Message", + newMessage + ) + ) + ( + cells.Table( + colFun=lambda: ['timestamp', 'lifetime', 'message'], + rowFun=lambda: sorted(Message.lookupAll(), key=lambda m: -m.timestamp), + headerFun=lambda x: x, + rendererFun=lambda m, col: cells.Subscribed( + lambda: + cells.Timestamp(m.timestamp) if col == 'timestamp' else + m.message if col == 'message' else + cells.Dropdown( + m.lifetime, + [1, 2, 5, 10, 20, 60, 300], + lambda val: setattr(m, 'lifetime', val) + ) + ), + maxRowsPerPage=100, + fillHeight=True + ) + ) diff --git a/docs/examples/consumer.py b/docs/examples/consumer.py index a0d1a6a43..66888c7d1 100644 --- a/docs/examples/consumer.py +++ b/docs/examples/consumer.py @@ -9,15 +9,14 @@ class Message: timestamp = float message = str - db = connect('localhost', 8000, 'TOKEN') + db.subscribeToSchema(schema) ts = 0 while True: - with db.view(): - Message(timestamp=time.time(), - message='message_5') + with db.transaction(): + Message(timestamp=time.time(), message="message_5") messages = Message.lookupAll() for m in messages: if m.timestamp > ts: diff --git a/quickstart/Dockerfile b/quickstart/Dockerfile index 5116d5ba5..46e0d3bd8 100644 --- a/quickstart/Dockerfile +++ b/quickstart/Dockerfile @@ -1,17 +1,20 @@ # Using 3.8 because typed_python doesn't seem to # compile with 3.9 (as of 2022-04-27) -FROM python:3.8-slim as builder +FROM python:3.9-slim as builder -ADD . /opt/object_database +ADD ./object_database /opt/object_database +ADD ./typed_python /opt/typed_python RUN apt update -y -qq \ && apt upgrade -y -qq \ + && apt-get update \ && apt install -y -qq curl libssl-dev build-essential git npm \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* RUN pip install --upgrade pip \ - && pip install typed_python \ + && cd /opt/typed_python \ + && pip install -e . \ && cd /opt/object_database \ && pip install -e . @@ -19,4 +22,4 @@ RUN cd /opt/object_database/object_database/web/content \ && npm install \ && npm run build -CMD ["object_database_webtest"] \ No newline at end of file +CMD ["object_database_webtest"]