perspective
The Python language bindings for Perspective, a high performance data-visualization and analytics component for the web browser.
A simple example which loads an Apache Arrow and computes a "Group By" operation, returning a new Arrow.
from perspective import Server
client = Server().new_local_client()
table = client.table(arrow_bytes_data)
view = table.view(group_by = ["CounterParty", "Security"])
arrow = view.to_arrow()
Perspective for Python uses the exact same C++ data engine used by the WebAssembly version and Rust version. The library consists of many of the same abstractions and API as in JavaScript, as well as Python-specific data loading support for NumPy, Pandas (and Apache Arrow, as in JavaScript).
Additionally, perspective-python
provides a session manager suitable for
integration into server systems such as
Tornado websockets,
AIOHTTP, or
Starlette/FastAPI,
which allows fully _virtual_ Perspective tables to be interacted with by
multiple <perspective-viewer>
in a web browser. You can also interact with a
Perspective table from python clients, and to that end client libraries are
implemented for both Tornado and AIOHTTP.
As <perspective-viewer>
will only consume the data necessary to render the
current screen, this runtime mode allows _ludicrously-sized_ datasets with
instant-load after they've been manifest on the server (at the expense of
network latency on UI interaction).
The included PerspectiveWidget
allows running such a viewer in
JupyterLab in either server or
client (via WebAssembly) mode, and the included PerspectiveTornadoHandler
makes it simple to extend a Tornado server with virtual Perspective support.
The perspective
module exports several tools:
Server
the constructor for a new isntance of the Perspective data engine.- The
perspective.widget
module exportsPerspectiveWidget
, the JupyterLab widget for interactive visualization in a notebook cell. - The
perspective.handlers
modules exports web frameworks handlers that interface with aperspective-client
in JavaScript.
This user's guide provides an overview of the most common ways to use
Perspective in Python: the Table
API, the JupyterLab widget, and the Tornado
handler.
More Examples are available on GitHub.
Installation
perspective-python
contains full bindings to the Perspective API, a JupyterLab
widget, and a WebSocket handlers for several webserver libraries that allow you
to host Perspective using server-side Python.
perspective-python
can be installed from PyPI via pip
:
pip install perspective-python
Quick Start
A Table
can be created from a dataset or a schema, the specifics of which are
discussed in the JavaScript section of the user's
guide. In Python, however, Perspective supports additional data types that are
commonly used when processing data:
pandas.DataFrame
polars.DataFrame
bytes
(encoding an Apache Arrow)objects
(either extracting a repr or via reference)str
(encoding as a CSV)
A Table
is created in a similar fashion to its JavaScript equivalent:
from datetime import date, datetime
import numpy as np
import pandas as pd
import perspective
data = pd.DataFrame({
"int": np.arange(100),
"float": [i * 1.5 for i in range(100)],
"bool": [True for i in range(100)],
"date": [date.today() for i in range(100)],
"datetime": [datetime.now() for i in range(100)],
"string": [str(i) for i in range(100)]
})
table = perspective.table(data, index="float")
Likewise, a View
can be created via the view()
method:
view = table.view(group_by=["float"], filter=[["bool", "==", True]])
column_data = view.to_columns()
row_data = view.to_json()
Pandas and Polars Support
Perspective's Table
can be constructed from pandas.DataFrame
and
polars.DataFrame
objects. Internally, this just uses
pyarrow::from_pandas
,
which dictates behavior of this feature including type support.
If the dataframe does not have an index set, an integer-typed column named
"index"
is created. If you want to preserve the indexing behavior of the
dataframe passed into Perspective, simply create the Table
with
index="index"
as a keyword argument. This tells Perspective to once again
treat the index as a primary key:
data.set_index("datetime")
table = perspective.table(data, index="index")
Time Zone Handling
When parsing "datetime"
strings, times are assumed _local time_ unless an
explicit timezone offset is parsed. All "datetime"
columns (regardless of
input time zone) are _output_ to the user as datetime.datetime
objects in
_local time_ according to the Python runtime.
This behavior is consistent with Perspective's behavior in JavaScript. For more
details, see this in-depth
explanation of
perspective-python
semantics around time zone handling.
Callbacks and Events
perspective.Table
allows for on_update
and on_delete
callbacks to be
set—simply call on_update
or on_delete
with a reference to a function or a
lambda without any parameters:
def update_callback():
print("Updated!")
# set the update callback
on_update_id = view.on_update(update_callback)
def delete_callback():
print("Deleted!")
# set the delete callback
on_delete_id = view.on_delete(delete_callback)
# set a lambda as a callback
view.on_delete(lambda: print("Deleted x2!"))
If the callback is a named reference to a function, it can be removed with
remove_update
or remove_delete
:
view.remove_update(on_update_id)
view.remove_delete(on_delete_id)
Callbacks defined with a lambda function cannot be removed, as lambda functions have no identifier.
Hosting Table
and View
instances
Server
"hosts" all perspective.Table
and perspective.View
instances
created by its connected Client
s. Hosted tables/views can have their methods
called from other sources than the Python server, i.e. by a perspective-viewer
running in a JavaScript client over the network, interfacing with
perspective-python
through the websocket API.
The server has full control of all hosted Table
and View
instances, and can
call any public API method on hosted instances. This makes it extremely easy to
stream data to a hosted Table
using .update()
:
server = perspective.Server()
client = server.new_local_client()
table = client.table(data, name="data_source")
for i in range(10):
# updates continue to propagate automatically
table.update(new_data)
The name
provided is important, as it enables Perspective in JavaScript to
look up a Table
and get a handle to it over the network. Otherwise, name
will be assigned randomlu and the Client
must look this up with
CLient.get_hosted_table_names()
Client/Server Replicated Mode
Using Tornado and
PerspectiveTornadoHandler
, as well as
Perspective
's JavaScript library, we can set up "distributed" Perspective
instances that allows multiple browser perspective-viewer
clients to read from
a common perspective-python
server, as in the
Tornado Example Project.
This architecture works by maintaining two Tables
—one on the server, and one
on the client that mirrors the server's Table
automatically using on_update
.
All updates to the table on the server are automatically applied to each client,
which makes this architecture a natural fit for streaming dashboards and other
distributed use-cases. In conjunction with multithreading,
distributed Perspective offers consistently high performance over large numbers
of clients and large datasets.
_*server.py*_
from perspective import Server
from perspective.hadnlers.tornado import PerspectiveTornadoHandler
# Create an instance of Server, and host a Table
SERVER = Server()
CLIENT = SERVER.new_local_client()
# The Table is exposed at `localhost:8888/websocket` with the name `data_source`
client.table(data, name = "data_source")
app = tornado.web.Application([
# create a websocket endpoint that the client JavaScript can access
(r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER})
])
# Start the Tornado server
app.listen(8888)
loop = tornado.ioloop.IOLoop.current()
loop.start()
Instead of calling load(server_table)
, create a View
using server_table
and pass that into viewer.load()
. This will automatically register an
on_update
callback that synchronizes state between the server and the client.
_*index.html*_
<perspective-viewer id="viewer" editable></perspective-viewer>
<script type="module">
// Create a client that expects a Perspective server
// to accept connections at the specified URL.
const websocket = await perspective.websocket(
"ws://localhost:8888/websocket"
);
// Get a handle to the Table on the server
const server_table = await websocket.open_table("data_source_one");
// Create a new view
const server_view = await table.view();
// Create a Table on the client using `perspective.worker()`
const worker = await perspective.worker();
const client_table = await worker.table(view);
// Load the client table in the `<perspective-viewer>`.
document.getElementById("viewer").load(client_table);
</script>
For a more complex example that offers distributed editing of the server dataset, see client_server_editing.html.
We also provide examples for Starlette/FastAPI and AIOHTTP:
Server-only Mode
The server setup is identical to Distributed Mode above,
but instead of creating a view, the client calls load(server_table)
: In
Python, use Server
and PerspectiveTornadoHandler
to create a websocket
server that exposes a Table
. In this example, table
is a proxy for the
Table
we created on the server. All API methods are available on _proxies_,
the.g.us calling view()
, schema()
, update()
on table
will pass those
operations to the Python Table
, execute the commands, and return the result
back to Javascript.
<perspective-viewer id="viewer" editable></perspective-viewer>
const websocket = perspective.websocket("ws://localhost:8888/websocket");
const table = websocket.open_table("data_source");
document.getElementById("viewer").load(table);
1# ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ 2# ┃ ██████ ██████ ██████ █ █ █ █ █ █▄ ▀███ █ ┃ 3# ┃ ▄▄▄▄▄█ █▄▄▄▄▄ ▄▄▄▄▄█ ▀▀▀▀▀█▀▀▀▀▀ █ ▀▀▀▀▀█ ████████▌▐███ ███▄ ▀█ █ ▀▀▀▀▀ ┃ 4# ┃ █▀▀▀▀▀ █▀▀▀▀▀ █▀██▀▀ ▄▄▄▄▄ █ ▄▄▄▄▄█ ▄▄▄▄▄█ ████████▌▐███ █████▄ █ ▄▄▄▄▄ ┃ 5# ┃ █ ██████ █ ▀█▄ █ ██████ █ ███▌▐███ ███████▄ █ ┃ 6# ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫ 7# ┃ Copyright (c) 2017, the Perspective Authors. ┃ 8# ┃ ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ ┃ 9# ┃ This file is part of the Perspective library, distributed under the terms ┃ 10# ┃ of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). ┃ 11# ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ 12 13__version__ = "3.7.4" 14__all__ = [ 15 "_jupyter_labextension_paths", 16 "Server", 17 "Client", 18 "Table", 19 "View", 20 "PerspectiveError", 21 "ProxySession", 22 "AsyncClient", 23 "AsyncServer", 24 "num_cpus", 25 "set_num_cpus", 26 "system_info", 27] 28 29__doc__ = """ 30The Python language bindings for [Perspective](https://perspective.finos.org), a 31high performance data-visualization and analytics component for the web browser. 32 33A simple example which loads an [Apache Arrow](https://arrow.apache.org/) and 34computes a "Group By" operation, returning a new Arrow. 35 36```python 37from perspective import Server 38 39client = Server().new_local_client() 40table = client.table(arrow_bytes_data) 41view = table.view(group_by = ["CounterParty", "Security"]) 42arrow = view.to_arrow() 43``` 44 45Perspective for Python uses the exact same C++ data engine used by the 46[WebAssembly version](https://docs.rs/perspective-js/latest/perspective_js/) and 47[Rust version](https://docs.rs/crate/perspective/latest). The library consists 48of many of the same abstractions and API as in JavaScript, as well as 49Python-specific data loading support for [NumPy](https://numpy.org/), 50[Pandas](https://pandas.pydata.org/) (and 51[Apache Arrow](https://arrow.apache.org/), as in JavaScript). 52 53Additionally, `perspective-python` provides a session manager suitable for 54integration into server systems such as 55[Tornado websockets](https://www.tornadoweb.org/en/stable/websocket.html), 56[AIOHTTP](https://docs.aiohttp.org/en/stable/web_quickstart.html#websockets), or 57[Starlette](https://www.starlette.io/websockets/)/[FastAPI](https://fastapi.tiangolo.com/advanced/websockets/), 58which allows fully _virtual_ Perspective tables to be interacted with by 59multiple `<perspective-viewer>` in a web browser. You can also interact with a 60Perspective table from python clients, and to that end client libraries are 61implemented for both Tornado and AIOHTTP. 62 63As `<perspective-viewer>` will only consume the data necessary to render the 64current screen, this runtime mode allows _ludicrously-sized_ datasets with 65instant-load after they've been manifest on the server (at the expense of 66network latency on UI interaction). 67 68The included `PerspectiveWidget` allows running such a viewer in 69[JupyterLab](https://jupyterlab.readthedocs.io/en/stable/) in either server or 70client (via WebAssembly) mode, and the included `PerspectiveTornadoHandler` 71makes it simple to extend a Tornado server with virtual Perspective support. 72 73The `perspective` module exports several tools: 74 75- `Server` the constructor for a new isntance of the Perspective data engine. 76- The `perspective.widget` module exports `PerspectiveWidget`, the JupyterLab 77 widget for interactive visualization in a notebook cell. 78- The `perspective.handlers` modules exports web frameworks handlers that 79 interface with a `perspective-client` in JavaScript. 80 - `perspective.handlers.tornado.PerspectiveTornadoHandler` for 81 [Tornado](https://www.tornadoweb.org/) 82 - `perspective.handlers.starlette.PerspectiveStarletteHandler` for 83 [Starlette](https://www.starlette.io/) and 84 [FastAPI](https://fastapi.tiangolo.com) 85 - `perspective.handlers.aiohttp.PerspectiveAIOHTTPHandler` for 86 [AIOHTTP](https://docs.aiohttp.org), 87 88This user's guide provides an overview of the most common ways to use 89Perspective in Python: the `Table` API, the JupyterLab widget, and the Tornado 90handler. 91 92[More Examples](https://github.com/finos/perspective/tree/master/examples) are 93available on GitHub. 94 95## Installation 96 97`perspective-python` contains full bindings to the Perspective API, a JupyterLab 98widget, and a WebSocket handlers for several webserver libraries that allow you 99to host Perspective using server-side Python. 100 101`perspective-python` can be installed from [PyPI](https://pypi.org) via `pip`: 102 103```bash 104pip install perspective-python 105``` 106 107## Quick Start 108 109A `Table` can be created from a dataset or a schema, the specifics of which are 110[discussed](#loading-data-with-table) in the JavaScript section of the user's 111guide. In Python, however, Perspective supports additional data types that are 112commonly used when processing data: 113 114- `pandas.DataFrame` 115- `polars.DataFrame` 116- `bytes` (encoding an Apache Arrow) 117- `objects` (either extracting a repr or via reference) 118- `str` (encoding as a CSV) 119 120A `Table` is created in a similar fashion to its JavaScript equivalent: 121 122```python 123from datetime import date, datetime 124import numpy as np 125import pandas as pd 126import perspective 127 128data = pd.DataFrame({ 129 "int": np.arange(100), 130 "float": [i * 1.5 for i in range(100)], 131 "bool": [True for i in range(100)], 132 "date": [date.today() for i in range(100)], 133 "datetime": [datetime.now() for i in range(100)], 134 "string": [str(i) for i in range(100)] 135}) 136 137table = perspective.table(data, index="float") 138``` 139 140Likewise, a `View` can be created via the `view()` method: 141 142```python 143view = table.view(group_by=["float"], filter=[["bool", "==", True]]) 144column_data = view.to_columns() 145row_data = view.to_json() 146``` 147 148#### Pandas and Polars Support 149 150Perspective's `Table` can be constructed from `pandas.DataFrame` and 151`polars.DataFrame` objects. Internally, this just uses 152[`pyarrow::from_pandas`](https://arrow.apache.org/docs/python/pandas.html), 153which dictates behavior of this feature including type support. 154 155If the dataframe does not have an index set, an integer-typed column named 156`"index"` is created. If you want to preserve the indexing behavior of the 157dataframe passed into Perspective, simply create the `Table` with 158`index="index"` as a keyword argument. This tells Perspective to once again 159treat the index as a primary key: 160 161```python 162data.set_index("datetime") 163table = perspective.table(data, index="index") 164``` 165 166#### Time Zone Handling 167 168When parsing `"datetime"` strings, times are assumed _local time_ unless an 169explicit timezone offset is parsed. All `"datetime"` columns (regardless of 170input time zone) are _output_ to the user as `datetime.datetime` objects in 171_local time_ according to the Python runtime. 172 173This behavior is consistent with Perspective's behavior in JavaScript. For more 174details, see this in-depth 175[explanation](https://github.com/finos/perspective/pull/867) of 176`perspective-python` semantics around time zone handling. 177 178#### Callbacks and Events 179 180`perspective.Table` allows for `on_update` and `on_delete` callbacks to be 181set—simply call `on_update` or `on_delete` with a reference to a function or a 182lambda without any parameters: 183 184```python 185def update_callback(): 186 print("Updated!") 187 188# set the update callback 189on_update_id = view.on_update(update_callback) 190 191 192def delete_callback(): 193 print("Deleted!") 194 195# set the delete callback 196on_delete_id = view.on_delete(delete_callback) 197 198# set a lambda as a callback 199view.on_delete(lambda: print("Deleted x2!")) 200``` 201 202If the callback is a named reference to a function, it can be removed with 203`remove_update` or `remove_delete`: 204 205```python 206view.remove_update(on_update_id) 207view.remove_delete(on_delete_id) 208``` 209 210Callbacks defined with a lambda function cannot be removed, as lambda functions 211have no identifier. 212 213### Hosting `Table` and `View` instances 214 215`Server` "hosts" all `perspective.Table` and `perspective.View` instances 216created by its connected `Client`s. Hosted tables/views can have their methods 217called from other sources than the Python server, i.e. by a `perspective-viewer` 218running in a JavaScript client over the network, interfacing with 219`perspective-python` through the websocket API. 220 221The server has full control of all hosted `Table` and `View` instances, and can 222call any public API method on hosted instances. This makes it extremely easy to 223stream data to a hosted `Table` using `.update()`: 224 225```python 226server = perspective.Server() 227client = server.new_local_client() 228table = client.table(data, name="data_source") 229 230for i in range(10): 231 # updates continue to propagate automatically 232 table.update(new_data) 233``` 234 235The `name` provided is important, as it enables Perspective in JavaScript to 236look up a `Table` and get a handle to it over the network. Otherwise, `name` 237will be assigned randomlu and the `Client` must look this up with 238`CLient.get_hosted_table_names()` 239 240### Client/Server Replicated Mode 241 242Using Tornado and 243[`PerspectiveTornadoHandler`](python.md#perspectivetornadohandler), as well as 244`Perspective`'s JavaScript library, we can set up "distributed" Perspective 245instances that allows multiple browser `perspective-viewer` clients to read from 246a common `perspective-python` server, as in the 247[Tornado Example Project](https://github.com/finos/perspective/tree/master/examples/python-tornado). 248 249This architecture works by maintaining two `Tables`—one on the server, and one 250on the client that mirrors the server's `Table` automatically using `on_update`. 251All updates to the table on the server are automatically applied to each client, 252which makes this architecture a natural fit for streaming dashboards and other 253distributed use-cases. In conjunction with [multithreading](#multi-threading), 254distributed Perspective offers consistently high performance over large numbers 255of clients and large datasets. 256 257_*server.py*_ 258 259```python 260from perspective import Server 261from perspective.hadnlers.tornado import PerspectiveTornadoHandler 262 263# Create an instance of Server, and host a Table 264SERVER = Server() 265CLIENT = SERVER.new_local_client() 266 267# The Table is exposed at `localhost:8888/websocket` with the name `data_source` 268client.table(data, name = "data_source") 269 270app = tornado.web.Application([ 271 # create a websocket endpoint that the client JavaScript can access 272 (r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER}) 273]) 274 275# Start the Tornado server 276app.listen(8888) 277loop = tornado.ioloop.IOLoop.current() 278loop.start() 279``` 280 281Instead of calling `load(server_table)`, create a `View` using `server_table` 282and pass that into `viewer.load()`. This will automatically register an 283`on_update` callback that synchronizes state between the server and the client. 284 285_*index.html*_ 286 287```html 288<perspective-viewer id="viewer" editable></perspective-viewer> 289 290<script type="module"> 291 // Create a client that expects a Perspective server 292 // to accept connections at the specified URL. 293 const websocket = await perspective.websocket( 294 "ws://localhost:8888/websocket" 295 ); 296 297 // Get a handle to the Table on the server 298 const server_table = await websocket.open_table("data_source_one"); 299 300 // Create a new view 301 const server_view = await table.view(); 302 303 // Create a Table on the client using `perspective.worker()` 304 const worker = await perspective.worker(); 305 const client_table = await worker.table(view); 306 307 // Load the client table in the `<perspective-viewer>`. 308 document.getElementById("viewer").load(client_table); 309</script> 310``` 311 312For a more complex example that offers distributed editing of the server 313dataset, see 314[client_server_editing.html](https://github.com/finos/perspective/blob/master/examples/python-tornado/client_server_editing.html). 315 316We also provide examples for Starlette/FastAPI and AIOHTTP: 317 318- [Starlette Example Project](https://github.com/finos/perspective/tree/master/examples/python-starlette). 319- [AIOHTTP Example Project](https://github.com/finos/perspective/tree/master/examples/python-aiohttp). 320 321### Server-only Mode 322 323The server setup is identical to [Distributed Mode](#distributed-mode) above, 324but instead of creating a view, the client calls `load(server_table)`: In 325Python, use `Server` and `PerspectiveTornadoHandler` to create a websocket 326server that exposes a `Table`. In this example, `table` is a proxy for the 327`Table` we created on the server. All API methods are available on _proxies_, 328the.g.us calling `view()`, `schema()`, `update()` on `table` will pass those 329operations to the Python `Table`, execute the commands, and return the result 330back to Javascript. 331 332```html 333<perspective-viewer id="viewer" editable></perspective-viewer> 334``` 335 336```javascript 337const websocket = perspective.websocket("ws://localhost:8888/websocket"); 338const table = websocket.open_table("data_source"); 339document.getElementById("viewer").load(table); 340``` 341 342""" 343 344 345import functools 346 347from .perspective import ( 348 Client, 349 PerspectiveError, 350 ProxySession, 351 Server, 352 AsyncServer, 353 AsyncClient, 354 # NOTE: these are classes without constructors, 355 # so we import them just for type hinting 356 Table, # noqa: F401 357 View, # noqa: F401 358 num_cpus, 359 set_num_cpus, 360) 361 362 363GLOBAL_SERVER = Server() 364GLOBAL_CLIENT = GLOBAL_SERVER.new_local_client() 365 366 367@functools.wraps(Client.table) 368def table(*args, **kwargs): 369 return GLOBAL_CLIENT.table(*args, **kwargs) 370 371 372@functools.wraps(Client.open_table) 373def open_table(*args, **kwargs): 374 return GLOBAL_CLIENT.table(*args, **kwargs) 375 376 377@functools.wraps(Client.get_hosted_table_names) 378def get_hosted_table_names(*args, **kwargs): 379 return GLOBAL_CLIENT.get_hosted_table_names(*args, **kwargs) 380 381 382@functools.wraps(Client.system_info) 383def system_info(*args, **kwargs): 384 return GLOBAL_CLIENT.system_info(*args, **kwargs) 385 386 387def _jupyter_labextension_paths(): 388 """ 389 Read by `jupyter labextension develop` 390 @private 391 """ 392 return [{"src": "labextension", "dest": "@finos/perspective-jupyterlab"}]
An instance of a Perspective server. Each Server instance is separate, and does not share Table (or other) data with other Servers.
Arguments
on_poll_request
A callback function which theServer
will invoke when there are updates that need to be flushed, after which you must _eventually_ callServer.poll
(or else no updates will be processed). This optimization allows batching updates, depending on context.
Create a new Session
bound to this Server.
Server.new_session
only needs to be called if you've implemented
a custom Perspective ['Client]/[
Server`] transport.
Flush pending updates to this Server, including notifications to
View.on_update
callbacks.
Server.poll
only needs to be called if you've implemented
a custom Perspective Server and provided the on_poll_request
constructor keyword argument.
Calling Session.poll
may result in the send_response
parameter
which was used to construct this (or other) Session
to fire.
Whenever a Session.handle_request
method is invoked for a
perspective_server::Server
, at least one Session.poll
should be
scheduled to clear other clients message queues.
poll()
_must_ be called after Table.update
or Table.remove
and on_poll_request
is notified, or the changes will not be applied.
An instance of a Client is a connection to a single Server, whether locally in-memory or remote over some transport like a WebSocket.
Client and Perspective objects derived from it have _synchronous_ APIs,
suitable for use in a repl or script context where this is the _only_
Client connected to its Server. If you want to
integrate with a Web framework or otherwise connect multiple clients,
use AsyncClient
.
Handle a message from the external message queue.
Client.handle_response
is part of the low-level message-handling
API necessary to implement new transports for a Client
connection to a local-or-remote Server, and
doesn't generally need to be called directly by "users" of a
Client once connected.
Creates a new Table from either a _schema_ or _data_.
The Client.table
factory function can be initialized with either a
_schema_ (see Table.schema
), or data in one of these formats:
- Apache Arrow
- CSV
- JSON row-oriented
- JSON column-oriented
- NDJSON
When instantiated with _data_, the schema is inferred from this data.
While this is convenient, inferrence is sometimes imperfect e.g.
when the input is empty, null or ambiguous. For these cases,
Client.table
can first be instantiated with a explicit schema.
When instantiated with a _schema_, the resulting Table is empty but
with known column names and column types. When subsqeuently
populated with Table.update
, these columns will be _coerced_ to
the schema's type. This behavior can be useful when
Client.table
's column type inferences doesn't work.
The resulting Table is _virtual_, and invoking its methods
dispatches events to the perspective_server::Server
this
Client connects to, where the data is stored and all calculation
occurs.
Arguments
arg
- Either _schema_ or initialization _data_.options
- Optional configuration which provides one of:limit
- The max number of rows the resulting Table can store.index
- The column name to use as an _index_ column. If thisTable
is being instantiated by _data_, this column name must be present in the data.name
- The name of the table. This will be generated if it is not provided.format
- The explicit format of the input data, can be one of"json"
,"columns"
,"csv"
or"arrow"
. This overrides language-specific type dispatch behavior, which allows stringified and byte array alternative inputs.
Python Examples
Load a CSV from a str
:
table = client.table("x,y\n1,2\n3,4")
Opens a Table that is hosted on the perspective_server::Server
that is connected to this Client.
The name
property of TableInitOptions
is used to identify each
Table. Table name
s can be looked up for each Client
via Client.get_hosted_table_names
.
Python Examples
table = client.open_table("table_one");
Retrieves the names of all tables that this client has access to.
name
is a string identifier unique to the Table (per Client),
which can be used in conjunction with Client.open_table
to get
a Table instance without the use of Client.table
constructor directly (e.g., one created by another Client).
Python Examples
tables = client.get_hosted_table_names();
Register a callback which is invoked whenever Client.table
(on this
Client) or Table.delete
(on a Table belinging to this
Client) are called.
Remove a callback previously registered via
Client.on_hosted_tables_update
.
Provides the SystemInfo
struct, implementation-specific metadata
about the perspective_server.Server
runtime such as Memory and
CPU usage.
Table is Perspective's columnar data frame, analogous to a Pandas/Polars
DataFrame
or Apache Arrow, supporting append & in-place updates, removal
by index, and update notifications.
A Table contains columns, each of which have a unique name, are strongly and consistently typed, and contains rows of data conforming to the column's type. Each column in a Table must have the same number of rows, though not every row must contain data; null-values are used to indicate missing values in the dataset. The schema of a Table is _immutable after creation_, which means the column names and data types cannot be changed after the Table has been created. Columns cannot be added or deleted after creation either, but a View can be used to select an arbitrary set of columns from the Table.
Returns the name of the index column for the table.
Python Examples
table = perspective.table("x,y\n1,2\n3,4", index="x");
index = client.get_index()
Returns the user-specified name for this table, or the auto-generated name if a name was not specified when the table was created.
Removes all the rows in the Table, but preserves everything else including the schema, index, and any callbacks or registered View instances.
Calling Table.clear
, like Table.update
and Table.remove
,
will trigger an update event to any registered listeners via
View.on_update
.
Returns the column names of this Table in "natural" order (the ordering implied by the input format).
# Python Examples
columns = table.columns()
Delete this Table and cleans up associated resources.
Tables do not stop consuming resources or processing updates when they are garbage collected in their host language - you must call this method to reclaim these.
Arguments
options
An options dictionary.lazy
Whether to delete this Table _lazily_. When false (the default), the delete will occur immediately, assuming it has no View instances registered to it (which must be deleted first, otherwise this method will throw an error). When true, the Table will only be marked for deltion once its View dependency count reaches 0.
Python Examples
table = client.table("x,y\n1,2\n3,4")
# ...
table.delete(lazy=True)
Create a unique channel ID on this Table, which allows
View::on_update
callback calls to be associated with the
Table::update
which caused them.
Register a callback which is called exactly once, when this Table is
deleted with the Table.delete
method.
Table.on_delete
resolves when the subscription message is sent, not
when the _delete_ event occurs.
Removes a listener with a given ID, as returned by a previous call to
Table.on_delete
.
Returns a table's Schema
, a mapping of column names to column types.
The mapping of a Table's column names to data types is referred to
as a Schema
. Each column has a unique name and a data type, one
of:
"boolean"
- A boolean type"date"
- A timesonze-agnostic date type (month/day/year)"datetime"
- A millisecond-precision datetime type in the UTC timezone"float"
- A 64 bit float"integer"
- A signed 32 bit integer (the integer type supported by JavaScript)"string"
- AString
data type (encoded internally as a _dictionary_)
Note that all Table columns are _nullable_, regardless of the data type.
Create a new View from this table with a specified
ViewConfigUpdate
.
See View struct.
Examples
view view = table.view(
columns=["Sales"],
aggregates={"Sales": "sum"},
group_by=["Region", "State"],
)
Removes all the rows in the Table, but preserves everything else including the schema, index, and any callbacks or registered View instances.
Calling Table.clear
, like Table.update
and Table.remove
,
will trigger an update event to any registered listeners via
View.on_update
.
Updates the rows of this table and any derived View instances.
Calling Table.update
will trigger the View.on_update
callbacks
register to derived View, and the call itself will not resolve until
_all_ derived View's are notified.
When updating a Table with an index
, Table.update
supports
partial updates, by omitting columns from the update data.
Arguments
input
- The input data for this Table. The schema of a Table is immutable after creation, so this method cannot be called with a schema.options
- Options for this update step - seeperspective_client.UpdateOptions
. ```
The View struct is Perspective's query and serialization interface. It
represents a query on the Table
's dataset and is always created from an
existing Table
instance via the Table.view
method.
Views are immutable with respect to the arguments provided to the
Table.view
method; to change these parameters, you must create a new
View on the same Table. However, each View is _live_ with
respect to the Table's data, and will (within a conflation window)
update with the latest state as its parent Table updates, including
incrementally recalculating all aggregates, pivots, filters, etc. View
query parameters are composable, in that each parameter works independently
_and_ in conjunction with each other, and there is no limit to the number of
pivots, filters, etc. which can be applied.
To construct a View, call the Table.view
factory method. A
Table can have as many Views associated with it as you need -
Perspective conserves memory by relying on a single Table to power
multiple Views concurrently.
Returns an array of strings containing the column paths of the View without any of the source columns.
A column path shows the columns that a given cell belongs to after pivots are applied.
Renders this View as a column-oriented JSON string. Useful if you want to save additional round trip serialize/deserialize cycles.
Returns this View's _dimensions_, row and column count, as well as
those of the crate.Table
from which it was derived.
num_table_rows
- The number of rows in the underlyingcrate.Table
.num_table_columns
- The number of columns in the underlyingcrate.Table
(including theindex
column if thiscrate.Table
was constructed with one).num_view_rows
- The number of rows in this View. If this View has agroup_by
clause,num_view_rows
will also include aggregated rows.num_view_columns
- The number of columns in this View. If this View has asplit_by
clause,num_view_columns
will include all _column paths_, e.g. the number ofcolumns
clause times the number ofsplit_by
groups.
The expression schema of this View, which contains only the
expressions created on this View. See View.schema
for
details.
A copy of the ViewConfig
object passed to the Table.view
method
which created this View.
Calculates the [min, max] of the leaf nodes of a column column_name
.
Returns
A tuple of [min, max], whose types are column and aggregate dependent.
The number of aggregated rows in this View. This is affected by the "group_by" configuration parameter supplied to this view's contructor.
Returns
The number of aggregated rows.
The schema of this View.
The View schema differs from the schema
returned by
Table.schema
; it may have different column names due to
expressions
or columns
configs, or it maye have _different
column types_ due to the application og group_by
and aggregates
config. You can think of Table.schema
as the _input_ schema and
View.schema
as the _output_ schema of a Perspective pipeline.
Unregister a previously registered View.on_delete
callback.
Register a callback with this View. Whenever the view's underlying
table emits an update, this callback will be invoked with an object
containing port_id
, indicating which port the update fired on, and
optionally delta
, which is the new data that was updated for each
cell or each row.
Arguments
on_update
- A callback function invoked on update, which receives an object with two keys:port_id
, indicating which port the update was triggered on, anddelta
, whose value is dependent on the mode parameter.options
- If this is provided asOnUpdateOptions { mode: Some(OnUpdateMode::Row) }
, thendelta
is an Arrow of the updated rows. Otherwisedelta
will beOption.None
.
Unregister a previously registered update callback with this View.
Arguments
id
- A callbackid
as returned by a recipricol call toView.on_update
.
Examples
let callback = |_| async { print!("Updated!") };
let cid = view.on_update(callback, OnUpdateOptions::default()).await?;
view.remove_update(cid).await?;
Returns the number of threads the internal threadpool will use.
Set the number of threads the internal threadpool will use. Can also be set
with NUM_OMP_THREADS
environment variable.
Provides the SystemInfo
struct, implementation-specific metadata
about the perspective_server.Server
runtime such as Memory and
CPU usage.