perspective.handlers.tornado

Perspective ships with a pre-built Tornado handler that makes integration with tornado.websockets extremely easy. This allows you to run an instance of Perspective on a server using Python, open a websocket to a Table, and access the Table in JavaScript and through <perspective-viewer>. All instructions sent to the Table are processed in Python, which executes the commands, and returns its output through the websocket back to Javascript.

Python setup

To use the handler, we need to first have a Server, a Client and an instance of a Table:

SERVER = Server()
CLIENT = SERVER.new_local_client()

Once the server has been created, create a Table instance with a name. The name that you host the table under is important — it acts as a unique accessor on the JavaScript side, which will look for a Table hosted at the websocket with the name you specify.

TABLE = client.table(data, name="data_source_one")

After the server and table setup is complete, create a websocket endpoint and provide it a reference to PerspectiveTornadoHandler. You must provide the configuration object in the route tuple, and it must contain "perspective_server", which is a reference to the Server you just created.

from perspective.handlers.tornado import PerspectiveTornadoHandler

app = tornado.web.Application([

    # ... other handlers ...

    # Create a websocket endpoint that the client JavaScript can access
    (r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER, "check_origin": True})
])

Optionally, the configuration object can also include check_origin, a boolean that determines whether the websocket accepts requests from origins other than where the server is hosted. See Tornado docs for more details.

JavaScript setup

Once the server is up and running, you can access the Table you just hosted using perspective.websocket and open_table(). First, create a client that expects a Perspective server to accept connections at the specified URL:

const websocket = await perspective.websocket("ws://localhost:8888/websocket");

Next open the Table we created on the server by name:

const table = await websocket.open_table("data_source_one");

table is a proxy for the Table we created on the server. All operations that are possible through the JavaScript API are possible on the Python API as well, thus calling view(), schema(), update() etc. on const table will pass those operations to the Python Table, execute the commands, and return the result back to JavaScript. Similarly, providing this table to a <perspective-viewer> instance will allow virtual rendering:

await viewer.load(table);

perspective.websocket expects a Websocket URL where it will send instructions. When open_table is called, the name to a hosted Table is passed through, and a request is sent through the socket to fetch the Table. No actual Table instance is passed inbetween the runtimes; all instructions are proxied through websockets.

This provides for great flexibility — while Perspective.js is full of features, browser WebAssembly runtimes currently have some performance restrictions on memory and CPU feature utilization, and the architecture in general suffers when the dataset itself is too large to download to the client in full.

The Python runtime does not suffer from memory limitations, utilizes Apache Arrow internal threadpools for threading and parallel processing, and generates architecture optimized code, which currently makes it more suitable as a server-side runtime than node.js.

  1#  ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
  2#  ┃ ██████ ██████ ██████       █      █      █      █      █ █▄  ▀███ █       ┃
  3#  ┃ ▄▄▄▄▄█ █▄▄▄▄▄ ▄▄▄▄▄█  ▀▀▀▀▀█▀▀▀▀▀ █ ▀▀▀▀▀█ ████████▌▐███ ███▄  ▀█ █ ▀▀▀▀▀ ┃
  4#  ┃ █▀▀▀▀▀ █▀▀▀▀▀ █▀██▀▀ ▄▄▄▄▄ █ ▄▄▄▄▄█ ▄▄▄▄▄█ ████████▌▐███ █████▄   █ ▄▄▄▄▄ ┃
  5#  ┃ █      ██████ █  ▀█▄       █ ██████      █      ███▌▐███ ███████▄ █       ┃
  6#  ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫
  7#  ┃ Copyright (c) 2017, the Perspective Authors.                              ┃
  8#  ┃ ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ ┃
  9#  ┃ This file is part of the Perspective library, distributed under the terms ┃
 10#  ┃ of the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). ┃
 11#  ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
 12
 13from tornado.websocket import WebSocketHandler, WebSocketClosedError
 14from tornado.ioloop import IOLoop
 15import perspective
 16
 17__doc__ = """
 18Perspective ships with a pre-built Tornado handler that makes integration with
 19`tornado.websockets` extremely easy. This allows you to run an instance of
 20`Perspective` on a server using Python, open a websocket to a `Table`, and
 21access the `Table` in JavaScript and through `<perspective-viewer>`. All
 22instructions sent to the `Table` are processed in Python, which executes the
 23commands, and returns its output through the websocket back to Javascript.
 24
 25### Python setup
 26
 27To use the handler, we need to first have a `Server`, a `Client` and an instance
 28of a `Table`:
 29
 30```python
 31SERVER = Server()
 32CLIENT = SERVER.new_local_client()
 33```
 34
 35Once the server has been created, create a `Table` instance with a name. The
 36name that you host the table under is important — it acts as a unique accessor
 37on the JavaScript side, which will look for a Table hosted at the websocket with
 38the name you specify.
 39
 40```python
 41TABLE = client.table(data, name="data_source_one")
 42```
 43
 44After the server and table setup is complete, create a websocket endpoint and
 45provide it a reference to `PerspectiveTornadoHandler`. You must provide the
 46configuration object in the route tuple, and it must contain
 47`"perspective_server"`, which is a reference to the `Server` you just created.
 48
 49```python
 50from perspective.handlers.tornado import PerspectiveTornadoHandler
 51
 52app = tornado.web.Application([
 53
 54    # ... other handlers ...
 55
 56    # Create a websocket endpoint that the client JavaScript can access
 57    (r"/websocket", PerspectiveTornadoHandler, {"perspective_server": SERVER, "check_origin": True})
 58])
 59```
 60
 61Optionally, the configuration object can also include `check_origin`, a boolean
 62that determines whether the websocket accepts requests from origins other than
 63where the server is hosted. See
 64[Tornado docs](https://www.tornadoweb.org/en/stable/websocket.html#tornado.websocket.WebSocketHandler.check_origin)
 65for more details.
 66
 67### JavaScript setup
 68
 69Once the server is up and running, you can access the Table you just hosted
 70using `perspective.websocket` and `open_table()`. First, create a client that
 71expects a Perspective server to accept connections at the specified URL:
 72
 73```javascript
 74const websocket = await perspective.websocket("ws://localhost:8888/websocket");
 75```
 76
 77Next open the `Table` we created on the server by name:
 78
 79```javascript
 80const table = await websocket.open_table("data_source_one");
 81```
 82
 83`table` is a proxy for the `Table` we created on the server. All operations that
 84are possible through the JavaScript API are possible on the Python API as well,
 85thus calling `view()`, `schema()`, `update()` etc. on `const table` will pass
 86those operations to the Python `Table`, execute the commands, and return the
 87result back to JavaScript. Similarly, providing this `table` to a
 88`<perspective-viewer>` instance will allow virtual rendering:
 89
 90```javascript
 91await viewer.load(table);
 92```
 93
 94`perspective.websocket` expects a Websocket URL where it will send instructions.
 95When `open_table` is called, the name to a hosted Table is passed through, and a
 96request is sent through the socket to fetch the Table. No actual `Table`
 97instance is passed inbetween the runtimes; all instructions are proxied through
 98websockets.
 99
100This provides for great flexibility — while `Perspective.js` is full of
101features, browser WebAssembly runtimes currently have some performance
102restrictions on memory and CPU feature utilization, and the architecture in
103general suffers when the dataset itself is too large to download to the client
104in full.
105
106The Python runtime does not suffer from memory limitations, utilizes Apache
107Arrow internal threadpools for threading and parallel processing, and generates
108architecture optimized code, which currently makes it more suitable as a
109server-side runtime than `node.js`.
110"""
111
112
113class PerspectiveTornadoHandler(WebSocketHandler):
114    """`PerspectiveTornadoHandler` is a `perspective.Server` API as a `tornado`
115    websocket handler.
116
117    Use it inside `tornado` routing to create a `perspective.Server` that can
118    connect to a JavaScript (Wasm) `Client`, providing a virtual interface to
119    the `Server`'s resources for e.g. `<perspective-viewer>`.
120
121    You may need to increase the `websocket_max_message_size` kwarg
122    to the `tornado.web.Application` constructor, as well as provide the
123    `max_buffer_size` optional arg, for large datasets.
124
125    # Arguments
126
127    -   `loop`: An optional `IOLoop` instance to use for scheduling IO calls,
128        defaults to `IOLoop.current()`.
129    -   `executor`: An optional executor for scheduling `perspective.Server`
130        message processing calls from websocket `Client`s.
131
132    # Examples
133
134    >>> server = psp.Server()
135    >>> client = server.new_local_client()
136    >>> client.table(pd.read_csv("superstore.csv"), name="data_source_one")
137    >>> app = tornado.web.Application([
138    ...     (r"/", MainHandler),
139    ...     (r"/websocket", PerspectiveTornadoHandler, {
140    ...         "perspective_server": server,
141    ...     })
142    ... ])
143    """
144
145    def check_origin(self, origin):
146        return True
147
148    def initialize(
149        self,
150        perspective_server=perspective.GLOBAL_SERVER,
151        loop=None,
152        executor=None,
153        max_buffer_size=None,
154    ):
155        self.server = perspective_server
156        self.loop = loop or IOLoop.current()
157        self.executor = executor
158        if max_buffer_size is not None:
159            self.request.connection.stream.max_buffer_size = max_buffer_size
160
161    def open(self):
162        def write(msg):
163            try:
164                self.write_message(msg, binary=True)
165            except WebSocketClosedError:
166                self.close()
167
168        def send_response(msg):
169            self.loop.add_callback(write, msg)
170
171        self.session = self.server.new_session(send_response)
172
173    def on_close(self) -> None:
174        self.session.close()
175        del self.session
176
177    def on_message(self, msg: bytes):
178        if not isinstance(msg, bytes):
179            return
180
181        if self.executor is None:
182            self.session.handle_request(msg)
183        else:
184            self.executor.submit(self.session.handle_request, msg)
class PerspectiveTornadoHandler(tornado.websocket.WebSocketHandler):
114class PerspectiveTornadoHandler(WebSocketHandler):
115    """`PerspectiveTornadoHandler` is a `perspective.Server` API as a `tornado`
116    websocket handler.
117
118    Use it inside `tornado` routing to create a `perspective.Server` that can
119    connect to a JavaScript (Wasm) `Client`, providing a virtual interface to
120    the `Server`'s resources for e.g. `<perspective-viewer>`.
121
122    You may need to increase the `websocket_max_message_size` kwarg
123    to the `tornado.web.Application` constructor, as well as provide the
124    `max_buffer_size` optional arg, for large datasets.
125
126    # Arguments
127
128    -   `loop`: An optional `IOLoop` instance to use for scheduling IO calls,
129        defaults to `IOLoop.current()`.
130    -   `executor`: An optional executor for scheduling `perspective.Server`
131        message processing calls from websocket `Client`s.
132
133    # Examples
134
135    >>> server = psp.Server()
136    >>> client = server.new_local_client()
137    >>> client.table(pd.read_csv("superstore.csv"), name="data_source_one")
138    >>> app = tornado.web.Application([
139    ...     (r"/", MainHandler),
140    ...     (r"/websocket", PerspectiveTornadoHandler, {
141    ...         "perspective_server": server,
142    ...     })
143    ... ])
144    """
145
146    def check_origin(self, origin):
147        return True
148
149    def initialize(
150        self,
151        perspective_server=perspective.GLOBAL_SERVER,
152        loop=None,
153        executor=None,
154        max_buffer_size=None,
155    ):
156        self.server = perspective_server
157        self.loop = loop or IOLoop.current()
158        self.executor = executor
159        if max_buffer_size is not None:
160            self.request.connection.stream.max_buffer_size = max_buffer_size
161
162    def open(self):
163        def write(msg):
164            try:
165                self.write_message(msg, binary=True)
166            except WebSocketClosedError:
167                self.close()
168
169        def send_response(msg):
170            self.loop.add_callback(write, msg)
171
172        self.session = self.server.new_session(send_response)
173
174    def on_close(self) -> None:
175        self.session.close()
176        del self.session
177
178    def on_message(self, msg: bytes):
179        if not isinstance(msg, bytes):
180            return
181
182        if self.executor is None:
183            self.session.handle_request(msg)
184        else:
185            self.executor.submit(self.session.handle_request, msg)

PerspectiveTornadoHandler is a perspective.Server API as a tornado websocket handler.

Use it inside tornado routing to create a perspective.Server that can connect to a JavaScript (Wasm) Client, providing a virtual interface to the Server's resources for e.g. <perspective-viewer>.

You may need to increase the websocket_max_message_size kwarg to the tornado.web.Application constructor, as well as provide the max_buffer_size optional arg, for large datasets.

Arguments

  • loop: An optional IOLoop instance to use for scheduling IO calls, defaults to IOLoop.current().
  • executor: An optional executor for scheduling perspective.Server message processing calls from websocket Clients.

Examples

>>> server = psp.Server()
>>> client = server.new_local_client()
>>> client.table(pd.read_csv("superstore.csv"), name="data_source_one")
>>> app = tornado.web.Application([
...     (r"/", MainHandler),
...     (r"/websocket", PerspectiveTornadoHandler, {
...         "perspective_server": server,
...     })
... ])
def check_origin(self, origin):
146    def check_origin(self, origin):
147        return True

Override to enable support for allowing alternate origins.

The origin argument is the value of the Origin HTTP header, the url responsible for initiating this request. This method is not called for clients that do not send this header; such requests are always allowed (because all browsers that implement WebSockets support this header, and non-browser clients do not have the same cross-site security concerns).

Should return True to accept the request or False to reject it. By default, rejects all requests with an origin on a host other than this one.

This is a security protection against cross site scripting attacks on browsers, since WebSockets are allowed to bypass the usual same-origin policies and don't use CORS headers.

This is an important security measure; don't disable it without understanding the security implications. In particular, if your authentication is cookie-based, you must either restrict the origins allowed by check_origin() or implement your own XSRF-like protection for websocket connections. See these articles for more.

To accept all cross-origin traffic (which was the default prior to Tornado 4.0), simply override this method to always return True::

def check_origin(self, origin):
    return True

To allow connections from any subdomain of your site, you might do something like::

def check_origin(self, origin):
    parsed_origin = urllib.parse.urlparse(origin)
    return parsed_origin.netloc.endswith(".mydomain.com")

New in version 4.0.

def initialize( self, perspective_server=<perspective.Server object>, loop=None, executor=None, max_buffer_size=None):
149    def initialize(
150        self,
151        perspective_server=perspective.GLOBAL_SERVER,
152        loop=None,
153        executor=None,
154        max_buffer_size=None,
155    ):
156        self.server = perspective_server
157        self.loop = loop or IOLoop.current()
158        self.executor = executor
159        if max_buffer_size is not None:
160            self.request.connection.stream.max_buffer_size = max_buffer_size

Hook for subclass initialization. Called for each request.

A dictionary passed as the third argument of a URLSpec will be supplied as keyword arguments to initialize().

Example::

class ProfileHandler(RequestHandler):
    def initialize(self, database):
        self.database = database

    def get(self, username):
        ...

app = Application([
    (r'/user/(.*)', ProfileHandler, dict(database=database)),
    ])
def open(self):
162    def open(self):
163        def write(msg):
164            try:
165                self.write_message(msg, binary=True)
166            except WebSocketClosedError:
167                self.close()
168
169        def send_response(msg):
170            self.loop.add_callback(write, msg)
171
172        self.session = self.server.new_session(send_response)

Invoked when a new WebSocket is opened.

The arguments to open are extracted from the tornado.web.URLSpec regular expression, just like the arguments to tornado.web.RequestHandler.get.

open may be a coroutine. on_message will not be called until open has returned.

Changed in version 5.1: open may be a coroutine.

def on_close(self) -> None:
174    def on_close(self) -> None:
175        self.session.close()
176        del self.session

Invoked when the WebSocket is closed.

If the connection was closed cleanly and a status code or reason phrase was supplied, these values will be available as the attributes self.close_code and self.close_reason.

Changed in version 4.0: Added close_code and close_reason attributes.

def on_message(self, msg: bytes):
178    def on_message(self, msg: bytes):
179        if not isinstance(msg, bytes):
180            return
181
182        if self.executor is None:
183            self.session.handle_request(msg)
184        else:
185            self.executor.submit(self.session.handle_request, msg)

Handle incoming messages on the WebSocket

This method must be overridden.

Changed in version 4.5: on_message can be a coroutine.