Skip to main content

Developer Guide

Thank you for your interest in contributing to Perspective! This guide will teach you everything you need to know to get started hacking on the Perspective codebase.

If you're coming to this project as principally a JavaScript developer, please be aware that Perspective is quite a bit more complex than a typical NPM package due to the mixed-language nature of the project; we've done quite a bit to make sure the newcomer experience is as straightforward as possible, but some things might not work the way you're used to!

Perspective is organized as a monorepo, and uses lerna to manage dependencies.

This guide provides instructions for both the JavaScript and Python libraries. To switch your development toolchain between the two, use yarn setup. Once the setup script has been run, common commands like yarn build and yarn test automatically call the correct build and test tools.

System Dependencies

Perspective.js and perspective-python require the following system dependencies to be installed:

This list may be non-exhaustive depending on your OS/environment; please open a thread in Discussions if you have any questions

Build

Make sure you have the system dependencies installed. For specifics depending on your OS, check the system-specific instructions below.

To run a build, use

yarn build

If this is the first time you've built Perspective, you'll be asked to generate a .perspectiverc via a short survey. This can be later re-configured via

yarn setup

If everything is successful, you should be able to run any of the examples/ packages, e.g. examples/blocks like so:

yarn start blocks

Perspective.js

To build the JavaScript library, which includes WebAssembly compilation, Emscripten and its prerequisites are required.

Perspective.js specifies its Emscripten version dependency in package.json, and the correct version of Emscripten will be installed with other JS dependencies by running yarn.

Building via local EMSDK

To build using an Emscripten install on your local system and not the Emscripten bundled with Perspective in its package.json, install the Emscripten SDK, then activate and export the latest emsdk environment via emsdk_env.sh:

source emsdk/emsdk_env.sh

We currently use Emscripten version 2.0.6 — deviating from this specific version of Emscripten can introduce various errors that are extremely difficult to debug.

To install this specific version of Emscripten:

./emsdk install 2.0.6

perspective-jupyterlab

To install the Jupyterlab plugin from your local working directory, give jupyter labextension install the path to the perspective-jupyterlab package:

jupyter labextension install ./packages/perspective-jupyterlab

Afterwards, you should see it listed as a "local extension" when you run jupyter labextension list.

Because we do not inline Perspective into the Jupyterlab plugin, your local changes will not show up in Jupyterlab unless you use yarn link according to the directions below:

  1. Ensure that your Jupyterlab is built by running jupyter lab build.
  2. Inside each directory in packages, run yarn link. This will create a symlink to your local build that we will use inside Jupyterlab.
  3. From the Perspective root, run yarn jlab_link. This is a script that will find your Jupyterlab installation and tell Jupyterlab to use these symlinks when it looks for Perspective packages, instead of fetching them from NPM.
  4. When you make a local change, make sure you run yarn build andjupyter lab build so that it fetches the newest changes.
  5. Whenever you run jupyter lab clean, you will need to run yarn jlab_link again to re-register the symlinks.

perspective-python

To build the Python library, first configure your project to build Python via yarn setup, then run:

yarn build

perspective-python supports Python 3.7 and upwards.


System-Specific Instructions

MacOS/OSX

Install system dependencies through Homebrew:

brew install cmake
brew install boost
brew install flatbuffers

On M1 (Apple Silicon) systems, make sure your brew-installed dependencies are in /opt/homebrew (the default location), and that /opt/homebrew/bin is on the PATH.

Windows 10

You need to use bash in order to build Perspective packages. To successfully build on Windows 10, enable Windows Subsystem for Linux (WSL) and install the Linux distribution of your choice.

Create symbolic links to easily access Windows directories and projects modified via Windows. This way, you can modify any of the Perspective files using your favorite editors on Windows and build via Linux.

Follow the Linux specific instructions to install Emscripten and all prerequisite tools.

Ubuntu/Debian

On Ubuntu, CMake will mistakenly resolve the system headers in /usr/include rather than the emscripten supplied versions. You can resolve this by moving boost and flatbuffers dependencies to somewhere other than /usr/include - into Perspective's own src dir (as per here).

apt-get install libboost-all-dev
cp -r /usr/include/boost ./packages/perspective/src/include/

cd ./packages/perspective/src/include/
git clone https://github.com/google/flatbuffers.git
cd flatbuffers
cmake -G "Unix Makefiles"
make
ln -s /usr/local/flatbuffers/flatc /usr/local/bin/flatc
chmod +x /usr/local/flatbuffers/flatc

Test

You can run the test suite simply with the standard NPM command, which will both build the test suite for every package and run them.

yarn test [--debug]

A test name regex can be passed to jest via the same -t flag:

yarn test -t 'button test (A|B)'

JavaScript

The JavaScript test suite is composed of two sections: a Node.js test, which asserts behavior of the @finos/perspective library, and a suite of Puppeteer tests, which assert the behavior of the rest of the UI facing packages.

The Puppeteer/UI tests are a form of characterization tests which use screenshots to compare current and previous behavior of <perspective-viewer> and its plugins. The results of each comparison are stored in each package's test/results/results.json file, and the screenshots themselves are stored in the package's tests/screenshots/ directory, though only the former should be checked into GIT. When a test in these suites fails, a file.failed.png and file.diff.png are also generated, showing the divergent screenshot and a contrast diff respectively, so you can verify that the changed behavior either does or does not reflect your patch. If you're confident that the screenshots reflect your change, you can update the new hashes manually in the test/results/results.json file, or update all hashes with the --write flag:

yarn test --write

Python

The Python test suite is built on Pytest, and it asserts the correct behavior of the Python library.

If you have built the library with the --python2 flag, make sure to run the test suite using the --python2 flag as well. Running a version of perspective-python built against Python 2 in Python 3 (and vice versa) is not supported.

Verbosity in the tests can be enabled with the --verbose flag.

Troubleshooting installation from source

If you are installing from a source distribution (sdist), make sure you have the System Dependencies installed.

Try installing in verbose mode:

pip install -vv perspective-python

The most common culprits are:

  • CMake version too old
  • Boost headers are missing or too old
  • Flatbuffers not installed prior to installing Perspective

Timezones in Python Tests

Python tests are configured to use the UTC time zone. If running tests locally, you might find that datetime-related tests fail to assert the correct values. To correct this, run tests with the TZ=UTC, i.e.

TZ=UTC yarn test --verbose

Benchmark

You can generate benchmarks specific to your machine's OS and CPU architecture with Perspective's benchmark suite, which will generate a report.html file in the build/ directory of every package which supports benchmarks, as well as a results.json file in the bench/results/, which can be checked in to GIT with your changes to preserve them for future comparison.

yarn bench

The benchmarks report and results.json show a histogram of current performance, as well as that of the previous results.json. Running this should probably be standard practice after making a large change which may affect performance, but please create a baseline results.json entry for your test machine on a commit before your changes first, such that the effects of your PR can be properly compared.