Chapter 4 Tools

Tools are the part of the MatrixDS platform where all the magic happens. Every tool mounts the same shared file system described in the previous section. However, every tool is independent and has its own compute resources (CPU and Memory).

4.1 Creating and Opening

Making new tools is simple. From the project view, select the tool tab and click the green +, a modal will appear showing all the available tools by category. Select an option from the list, and a short description is displayed. Next click Create and the system will build a new tool ready for your use. One important point is that some tools create files in the file system for sharing libraries and other dependencies within project users.

Tools should only take a few minutes to build. Once built clicking Open launches the tool in a new browser tab ready for use.

A couple of things to keep in mind. Files will persist, every user gets 3 permanent tools, all other tools will only be ‘active’ by default for 6 hours for community users. After six hours the tool will stop (AKA shutdown) and must be restarted. Any unsaved work will be lost. If a tool has shell access built in (like Jupyter), you can access it from there.

4.2 Stopping, Starting and Deleting

You have the control of stopping tools (shutting them down) to save resources while a tool is not in use by clicking the Stop button in the tool detail. Once a tool is stopped, either manually or timing out, you can start it back up by using the Start button. Tools can be permanently deleted from a project if they are no longer needed. Click the Delete button on the tool detail page.

4.3 Tool Detail

Each tool is a separate Docker container, and there are some global variables which you can manipulate. However, for the MatrixDS provided tools, it is advisable to use the default options. The following is a brief description of the options.

  • Application: The name of an application
  • Port: The port where the application UI will be presented
  • Docker Image: The location of the docker image used for the tool
  • Mount Path: The location on the docker container to mount the project file structure
  • Health Check: An optional health check for the container

There are some other system generated values that are useful when projects require tools to interact.

One of the most significant options is making a tool Public. Doing this will make the tool URL available to anyone on the internet. If an analytics project requires public output (like a web app, blog or API) check the public option.

4.4 Launching A Docker Hub Tool

The tool service can build and present any existing docker image. The most common repository of docker images is Docker Hub. You can use any container image path here for the custom tool option.

Using any docker container creates quite a bit of flexibility for adding cutting-edge tools to your analytics project. Each tool has a hostname in the detail panel for connectivity purposes. A word of caution is to pay close attention to where you put the mount path for each tool as that is where you will be able to access project files.

4.5 Store Tools

Several different databases are provided as store tools. They are there so that we can organize the data inside our projects in a simple way. Whether it is data that needs to be prepped for analysis, ready to analyze or the output of analysis, you have your database of choice to store it all in there.

4.5.1 Connecting to your new database

Only tools within your project will have access to the database. To connect, you’ll need the hostname, the username, and the password.

To find the hostname click on your new database from the database tab. The hostname will be shown in the details. If you didn’t change the default settings when you created the database the login details are as follows:

4.5.1.1 MongoDB

No username or password is set, so do not add credentials to your Mongo connection string.

4.5.1.2 MySQL

Just the root account password is set up by default:

  • Username: root
  • Password: matrix

4.5.1.3 PostgreSQL

  • Username: matrix
  • Password: matrix

4.5.2 Advanced Use

You can enter your own command line options and environment variables when creating databases. More information on the database containers can be found from these links:

4.6 Present Tools

Present tools are those that help the creation of data web applications or interactive dashboards. These tools are used to host a finished product for others to consume and interact.

4.6.1 Hosting your Visualization

All present tools when created, add a folder to project the file directory. Anything inside that folder is continuously deployed by the tool.

4.6.1.1 NGINX

The NGINX tool is designed to deploy a static web server. It creates a folder called html; the tool defaults to searching for a file called index.html. All files related to the web application must be inside the hosted folder.

4.6.1.2 Bokeh

Bokeh is a python based web app and visualization environment. The tool creates a folder called bokeh and continuously serves everything inside that folder.

4.6.1.3 Shiny

Shiny is a lightweight web application hosting environment that uses R as it’s base language. The Shiny tool creates a folder called shiny-server. Shiny apps must be in their own directory inside the shiny-server folder and must be named app.R

4.6.2 Making Present Tools Public

All present tools have a Private/Public Toggle. When made public, the URL for the tools is made available to anyone online even if they are not MatrixDS users. Public tools are a simple way to share your visualizations with consumers who just want to interact with the finished product. MatrixDS makes hosting your dashboards and apps very easy.