Lotus Farcaster

The visualization and analytics tool for Lotus Filecoin node.

Version 3 is Available ! For V2 users upgrading to V3:
- Don't forget to update Grafana to version 8.1.2
- Reimport the new Grafana Dashboard
- Install the new lotus-exporter-farcaster (instructions below)

Advanced analytics: Like average sealing time, historical
Realtime monitoring: Visualize all the different aspect of your miner at a glance
Designed to replace your terminal monitoring
Farcaster is developped in cooperation with Protocol Labs.
Leverages Prometheus, Grafana and Python.

Dashboard Features

Feature	Implementation	Comments
Sectors
Mpool
Storage Info
Workers Info
Sealing
Power
Wallets
Chain
Deadlines
Deals	Partial
Online data transfers		Realtime view of on-going transfers
Fil+		Visualize granted datacap
Address lookup		View friendly names instead of boring addresses

Architecture

 --------------------------            ------------            ---------------------            ------------
| Lotus Miner              |          |            |          | Grafana             |          | Web Client |   
| lotus-exporter-farcaster | <------- | Prometheus | <------- | farcaster-dashboard | <------- |            |
 --------------------------            ------------            ---------------------            ------------  
   |        
   |    -------------------------   
   |-> | Lotus Daemon            | 
   |     -------------------------  
   | 
   |    -------------------------  
   |-> | Lotus Markets           | 
        -------------------------

Lotus farcaster comes with 2 Components :

A Grafana Dashboard
Lotus-exporter-farcaster : A standalone and configuration-less python script executed every minyte by crontab

Security

The design and choice of these components, was made to keep a high level of security :

metrics are pulled from the miner to avoid compromising network or management servers
lotus-exporter-farcaster doesn’t bind any ports to avoid exposing the miner to external threat
lotus-exporter-farcaster run under unprivileged user (ideally lotus user) to avoid system being compromised
The global solution doesn’t rely on any cloud or remote component. To avoid data leak.

lotus-exporter-farcaster is running locally on the miner without any specific privilege nor access to other servers.

It will follow prometheus guidelines :

configuration-less
ressources inexpensive
standard prometheus metrics naming
compatible with 3rd parties dashboards and exporters

Benefits

Easy to deploy
Configuration less
Small footprint
Collect lotus node and miner data
Only rely on API
Data are pulled from the Prometheus (increase security)
Deploy on the miner node only
Run under Unprivileged user

Tutorials

Deep dive in Lotus farcaster features

Step by step installation tutorial

Installation

Requirements

A Grafana / Prometheus working environement (local or remote)

Grafana installation guide (Version 8.1.2)
Prometheus is available as package in Ubuntu 20.0 (Version 2.20.1)

Locally installed on the miner node only

prometheus-node-exporter
python3-toml
python3-aiohttp
py-multibase
lotus-exporter-farcaster

Tested environment

Grafana : 8.1.0
Prometheus : 2.20.1
Ubuntu : 20.04.1 LTS
Python : 3.8

Install lotus-exporter-farcaster - Standalone

                                  git clone https://github.com/s0nik42/lotus-farcaster.git
                                  cd lotus-farcaster/lotus-exporter-farcaster
                                  chmod +x install.sh
                                  ./install.sh LOTUS_USER_USERNAME

Install lotus-exporter-farcaster - Docker (alternative)

Farcaster can also run as a Docker container.

The container that corresponds to this repository will run the `docker_run_script.sh` script which just loops over calling lotus-farcaster code at a specific frequency (default: every minute)

This can be overriden by editing the Dockerfile. In case execution exceed the set requency, the execution restart after 10 seconds.

The output of the lotus-farcaster is written to `/data/farcaster.prom` inside the container which should be a bind mount in prometheus node exporter path.

Optional : This could be use in conjunction with a dockerised version of prometheus node_exporter (instructions to get the node_exporter container going

Building the container (as root)

                                  apt install docker.io
                                  docker build -t lotus-farcaster:latest -f dockerfiles/Dockerfile .

Running the container (as root)

lotus-exporter-farcaster use 2 configuration files:

config.toml contains connection strings to miner,daemon and markets API. This file is mandatory.
addresses.toml contains address lookup that replace filecoin addresses by readable names. This file is optional.

At this point you have to create the config.toml file by copying config.toml.example and edit it :

                                  cd lotus-exporter-farcaster
                                  cp config.toml.example config.toml
                                  nano config.toml

                                  # Optional
                                  cp addresses.toml.example addresses.toml
                                  nano addresses.toml

                                  export PROMETHEUS_NODE_EXPORTER_PATH="/var/lib/prometheus/node-exporter/"

                                  docker run --name lotus-farcaster -d \
                                    --mount type=bind,source=$PROMETHEUS_NODE_EXPORTER_PATH,target=/data \
                                    --network=host \
                                    lotus-farcaster

Docker Debug (as root)

                                  docker ps [-a]
                                  docker logs lotus-farcaster
                                  docker exec -it lotus-farcaster bash

Uninstall docker (as root)

                                  docker stop lotus-farcaster
                                  docker rm lotus-farcaster
                                  docker image rm lotus-farcaster

Install the Grafana Dashboard

Import in Grafana the relevent dashboard file from ./lotus-farcaster/grafana-dashboard

Contributing

If you're interested in contributing to the Grafana project:

Translate : help us by translating our dashboard to other languages.
Panels : submit your customized panels
Donate : Filcoin Address : f3v3lj5jrsvv3nwmsvvj57yyty6ndb27oyi4yaqhwzst3emdv25hefna6vxhtpjb5pytwahdod67syxjyzba3q

Contact

* Filecoin Slack : @Julien_NOEL_-_Twin_Quasar_(s0nik42)