Lotus Farcaster

The visualization and analytics tool for Lotus Filecoin node.

Version 3 is Available ! For V2 users upgrading to V3:
- Don't forget to update Grafana to version 8.1.2
- Reimport the new Grafana Dashboard
- Install the new lotus-exporter-farcaster (instructions below)



  • Advanced analytics: Like average sealing time, historical
  • Realtime monitoring: Visualize all the different aspect of your miner at a glance
  • Designed to replace your terminal monitoring
  • Farcaster is developped in cooperation with Protocol Labs.
  • Leverages Prometheus, Grafana and Python.

Dashboard Features

Feature Implementation Comments
Sectors
Mpool
Storage Info
Workers Info
Sealing
Power
Wallets
Chain
Deadlines
Deals Partial
Online data transfers Realtime view of on-going transfers
Fil+ Visualize granted datacap
Address lookup View friendly names instead of boring addresses

Architecture

 --------------------------            ------------            ---------------------            ------------
| Lotus Miner              |          |            |          | Grafana             |          | Web Client |   
| lotus-exporter-farcaster | <------- | Prometheus | <------- | farcaster-dashboard | <------- |            |
 --------------------------            ------------            ---------------------            ------------  
   |        
   |    -------------------------   
   |-> | Lotus Daemon            | 
   |     -------------------------  
   | 
   |    -------------------------  
   |-> | Lotus Markets           | 
        -------------------------
                                      
Lotus farcaster comes with 2 Components :
  • A Grafana Dashboard
  • Lotus-exporter-farcaster : A standalone and configuration-less python script executed every minyte by crontab

Security

The design and choice of these components, was made to keep a high level of security :
  • metrics are pulled from the miner to avoid compromising network or management servers
  • lotus-exporter-farcaster doesn’t bind any ports to avoid exposing the miner to external threat
  • lotus-exporter-farcaster run under unprivileged user (ideally lotus user) to avoid system being compromised
  • The global solution doesn’t rely on any cloud or remote component. To avoid data leak.
lotus-exporter-farcaster is running locally on the miner without any specific privilege nor access to other servers.

It will follow prometheus guidelines :
  • configuration-less
  • ressources inexpensive
  • standard prometheus metrics naming
  • compatible with 3rd parties dashboards and exporters

Benefits

  • Easy to deploy
  • Configuration less
  • Small footprint
  • Collect lotus node and miner data
  • Only rely on API
  • Data are pulled from the Prometheus (increase security)
  • Deploy on the miner node only
  • Run under Unprivileged user

Tutorials

Deep dive in Lotus farcaster features


Step by step installation tutorial


Installation

Requirements

A Grafana / Prometheus working environement (local or remote)

Locally installed on the miner node only

  • prometheus-node-exporter
  • python3-toml
  • python3-aiohttp
  • py-multibase
  • lotus-exporter-farcaster

Tested environment

  • Grafana : 8.1.0
  • Prometheus : 2.20.1
  • Ubuntu : 20.04.1 LTS
  • Python : 3.8

Install lotus-exporter-farcaster - Standalone

                                  git clone https://github.com/s0nik42/lotus-farcaster.git
                                  cd lotus-farcaster/lotus-exporter-farcaster
                                  chmod +x install.sh
                                  ./install.sh LOTUS_USER_USERNAME
                                

Install lotus-exporter-farcaster - Docker (alternative)

Farcaster can also run as a Docker container.

The container that corresponds to this repository will run the `docker_run_script.sh` script which just loops over calling lotus-farcaster code at a specific frequency (default: every minute)

This can be overriden by editing the Dockerfile. In case execution exceed the set requency, the execution restart after 10 seconds.

The output of the lotus-farcaster is written to `/data/farcaster.prom` inside the container which should be a bind mount in prometheus node exporter path.

Optional : This could be use in conjunction with a dockerised version of prometheus node_exporter (instructions to get the node_exporter container going


Building the container (as root)
                                  apt install docker.io
                                  docker build -t lotus-farcaster:latest -f dockerfiles/Dockerfile .
                                

Running the container (as root)

lotus-exporter-farcaster use 2 configuration files:

  • config.toml contains connection strings to miner,daemon and markets API. This file is mandatory.
  • addresses.toml contains address lookup that replace filecoin addresses by readable names. This file is optional.
At this point you have to create the config.toml file by copying config.toml.example and edit it :
                                  cd lotus-exporter-farcaster
                                  cp config.toml.example config.toml
                                  nano config.toml

                                  # Optional
                                  cp addresses.toml.example addresses.toml
                                  nano addresses.toml
                                

                                  export PROMETHEUS_NODE_EXPORTER_PATH="/var/lib/prometheus/node-exporter/"

                                  docker run --name lotus-farcaster -d \
                                    --mount type=bind,source=$PROMETHEUS_NODE_EXPORTER_PATH,target=/data \
                                    --network=host \
                                    lotus-farcaster
                                

Docker Debug (as root)
                                  docker ps [-a]
                                  docker logs lotus-farcaster
                                  docker exec -it lotus-farcaster bash
                                

Uninstall docker (as root)
                                  docker stop lotus-farcaster
                                  docker rm lotus-farcaster
                                  docker image rm lotus-farcaster
                                

Install the Grafana Dashboard

Import in Grafana the relevent dashboard file from ./lotus-farcaster/grafana-dashboard

Contributing

If you're interested in contributing to the Grafana project:

  • Translate : help us by translating our dashboard to other languages.
  • Panels : submit your customized panels
  • Donate : Filcoin Address : f3v3lj5jrsvv3nwmsvvj57yyty6ndb27oyi4yaqhwzst3emdv25hefna6vxhtpjb5pytwahdod67syxjyzba3q

Sponsors

Protcol Labs

Contact

  • * Filecoin Slack : @Julien_NOEL_-_Twin_Quasar_(s0nik42)