LOTUS FARCASTER

The visualization and analytics tool for Lotus Filecoin node.

CHECK OUT ON GITHUB

Navigation

Description Features Architecture Security Benefits Tutorials Installation Contributing

Version 3 is Available ! For V2 users upgrading to V3:

- Don't forget to update Grafana to version 8.1.2
- Reimport the new Grafana Dashboard
- Install the new lotus-exporter-farcaster (instructions below)

Advanced analytics: Like average sealing time, historical
Realtime monitoring: Visualize all the different aspect of your miner at a glance
Designed to replace your terminal monitoring
Farcaster is developped in cooperation with Protocol Labs.
Leverages Prometheus, Grafana and Python

Dashboard Features

Implementation Comments
Sectors
Mpool
Storage Info
Workers Info
Sealing
Power
Wallets
Chain
Deadlines
Deals Partial
Online data transfers Visualize granted datacap
Fil+ Visualize granted datacap
Address lookup View friendly names instead of boring addresses

Architecture

                                        --------------------------            ------------            ---------------------            ------------
                                        | Lotus Miner              |          |            |          | Grafana             |          | Web Client |   
                                        | lotus-exporter-farcaster | <------- | Prometheus | <------- | farcaster-dashboard | <------- |            |
                                        --------------------------            ------------            ---------------------            ------------  
                                        |        
                                        |    -------------------------   
                                        |-> | Lotus Daemon            | 
                                        |     -------------------------  
                                        | 
                                        |    -------------------------  
                                        |-> | Lotus Markets           | 
                                                -------------------------
Lotus farcaster comes with 2 Components :
A Grafana Dashboard
Lotus-exporter-farcaster : A standalone and configuration-less python script executed every minyte by crontab

Security

The design and choice of these components, was made to keep a high level of security :
metrics are pulled from the miner to avoid compromising network or management servers
lotus-exporter-farcaster doesn’t bind any ports to avoid exposing the miner to external threat
lotus-exporter-farcaster run under unprivileged user (ideally lotus user) to avoid system being compromised
The global solution doesn’t rely on any cloud or remote component. To avoid data leak.

Lotus-exporter-farcaster is running locally on the miner without any specific privilege nor access to other servers.
It will follow prometheus guidelines :

configuration-less
ressources inexpensive
standard prometheus metrics naming
compatible with 3rd parties dashboards and exporters

Benefits

Easy to deploy
Configuration less
Small footprint
Collect lotus node and miner data
Only rely on API
Data are pulled from the Prometheus (increase security)
Deploy on the miner node only
Run under Unprivileged user

Tutorials

Deep dive in Lotus farcaster features


Step by step installation tutorial


Installation & requirements

A Grafana / Prometheus working environement (local or remote)

Locally installed on the miner node only

  • prometheus-node-exporter
  • python3-toml
  • python3-aiohttp
  • py-multibase
  • lotus-exporter-farcaster

Tested environment

  • Grafana : 8.1.0
  • Prometheus : 2.20.1
  • Ubuntu : 20.04.1 LTS
  • Python : 3.8

Install lotus-exporter-farcaster - Standalone 

                                            git clone https://github.com/s0nik42/lotus-farcaster.git
                                            cd lotus-farcaster/lotus-exporter-farcaster
                                            chmod +x install.sh
                                            ./install.sh LOTUS_USER_USERNAME

Install lotus-exporter-farcaster - Docker (alternative)

Farcaster can also run as a Docker container.

The container that corresponds to this repository will run the `docker_run_script.sh` script which just loops over calling lotus-farcaster code at a specific frequency (default: every minute)

This can be overriden by editing the Dockerfile. In case execution exceed the set requency, the execution restart after 10 seconds.

The output of the lotus-farcaster is written to `/data/farcaster.prom` inside the container which should be a bind mount in prometheus node exporter path.

Optional : This could be use in conjunction with a dockerised version of prometheus node_exporter (instructions to get the node_exporter container going

Building the container (as root) 

                                        apt install docker.io
                                        docker build -t lotus-farcaster:latest -f dockerfiles/Dockerfile .

Running the container (as root)

Lotus-exporter-farcaster use 2 configuration files:

  • config.toml contains connection strings to miner,daemon and markets API. This file is mandatory.
  • addresses.toml contains address lookup that replace filecoin addresses by readable names. This file is optional.
At this point you have to create the config.toml file by copying config.toml.example and edit it : 
                                                cd lotus-exporter-farcaster
                                                cp config.toml.example config.toml
                                                nano config.toml
    
                                                # Optional
                                                cp addresses.toml.example addresses.toml
                                                nano addresses.toml
                                            

                                            export PROMETHEUS_NODE_EXPORTER_PATH="/var/lib/prometheus/node-exporter/"
    
                                            docker run --name lotus-farcaster -d \
                                            --mount type=bind,source=$PROMETHEUS_NODE_EXPORTER_PATH,target=/data \
                                            --network=host \
                                            lotus-farcaster

Docker Debug (as root) 

                                            docker ps [-a]
                                            docker logs lotus-farcaster
                                            docker exec -it lotus-farcaster bash

Uninstall docker (as root) 

                                            docker stop lotus-farcaster
                                            docker rm lotus-farcaster
                                            docker image rm lotus-farcaster

Install the Grafana Dashboard

Import in Grafana the relevent dashboard file from ./lotus-farcaster/grafana-dashboard

Contributing

If you're interested in contributing to the Grafana project:<

  • Translate : help us by translating our dashboard to other languages.
  • Panels : submit your customized panels
  • Donate : Filcoin Address : f3v3lj5jrsvv3nwmsvvj57yyty6ndb27oyi4yaqhwzst3emdv25hefna6vxhtpjb5pytwahdod67syxjyzba3q

Sponsors

Protcol Labs